Skip to content

Commit 7b17e4b

Browse files
authored
Merge pull request #170 from manikanta-hitunik-com/patch-956353
Update 487-building-rust-extensions-for-python.txt
2 parents 06be6ef + e42129f commit 7b17e4b

File tree

1 file changed

+27
-28
lines changed

1 file changed

+27
-28
lines changed

transcripts/487-building-rust-extensions-for-python.txt

Lines changed: 27 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -140,7 +140,7 @@
140140

141141
00:03:57 So I started Pydantic back in 2017 as a side project.
142142

143-
00:04:01 And I toodled along.
143+
00:04:01 And I toddled along.
144144

145145
00:04:02 And then 2021, it somehow, something happened.
146146

@@ -340,7 +340,7 @@
340340

341341
00:09:55 These banks to have enough money to, you know, support open source.
342342

343-
00:09:58 We're just trying to persuade them to use Lockfire right now.
343+
00:09:58 We're just trying to persuade them to use Logfire right now.
344344

345345
00:10:00 So I'll stop being rude about banks.
346346

@@ -352,41 +352,41 @@
352352

353353
00:10:06 To get those folks to open their checkbooks and pay for it.
354354

355-
00:10:09 Right. They they don't have charity usually in their bylines or whatever they're supposed to be doing, right?
355+
00:10:09 Right. They don't have charity usually in their bylines or whatever they're supposed to be doing, right?
356356

357-
00:10:15 Like they but with something like Lockfire, they can say, well, here's a service that we can use.
357+
00:10:15 Like they but with something like Logfire, they can say, well, here's a service that we can use.
358358

359359
00:10:20 And by using it, we might be able to support Pydantic and the other things that are important to us.
360360

361361
00:10:25 So I think that's great.
362362

363-
00:10:27 You know, tell people just real quickly about Lockfire out there.
363+
00:10:27 You know, tell people just real quickly about Logfire out there.
364364

365365
00:10:30 Yeah, absolutely.
366366

367-
00:10:31 So Lockfire is an observability platform built, obviously, by the team behind Pydantic.
367+
00:10:31 So Logfire is an observability platform built, obviously, by the team behind Pydantic.
368368

369-
00:10:36 And if you go to Lockfire along the along the top, the I suppose the two things that make it different from some of the stuff that's come before is Lockfire is built on open telemetry.
369+
00:10:36 And if you go to Logfire along the along the top, the I suppose the two things that make it different from some of the stuff that's come before is Logfire is built on open telemetry.
370370

371371
00:10:45 So on open standards that mean that, yeah, the rails of that where the data is being transferred are on an open standard.
372372

373-
00:10:50 And if you decide you didn't want to use the Lockfire platform anymore, you can send that data to anything else that supports open telemetry.
373+
00:10:50 And if you decide you didn't want to use the Logfire platform anymore, you can send that data to anything else that supports open telemetry.
374374

375-
00:10:56 But unlike lots of other companies in our space, instead of using open telemetry as an excuse to abandon the SDK space and just say use the horrible open telemetry SDK directly, we have the Lockfire package, which tries to make that super nice and easy to use.
375+
00:10:56 But unlike lots of other companies in our space, instead of using open telemetry as an excuse to abandon the SDK space and just say use the horrible open telemetry SDK directly, we have the Logfire package, which tries to make that super nice and easy to use.
376376

377377
00:11:10 And then the other big change we have, I think there'll be an example maybe further on down the page.
378378

379-
00:11:15 Like we maybe maybe there isn't one right there, but we allow you to write SQL directly against your data.
379+
00:11:15 Like we maybe there isn't one right there, but we allow you to write SQL directly against your data.
380380

381381
00:11:20 So instead of having to use kind of ClickOps to go around and do surveys.
382382

383383
00:11:25 So if you look here, right, you can go and like write SQL to query your data.
384384

385-
00:11:29 And so there are although we're still early, there are things you can do in Lockfire that you can't you cannot and never been able to do in like one of the big incumbents like Datadog because it's just SQL.
385+
00:11:29 And so there are although we're still early, there are things you can do in Logfire that you can't you cannot and never been able to do in like one of the big incumbents like Datadog because it's just SQL.
386386

387387
00:11:38 And it's obviously much easier to learn for you, much easier for LLMs to write.
388388

389-
00:11:43 So we have the like plausible chance in the future that you could basically chat with Lockfire and say, what's wrong with my app?
389+
00:11:43 So we have the like plausible chance in the future that you could basically chat with Logfire and say, what's wrong with my app?
390390

391391
00:11:49 And the agent can go off and run SQL to investigate things in a way that is much harder if you have your own dialect.
392392

@@ -532,7 +532,7 @@
532532

533533
00:16:52 Is it just a Rust level thing?
534534

535-
00:16:53 If you go up actually, it's a, it's a, this is a monorepa.
535+
00:16:53 If you go up actually, it's a, it's a, this is a monorepo.
536536

537537
00:16:56 So if you go up into source, into crates, which is, so this is, you have a bunch of different crates in here.
538538

@@ -608,7 +608,7 @@
608608

609609
00:19:32 I've been surprised by how, like, the legs that this has had, which I wasn't expecting when we first started it.
610610

611-
00:19:37 But yeah, the nice bit is it's all the input, the actual JSON parsing is Rust, but then we have the logic to, yeah, basically access that from Python, both in this package and in Pylandic core.
611+
00:19:37 But yeah, the nice bit is it's all the input, the actual JSON parsing is Rust, but then we have the logic to, yeah, basically access that from Python, both in this package and in Pydantic core.
612612

613613
00:19:47 Well, you sent me over to this cargo section, or the crate section, rather.
614614

@@ -702,7 +702,7 @@
702702

703703
00:23:03 I'm not clever enough to write C.
704704

705-
00:23:04 That's a void pointer pointer.
705+
00:23:04 That's a void pointer.
706706

707707
00:23:08 What are we doing here?
708708

@@ -818,7 +818,7 @@
818818

819819
00:27:26 We don't really think about compiling much in Python.
820820

821-
00:27:28 We don't think about it, but you can think of import time when you do different stuff or you think about static typing when you go and run PyWrite over your code.
821+
00:27:28 We don't think about it, but you can think of import time when you do different stuff or you think about static typing when you go and run PyWright over your code.
822822

823823
00:27:37 And that is very similar in some ways to compile time.
824824

@@ -844,7 +844,7 @@
844844

845845
00:28:30 I mean, putting the controversy over Surdi to one side for a minute, you would have to take care of compiling that code yourself every time you wanted to go and go and use it.
846846

847-
00:28:38 Whereas ActionPyPI does an amazing job of distributing compiled Rust for virtually every ecosystem, every architecture, and it just working when you install it.
847+
00:28:38 Whereas Action PyPI does an amazing job of distributing compiled Rust for virtually every ecosystem, every architecture, and it just working when you install it.
848848

849849
00:28:46 Absolutely.
850850

@@ -1280,7 +1280,7 @@
12801280

12811281
00:43:32 I don't know how long ago that was you, you were trying, but I think it's, yeah, reduced significantly.
12821282

1283-
00:43:37 So, I mean, I did this quite a lot of the same work in PyNantic Core to avoid the overhead of calling into and out of it lots of times.
1283+
00:43:37 So, I mean, I did this quite a lot of the same work in Pydantic Core to avoid the overhead of calling into and out of it lots of times.
12841284

12851285
00:43:43 And I think that's less of a thing now than it used to be.
12861286

@@ -1310,11 +1310,11 @@
13101310

13111311
00:44:46 I don't know if that's in here as well.
13121312

1313-
00:44:48 I maintain watch files, which is the file watching library used by UV-icorn and some other things.
1313+
00:44:48 I maintain watch files, which is the file watching library used by UV-con and some other things.
13141314

13151315
00:44:55 And that is, again, wrapping a Rust library for getting fast system notifications.
13161316

1317-
00:45:00 And also R-toml, which is the fastest tomml parser in Python, which is, again, just wrapping the Rust library.
1317+
00:45:00 And also R-toml, which is the fastest toml parser in Python, which is, again, just wrapping the Rust library.
13181318

13191319
00:45:06 So there are a number of places where you can get enormous performance improvements and indeed like fundamentally more reliable libraries because you're building in Rust.
13201320

@@ -1414,7 +1414,7 @@
14141414

14151415
00:47:55 But RustUp is kind of like one level up.
14161416

1417-
00:47:58 And it's maybe a bit like PyM if people have used that.
1417+
00:47:58 And it's maybe a bit like PyPI if people have used that.
14181418

14191419
00:48:02 But it sort of does a bit more than that.
14201420

@@ -1426,7 +1426,7 @@
14261426

14271427
00:48:30 What you'll do, your first step is to go to the Rust website and install RustUp.
14281428

1429-
00:48:35 And then hopefully at the command line, you'll be able to type RustC and then you can compile a Rust file if you want to.
1429+
00:48:35 And then hopefully at the command line, you'll be able to type Rust C and then you can compile a Rust file if you want to.
14301430

14311431
00:48:40 But you don't actually need to do that because that's what you use Cargo for.
14321432

@@ -1508,7 +1508,7 @@
15081508

15091509
00:51:57 Maturin effectively gets rid of that whole challenge for you and just works and lets you work on it like you're, it's as easy as writing Python or Rust interacting with the two.
15101510

1511-
00:52:07 Yeah, because cargo and Rust C will output Rust conventions, but Python wants C extension conventions, naming like .so versus Dynelib and things like that, right?
1511+
00:52:07 Yeah, because cargo and Rust C will output Rust conventions, but Python wants C extension conventions, naming like .so versus Dynamo lib and things like that, right?
15121512

15131513
00:52:18 And getting the inputs right and yeah.
15141514

@@ -1536,7 +1536,7 @@
15361536

15371537
00:53:26 The problem you then have is you end up with two separate definitions of what your functions have in them.
15381538

1539-
00:53:31 And so there's a really neat tool, which I didn't know about until quite recently, but we use it in Pydantic Core, which mypy can basically type check that the PYI file stubs match the definitions inside the DLL.
1539+
00:53:31 And so there's a really neat tool, which I didn't know about until quite recently, but we use it in Pydantic Core, which MyPy can basically type check that the PYI file stubs match the definitions inside the DLL.
15401540

15411541
00:53:44 And so you can have like guarantees that where you've written in a PYI file, my function foobar takes A as a string and C as a list of bytes.
15421542

@@ -1548,7 +1548,7 @@
15481548

15491549
00:53:56 Fixes all together.
15501550

1551-
00:53:57 Is it a mypy plugin?
1551+
00:53:57 Is it a MyPy plugin?
15521552

15531553
00:53:58 If you have a look at Pydantic Core, we run it in CI or in maybe even pre-commit so you can find it there.
15541554

@@ -1722,7 +1722,7 @@
17221722

17231723
00:58:24 But I think if you're starting from scratch and you're trying to write something high performance, the experience of doing it in Rust is completely different.
17241724

1725-
00:58:30 from trying to do it in C or C++ or C Sharp or any of the other or Fortran or Julia or any of those other languages.
1725+
00:58:30 from trying to do it in C or C++ or C# or any of the other or Fortran or Julia or any of those other languages.
17261726

17271727
00:58:35 Rust is awesome.
17281728

@@ -1844,7 +1844,7 @@
18441844

18451845
01:01:34 We should be right at the top.
18461846

1847-
01:01:36 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm.
1847+
01:01:36 You can also find the iTunes feed at /iTunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm.
18481848

18491849
01:01:45 We're live streaming most of our recordings these days.
18501850

@@ -1859,4 +1859,3 @@
18591859
01:02:00 Now get out there and write some Python code.
18601860

18611861
01:02:02 I'll see you next time.
1862-

0 commit comments

Comments
 (0)