From 1fe2373793dd7c3546aa04032882382b15847f52 Mon Sep 17 00:00:00 2001 From: Michael Kennedy Date: Mon, 11 Aug 2025 13:42:11 -0700 Subject: [PATCH 1/5] transcripts --- transcripts/515-temporal.txt | 2280 ++++++++++++++++++++ transcripts/515-temporal.vtt | 3863 ++++++++++++++++++++++++++++++++++ 2 files changed, 6143 insertions(+) create mode 100644 transcripts/515-temporal.txt create mode 100644 transcripts/515-temporal.vtt diff --git a/transcripts/515-temporal.txt b/transcripts/515-temporal.txt new file mode 100644 index 0000000..b9f9515 --- /dev/null +++ b/transcripts/515-temporal.txt @@ -0,0 +1,2280 @@ +00:00:00 What if your code was crash-proof? That's the value prop for a framework called Temporal. + +00:00:05 Temporal is a durable execution platform that enables developers to build + +00:00:09 scalable applications without sacrificing productivity or reliability. + +00:00:14 The Temporal server executes units of application logic called workflows in a resilient manner that + +00:00:19 automatically handles intermittent failures and retries failed operations. We have Mason Egger + +00:00:25 from Temporal on to dive into durable execution in Python. This is Talk Python To Me, episode 515, + +00:00:32 recorded June 19th, 2025. Welcome to Talk Python To Me, a weekly podcast on Python. + +00:00:54 This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the + +00:00:59 podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and + +00:01:05 listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, + +00:01:11 you can find the live streams over on YouTube. Subscribe to our YouTube channel over at + +00:01:15 talkpython.fm/youtube and get notified about upcoming shows. This episode is sponsored by + +00:01:21 Posit Connect from the makers of Shiny. Publish, share, and deploy all of your data projects that + +00:01:27 you're creating using Python. Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, + +00:01:33 Dashboards, and APIs. Posit Connect supports all of them. Try Posit Connect for free by going to + +00:01:39 talkpython.fm/posit, P-O-S-I-T. The PyBay Conference is returning to the UCSF Mission Bay + +00:01:46 Conference Center in San Francisco, California on October 18th, 2025. Get your ticket and pick up a + +00:01:53 free conference course bundle from Talk Python. Get started at talkpython.fm/PyBay. + +00:02:00 Big announcement, we have a brand new course for you all data science types out there. + +00:02:04 Just Enough Python for Data Scientists. Data scientists get things done in notebooks, + +00:02:09 but production quality work needs more than ad hoc scripts. Just Enough Python for Data + +00:02:14 scientist gives you the essential Python and software engineering habits to level up your + +00:02:20 analysis without drowning in theory. In a few focused hours, you'll tighten up your core Python, + +00:02:25 write clean and reasonable functions, organize code into importable modules, track work with + +00:02:30 Git and GitHub, debug confidently, and make your results reproducible with pinned environments and + +00:02:36 Docker. You'll also see how modern agentic AI tools can accelerate data exploration, bug detection, + +00:02:43 refactoring, and documentation. The outcome is simple. You keep your notebook speed while gaining + +00:02:48 the reliability, collaboration, and professionalism your projects deserve. Just visit talkpython.fm + +00:02:55 and click on courses in the nav bar. The link to the course is also in your podcast player show + +00:02:59 notes. And if you'd rather focus purely on building with LLMs, check out Vincent Warmerdam's + +00:03:05 LLM building blocks for Python course we just recently released as well. Now on to that interview. + +00:03:11 Mason, welcome to Talk Python To Me. Fantastic to have you here. + +00:03:14 It's great to be here. Long time listener. + +00:03:16 Oh, wonderful. I'm definitely a fan of stuff I've seen you doing online as well. + +00:03:20 And it's super cool to get together here and, you know, share it with a couple of people. + +00:03:24 Yeah, definitely excited. + +00:03:26 Temporal. Wow, what a cool topic. Durable execution. + +00:03:29 What a neat idea that I've seen in other places, but I've seen less of it in Python. + +00:03:34 So I'm real excited to dive into this. + +00:03:35 This is something I learned about recently. + +00:03:37 We'll go into that in a bit, but there's a lot here. + +00:03:41 Let's just leave it like this could be a two hour show and we'd still be going easy. + +00:03:45 Definitely. + +00:03:46 Yeah. + +00:03:46 I spend a lot of my time educating people about durable execution. + +00:03:49 that's, that's what my role is. + +00:03:51 I'm a develop, one of the developer educators at temporal. + +00:03:53 and definitely you could spend hours on this and we would, we could be going forever. + +00:03:58 So yeah. + +00:03:59 Yeah. + +00:03:59 And we definitely could. + +00:04:00 Well, before we do go on for hours, who are you? + +00:04:03 It was Mason. + +00:04:04 Oh yeah. + +00:04:04 my name is Mason Egger. + +00:04:06 I am a developer educator at temporal, as I mentioned. + +00:04:09 And I also help run the PyTexas conference. + +00:04:13 So PyTexas is one of the oldest, actually the oldest regional Python conference in the + +00:04:18 world that we know of. + +00:04:20 No one else has come to claim that spot yet. + +00:04:22 We started in the late fall of 2007, like right after PyCon US would have started. + +00:04:28 We'll be experiencing our 20th year this upcoming year, which is exciting. + +00:04:31 And I also am the president of the PyTexas Foundation. + +00:04:34 So PyTexas has its own 501c3 organization that is used to basically act as a sheltering organization for all of the Python meetups and events that go on within the state of Texas, which is really nice. + +00:04:46 So I was elected president of that back in 2002. + +00:04:50 I've been helping run and build the Python community there ever since. + +00:04:53 And it's a lot of work, but it's a lot of fun. + +00:04:55 And I really enjoy my community work. + +00:04:57 Yeah, it sounds extremely fun, although also challenging. + +00:05:00 You say for the entire state of Texas, Texas is a big place. + +00:05:05 It's like countries as well, if you look at the size of Texas. + +00:05:09 It really is. + +00:05:09 Texas has some unique issues with it when it comes to running community stuff. + +00:05:17 We founded our own virtual meetup for basically anyone in Texas, and anyone else can join, + +00:05:22 obviously, too. + +00:05:22 We don't say, oh, no, no Texans can join it. + +00:05:25 But we did it because the amount of feedback we would get, it's like, oh, I live in, say, the DFW area, the Dallas-Fort Worth area, and I would love to go to the meetup, but I live two hours away from the meetup because of how large the Texas cities are. + +00:05:37 Texas cities are sprawling metroplexes. + +00:05:40 I grew up in the Houston area, and the joke when you grew up in Houston is you're always three hours away by car from every other part of Houston. + +00:05:48 And that really is the case. + +00:05:49 So it does have its own set of unique challenges, trying to coordinate four major cities across, + +00:05:55 you know, technically two time zones. + +00:05:57 If we can, if we include El Paso, the very, very edge, the very west part of Texas is + +00:06:03 in the mountain time zone. + +00:06:05 So it's an interesting bit of work. + +00:06:07 And we've been doing a really good job of expanding and like adding more and more offerings + +00:06:11 that we can offer to the community year over year. + +00:06:13 And it's been a lot of fun and I really enjoy it. + +00:06:16 It keeps me busy. + +00:06:17 That's for sure. + +00:06:17 Yeah, I could say. + +00:06:19 is that why people drive so fast in Texas? They've got so far to go. + +00:06:23 Exactly. Yes. I mean, the fastest speed limit in the United States is on the highway next to my + +00:06:28 house. It's 85 miles an hour and it's on the toll road next to my house because you're right. Like + +00:06:32 if you're trying to get from Austin to San Antonio, that's, I think it's about a hundred miles + +00:06:37 down. And like, if you're going 55, you'll get there in, you know, two and a half hours, + +00:06:41 but I can make it in an hour and a half hour and 15 minutes. If you just let me fly, + +00:06:45 I had no idea that the speed limit were that high. + +00:06:49 I knew they were high, but not that high in Texas. + +00:06:51 Yep. + +00:06:52 Wild, you know. + +00:06:53 You guys in Germany, come on. + +00:06:55 Yeah, it is the Southern Autobahn, basically. + +00:06:59 Fantastic. + +00:07:00 So you've had a bit of a tougher challenge, I suppose, or more challenges than a lot of people with this whole Pi Texas thing, + +00:07:07 which I've always thought was super cool. + +00:07:09 I'd love to go sometime. + +00:07:10 It's far from Oregon, but still would be really lovely. + +00:07:13 That said, you started working on this COVID time, right? + +00:07:17 Which for conferences is the kiss of death. + +00:07:19 You want to talk us through how that went? + +00:07:21 Yeah, I could definitely talk about that for a while. + +00:07:24 So I got involved in conference organizing, as most people do, by first speaking at the + +00:07:29 conference. + +00:07:30 So I spoke at the conference in 2019. + +00:07:32 I had been a PyCon attendee prior to that. + +00:07:35 My first PyCon attendance was the last Portland one, which I believe was 2017. + +00:07:42 and then spoke at Pitexas in 2019 and volunteered to help for the 2020 conference. + +00:07:47 And that was really interesting because, you know, we had planned to do it in person. + +00:07:50 I think everybody had planned, was planning in person 2020, you know, because you, for + +00:07:55 those that don't know, when you start planning a conference, you start planning about eight, + +00:07:59 10 months prior. + +00:08:00 So if you're a spring conference as Pitexas is, we were planning, you know, Pitexas 2020 + +00:08:05 in about summer of 2019. + +00:08:07 No one knew what was coming. + +00:08:09 We were so innocent then. + +00:08:11 Yes. + +00:08:12 We really were. So we were planning that. And then, you know, the events that we know of happened + +00:08:18 and we kept chasing it. We kept pushing it back. I think a lot of conferences at that time kept + +00:08:21 pushing back thinking, Oh, we'll be out of this in two weeks. We'll be out of this in three months. + +00:08:26 We ended up going virtual. And then in 2021, we made the conscious decision to just not have a + +00:08:30 conference. The virtual conference for us in 2020 wasn't overly successful. And we've, at that time, + +00:08:36 we were feeling there was just a lot of virtual conference fatigue and we didn't have a really + +00:08:39 good story on how to like make it work and dedicate the time. Everybody was also struggling + +00:08:48 to handle the world at that time. So being able to put more resources into it was difficult. + +00:08:53 So we pushed that off and then we came back in 2022. We made a very conscious decision about it. + +00:08:58 Like we were like, we're going to come back in the safest way possible. PyCon US had announced + +00:09:01 they were coming back. We decided we were going to come back. We had, we have a venue that has + +00:09:05 custom air filtration. We instilled mask mandates and vaccine mandates and all of that stuff. + +00:09:10 And we had 76 people return. But we knew that if we didn't come back, if we let this kind of like + +00:09:16 continue on, that the likelihood of getting the community to come back, that the memory + +00:09:20 is short, that if we didn't come back, we might lose it forever. And having run this for 18 years, + +00:09:26 we were concerned about losing it at that point. I think that's totally valid. A lot of this + +00:09:30 conference attending is it's habit. You form, you've gone a bunch of times. You really enjoy + +00:09:36 it. Like, yeah, of course I'm going to go. You don't even question like, well, when is it? I'm + +00:09:38 just going to sign up when it comes out. But you know, if you stop going, then I think that does + +00:09:43 make a big change. So it definitely does. So in 22, we, we did it and then we kept coming back. + +00:09:48 And then, you know, every year prior after that, we, we continue to grow. it took us three + +00:09:53 years to get back to above, what, what I would consider pre-pandemic normals. and the, + +00:09:59 the fun fact about that is, and the thing that I think that really kind of helped us out is like, + +00:10:03 we didn't really start seeing the growth back in the conference attendance until us as the + +00:10:08 foundation turned our inwards local or sorry, our vision local or our site to start focusing on the + +00:10:15 local meetups, because the local meetup scene had not returned. And that was the vast majority of + +00:10:19 our of our of our marketing was that we would send out these emails and stuff or the local meetups + +00:10:24 would promote us. And these are huge meetups. I mean, the Python, the some of the Python meetups + +00:10:29 are some of the oldest meetups that I know of. I mean, like they started in the early, + +00:10:33 in the late aughts in the early teens. Yeah, I would say they probably were not even called + +00:10:38 meetups when they started, right? The user groups or something. They predate meetup. Yeah, + +00:10:43 meetup.com and all that. Yeah, I think the PyHouston meetup group is called, + +00:10:47 like their tag on meetup.com is Python-14. So I'm assuming it was the 14th Python meetup when it was + +00:10:55 created in the world on meetup.com. so, and you know, large groups, but these meetups had + +00:11:01 gone dormant. So what I did as, as foundation president at that time was I was like, okay, + +00:11:05 I'm going to spend all of my effort while my organizers are helping get the conference going. + +00:11:09 I'm going to spend all of my effort finding organizers and rebuilding these meetups in + +00:11:12 these areas. so I would connect with people that I knew in the area. I would reach out to + +00:11:16 friends. I would put out all calls and after time of rebuilding the ecosystem, then we basically + +00:11:22 everything came back to life. And that's kind of where a little, you know, the things that I have + +00:11:25 learned is that if your, if your meetup ecosystem is not healthy, then your regional Python conference + +00:11:30 ecosystem will not be healthy because it has to feed up into it. Yeah. Yeah. I think it's, + +00:11:36 it's such a challenge to run like a conference or podcast or whatever to draw in new people. + +00:11:43 I mean, you can do all the amazing mailing lists and announcements and everything, + +00:11:47 but you're speaking to your existing people that know about you and finding new ones is really + +00:11:51 interesting and this cultivating like user groups to sort of be the foundation. It's a very + +00:11:56 interesting idea. Definitely. User groups work. I think we also started the PyTexas meetup, + +00:12:01 which was a virtual meetup that we do monthly that we allow anyone to join. And we cultivated + +00:12:05 amongst our network of friends and social media and word of mouth. And then we all know what + +00:12:10 happened with the social media situation, which really did not help. I mean, there was a lot of + +00:12:14 just a lot of things that have happened with like the dissolution of Twitter really did not help + +00:12:20 the conference and the tech ecosystem. Basically, we've all fractured now. We're all in like six + +00:12:24 different places. Like the move to Mastodon is not complete. Blue Sky had its moment, but I honestly + +00:12:30 get less engagement on Blue Sky than anything else. LinkedIn has surprisingly been our most successful + +00:12:36 social media platform. It seems like a lot of people have moved there for some reason or another. + +00:12:41 But basically, it means that you just have to reapply your marketing strategies. And the fun + +00:12:46 thing that I've had the benefit of is that as my work as a developer advocate, all the roles that + +00:12:51 I have done, they tend to sit in marketing. Developer advocacy tends to either sit in + +00:12:55 product or marketing. All the roles that I have taken sit in marketing. And I've had the benefit + +00:12:59 of like, whenever this has started going weird, I can ask all of my marketing friends, hey, what + +00:13:04 would you do in this situation? How would you approach this? So I've got to learn through + +00:13:08 osmosis kind of how marketing stuff works and being able to apply that to conference organizing + +00:13:12 and meetup organizing has actually been substantially beneficial to us. + +00:13:16 Yeah. + +00:13:17 Certainly, I think companies that are turned on enough to have developer advocates have + +00:13:22 developer intelligent marketing teams, and that's a real resource. + +00:13:25 Definitely. + +00:13:26 Yeah. + +00:13:26 It's been really useful to be able to get other people's opinions. + +00:13:29 And then, you know, just ask other conference organizers, what are they doing? + +00:13:32 I think that, you know, finding out what works and telling other people about it. + +00:13:36 I mean, we, I haven't had a chance to write this year's blog post yet, unfortunately, + +00:13:40 for this year's conference. + +00:13:41 But whenever I became conference chair and then president, I was like, we're going to be as transparent as possible. + +00:13:47 Every misstep we make, we're going to blog about it. + +00:13:50 We talk about our budget. + +00:13:51 I mean, we're also a 501c3, so that's kind of like part of our bylaws. + +00:13:54 But at the same time, it's like this is everything that we do. + +00:13:57 You can find it on all of our websites. + +00:13:59 And this is what worked. + +00:14:00 This is what didn't. + +00:14:00 Because there's so many first-time conference organizers who may want to start a conference who don't know how to achieve these goals or whatever they're trying to do. + +00:14:08 And we have 20 years of experience doing it. + +00:14:11 And like it, we need to help each other out and make sure that we distill this, this knowledge + +00:14:15 outward. + +00:14:16 I don't have 20 years of experience. + +00:14:17 So I'm only doing it for four, but you know, institutional knowledge. + +00:14:20 There are, there are Google docs with like years of back data that I can go and look + +00:14:24 at and be like, oh yeah, in 2009, we ordered way too many meals because we, you know, didn't + +00:14:30 charge enough for, we didn't charge anything for tickets. + +00:14:32 Like PipeSex used to be a free conference. + +00:14:34 And basically when you don't charge anything for tickets, one of the lessons that we learned + +00:14:37 is that people will just sign up for a ticket and then not show up. + +00:14:40 And then your catering gets all kind of out of whack. + +00:14:42 So even charging just a little bit of money, like five bucks, I think we charge like five dollars for our network event. + +00:14:47 It's not because I need the five dollars. + +00:14:48 I mean, I've told I've told my we spend, I think, thirty dollars per person on the food. + +00:14:52 It's like it's to make sure that you have a little bit of like skin in the game + +00:14:55 to make sure that you show up so I can get an accurate headcount for the for the for the catering. + +00:15:00 So we don't blow the budget by ten thousand dollars. + +00:15:02 Like I'm not envious of that. + +00:15:04 I certainly it's easy to just check a checkbox. + +00:15:07 Yeah, I'm interested in food. + +00:15:07 Why not? + +00:15:08 Yeah. + +00:15:09 If I come. + +00:15:10 Yeah. + +00:15:10 Exactly. + +00:15:13 This portion of Talk Python To Me is brought to you by the folks at Posit. + +00:15:16 Posit has made a huge investment in the Python community lately. + +00:15:20 Known originally for RStudio, they've been building out a suite of tools and services for Team Python. + +00:15:26 Today, I want to focus on hosting your Python-based data science workloads. + +00:15:31 This includes dashboards, reports, plots, interactive web apps, all the way to custom Flask and Django apps. + +00:15:38 Their service is Posit Connect. + +00:15:40 Posit Connect makes it easy for data scientists to share work built with Python code. + +00:15:46 If you have a streamlet app, Dash, dashboard, Plotly interactive plot, a FastAPI service, or even a Quarto report, + +00:15:53 just give Posit Connect the code it needs to maintain the asset and Connect automatically does the rest. + +00:15:59 Connect will manage your APIs and serve your interactive apps for you. + +00:16:03 And if you want, you can update your reports and dashboards on a scheduled basis. + +00:16:07 That's right. + +00:16:08 No need to explain to the stakeholders why that dashboard or plot stopped updating last week. + +00:16:14 You get a focus on your data science and leveraging your skill set while Connect makes you look good, + +00:16:19 keeping your code running and private. + +00:16:21 With Connect, you get a private URL on your Connect server, ensuring that your asset is continuously available to your shareholders. + +00:16:29 And you can control which users have access to the asset. + +00:16:32 Let Posit Connect handle the delivery and DevOps involved in sharing your work. + +00:16:37 You focus on what you do best. + +00:16:39 So if you work on a data science team, you owe it to you and your org to check out Posit Connect. + +00:16:45 Visit talkpython.fm/connect today and get a three-month free trial to see if it's a good fit. + +00:16:51 That's talkpython.fm/connect. + +00:16:54 The link is in your podcast player's show notes. + +00:16:56 Thank you to Posit for supporting Talk Python To Me. + +00:16:59 out in the audience. + +00:17:01 Toon Army says, yay, PyTexas Foundation. + +00:17:03 That's pretty awesome. + +00:17:04 Yeah. + +00:17:05 All right. + +00:17:05 Well, let's talk. + +00:17:07 Emporal. + +00:17:08 Now, I came across this a few weeks ago from this, you know, I think this, + +00:17:12 ironically, I think this might be the blue sky that I came across it on. + +00:17:15 Interesting. + +00:17:17 No. + +00:17:18 Okay. + +00:17:18 It was on X. + +00:17:19 There's still apparently stuff that happens on there. + +00:17:22 I miss Twitter. + +00:17:23 I do too. + +00:17:23 And, you know, people like post weird comments on, like reviews like oh you know they said they don't like twitter so they must be whatever like no just that used to + +00:17:35 be so active and it's not so active like put put aside everything else it used to be like you could + +00:17:39 have great conversation you still can but it's so much less than it used to it's an easy it's an easy + +00:17:44 explanation the signal to noise ratio is completely messed up now yes it used to be such a much better + +00:17:49 place because everybody was posting there and there wasn't as much noise and now the signal to noise + +00:17:52 ratio makes it almost unintelligible to be able to find anything of any use and that's the number + +00:17:57 one problem with it. And you pay for attention and there's six other places like you said. So anyway, + +00:18:01 so I found this post from, we'll work it backwards, from Pietro. It says, people aren't talking enough + +00:18:08 about how most of OpenAI's tech stack runs on Python, which I thought was a super cool post. + +00:18:14 It comes from the Pragmatic Engineer newsletter and it talks about how most of the product's code + +00:18:20 is written in Python, uses FastAPI, C for certain parts. And then, so all that stuff made, I'm like, + +00:18:26 yep, yep. Temporal, use for asynchronous workflows. I'm like, what is Temporal? And then I think you + +00:18:32 sent me a message and said, hey, I would love to talk to you about Temporal. And I looked at him, + +00:18:36 yeah, this is super cool. So that's how I learned about you guys, which I thought was pretty neat. + +00:18:42 And yeah, let's talk about it. What is Temporal? Yeah. So Temporal is essentially what we call a + +00:18:51 durable execution platform. And it's durable execution is kind of a new term or a new like, + +00:18:57 like field within the zeitgeist, as you would call it. And we're kind of going with like what + +00:19:03 we're calling crash proof execution. And the way that we kind of talk about it now is like, + +00:19:08 essentially, it handles the failure state of your of your applications and ensures that your code + +00:19:13 will continue execution regardless of the failures that it encounters. So say, for example, you have + +00:19:20 a application that's making a couple calls out to a microservice. Okay. And that microservice goes + +00:19:25 temporarily down. You as the developer would have to traditionally write a whole bunch of logic + +00:19:31 around handling that failure. Okay. So we have to detect that the failure has happened. + +00:19:35 What type of error did we receive? Now, what do we do? Do we back off and retry? Do we decide that + +00:19:41 this failure is non-retriable and we just don't do it? Like the difference between we're not + +00:19:44 authenticated versus a 404, those are completely different failure handling modes. And then, + +00:19:49 So there's all of this kind of logic that you have to build in to handle failure. + +00:19:53 Temporal basically abstracts this away from you. + +00:19:56 So whenever you have like a function call or a method call in Python, when implemented with Temporal, you automatically get like retries, for example, by default. + +00:20:04 So you get this like declarative policy that happens as a default policy, + +00:20:08 and it will automatically retry your application or your call until eventually it comes back online. + +00:20:14 Because let's be honest, in most distributed systems like this, those are most of the time intermittent failures. + +00:20:19 Like a microservice going offline, are you being rate limited? + +00:20:22 Those are usually fixable with a little bit of time and some retries. + +00:20:27 Now, there are other cases where they're not, but that's like the default policy. + +00:20:30 That's the default use case or the default for retries alone. + +00:20:34 And then there's a lot of other cases. + +00:20:36 So Temporal maintains the application state of your application. + +00:20:41 And so say you have like a Python application that has 10 steps in it and you get through + +00:20:46 step five and then the python application were to crash for some reason out of memory uh maybe + +00:20:51 your kubernetes pod got descheduled something like this and production happens happens all the time + +00:20:55 this is as a former sre this was this was the life that i lived for years um yeah even hey + +00:21:02 there's a security patch for the Linux server that's the yeah running this and we got to restart + +00:21:07 it and like uh we have to reboot yeah and now yes it's not wrong but like stuff is going on exactly + +00:21:14 So now when you have to deal with that, you're like, okay, so do we let everything get through and risk the attack surface? + +00:21:19 Or then you have to make the calculus. + +00:21:21 Do we just kill all the processes and then wait for, and then restart them? + +00:21:26 What's the cost on the restart? + +00:21:28 And then redoing all of that logic. + +00:21:29 Did they make any rights to the database? + +00:21:31 Do we have to worry about cleaning all that? + +00:21:33 There's all of this conversation. + +00:21:35 Since Temporal maintains that application state, that becomes a lot simpler. + +00:21:39 Because as we, I think we kind of alluded to it, but I'll state it outright. + +00:21:42 when that when you when that application crashes, and in a typical application, you have to start + +00:21:47 back over from the beginning. And you have to replay every you basically re execute everything + +00:21:51 you did unless you were have some like advanced sort of like event sourcing style model where you + +00:21:56 are keeping track of it. Temporal does this by default. So it maintains the event history and + +00:22:01 what's called an event history of your application. And what that does is every time a basically + +00:22:06 creates checkpoints, every time a function executes, it stores the result of it. And if that application + +00:22:10 were to crash, it will reschedule it onto another worker. That's what they're called in temporal. + +00:22:15 And we'll get into that in a minute within your fleet, re reconstruct the state of that application + +00:22:20 up into that point of failure, and then continue onward as if the failure had never happened. + +00:22:24 And more often than not, this happens without the developer even being aware of it. Like no, + +00:22:29 no alarms will go off unless you personally do them, because this is the design of the system. + +00:22:33 It's to ensure that in spite of failure, your application will continue executing. + +00:22:37 Yeah, that's awesome. And there's a bunch of neat ways in which that happens, which we'll talk about. + +00:22:42 But I think one mental model people should maybe think about when they consider these is it centralizes a lot of the error handling and the retry code and all of that stuff. + +00:22:54 So a lot of your application becomes simpler, not more complex. + +00:22:58 Exactly. Yes. That's actually one of the main feedback things that we get from people is, you know, all of this. + +00:23:04 There's so many mechanisms. + +00:23:05 There's so many actually like patterns that we have now built to handle all of these kind of things. + +00:23:11 Event sourcing, event-driven architecture, saga pattern, all of these different like CQRS, + +00:23:15 like all of these different distributed systems patterns that exist to handle all of these things. + +00:23:20 And Temporal basically abstracts away a lot of them and you get them basically for free + +00:23:24 out of the box with the project. + +00:23:25 And it does make your code a lot simpler. + +00:23:27 It makes it a lot more contained so you can walk through it straight down. + +00:23:32 Basically, as like almost as like it's a workflow, you know, the core primitive of temporal is called a workflow. + +00:23:37 We don't tend to like to refer to ourself as a workflow engine, but some people will say that. + +00:23:41 And I typically don't correct them on it. + +00:23:44 If that's if that's what it takes for people to understand, that's what it is. + +00:23:46 Fine. But durable execution is the actual phrase. + +00:23:50 It's a meaty topic. It's a it's kind of hard for people to like kind of wrap their heads around it. + +00:23:54 So I'm like, let's get you to understanding it and then we'll correct the little the little tidbits here and there. + +00:23:59 Yeah, excellent. + +00:24:00 So, you know, people are probably familiar with Hennick's stamina or with tenacity, which + +00:24:07 is interesting, where you might put a decorator onto a function and say, hey, if this fails, + +00:24:12 instead of just crashing, try it again. + +00:24:15 Yeah. + +00:24:15 Maybe a few times, maybe with exponential back off. + +00:24:18 And this is kind of a single thread of execution type of form of durability a little bit, right? + +00:24:25 Where it's like this function now becomes somewhat more reliable based on like maybe the service comes back or maybe your network comes back or whatever it happened to be that caused this particular example. + +00:24:36 But with temporal, it's a lot broader, right? + +00:24:40 Like every step, it saves the state. + +00:24:45 If something goes wrong, it can basically resume. + +00:24:48 I guess maybe this resumable idea is like one of the big difference, right? + +00:24:52 It's like, let's take our reboot the server example. + +00:24:56 Let's suppose we got some service that we work on that our app depends upon. + +00:25:00 We had to reboot it. + +00:25:01 Now what? + +00:25:02 It happens. + +00:25:03 Yeah. + +00:25:03 So, I mean, yeah, you have a service that our app depends on. + +00:25:06 We had to reboot it. + +00:25:07 So if the service, so we're assuming our service is on in like an external machine and it's + +00:25:13 calling out. + +00:25:13 Yeah, for some reason. + +00:25:14 Yeah. + +00:25:14 Or maybe it's a Docker container and we rebuilt the Docker container. + +00:25:18 It takes five seconds to start. + +00:25:19 Something like that. + +00:25:20 Yeah. + +00:25:20 So our service is, you know, calling out to it. + +00:25:22 it will basically retry until that comes back online. + +00:25:24 Now, if we were to reboot our service, say our service was running and we rebooted the container that contained it, + +00:25:30 it would basically, if we're in a Kubernetes system, the Kubernetes scheduler would probably reschedule it + +00:25:36 onto another node within the pod. + +00:25:39 And then it would reconstruct it. + +00:25:41 And what happens is there's actually, the Temporal exists as a kind of orchestrator executor model, like a service worker model. + +00:25:50 So the service maintains the history. + +00:25:51 So what will happen is whenever that new service comes online, or whenever the new execution comes online, + +00:25:57 we've rebooted it, it got rescheduled, it comes online, it will stream the old history from the service + +00:26:02 and then reconstruct it, basically going step by step through it. + +00:26:06 And so like the function A that was executed, the input and output values are stored in that event history. + +00:26:13 So the output value, it's like, oh, we've successfully completed execution of function A, + +00:26:17 store that in variable foo, and then continue onward. + +00:26:20 And then we can continue reconstructing that service up until it gets to the point where it's like, okay, we have no more events in the history about what happened. + +00:26:27 So now we know we've, you know, we have reconstructed successfully. + +00:26:31 Now we continue forward and execute as if nothing had happened. + +00:26:35 Yeah. + +00:26:35 So in your non-durable execution code, you might have try, accept, do something else. + +00:26:42 You might have stuff like stamina or tenacity where you're like, okay, we're going to try this again. + +00:26:48 There's a lot of that kind of code that you write. + +00:26:51 And in this world, you could just say like top to bottom, write the happy path, which is great. + +00:26:57 And then what happens is temporal says, okay, I tried to run this, it failed. + +00:27:01 So we'll reschedule it with a back off or something along those lines, right? + +00:27:06 So you can basically not deal with a lot of this and get visibility into the errors and the state of flow + +00:27:12 by letting the orchestrator manage that, right? + +00:27:14 Yes. + +00:27:15 And I'm covering like the top 10% of all the different features. + +00:27:19 There are so many interesting other features that Temporal provides as an ecosystem. + +00:27:25 So like one of the other really neat ones is that we provide a really simple way for + +00:27:28 things to do a human in the loop interaction. + +00:27:31 So you can very easily send in what's called a signal. + +00:27:34 So basically sending in data into a running execution and then basically doing some processing + +00:27:38 on it. + +00:27:39 So you're waiting on a confirmed signal from someone like your application is doing something + +00:27:43 you're waiting on to confirm. + +00:27:44 you can send that in directly into the workflow. + +00:27:47 And then that will basically be, that's persisted again within the events, + +00:27:50 within the event history. + +00:27:52 So if it crashes after that confirmation, that confirmation is stored as well. + +00:27:57 So you have that, you have the ability to do long running schedules. + +00:28:00 So there's some cron syntax, like what are called schedules in Temporal. + +00:28:04 There's so many different features in Temporal that are just really neat + +00:28:09 and can solve a lot of the problems that you're trying to do. + +00:28:12 And scaling becomes super easy as well. + +00:28:14 So like you want to scale, just add more workers to the fleet. + +00:28:16 That's the easiest thing you can do is just add more of these workers and they basically + +00:28:21 can be executed across all of your different fleets. + +00:28:23 So the scaling story is super awesome as well. + +00:28:28 This portion of Talk Python To Me is brought to you by PyBay. + +00:28:31 PyBay is an annual conference gathering of Pythonistas put on by the Bay Area Python + +00:28:36 Association. + +00:28:37 This year is returning to the UCSF Mission Bay Conference Center in San Francisco, California. + +00:28:43 It's a one-day conference on October 18th, 2025. + +00:28:47 I've spoken there previously and had a great time attending. + +00:28:50 And there's a bonus. + +00:28:52 Talk Python is sponsoring the conference. + +00:28:54 Every attendee gets a special conference bundle of paid courses for free as a conference gift. + +00:29:00 Plus, we'll be giving away a complete set of training materials for a dev team of some lucky attendees. + +00:29:06 So if you want to connect with the Python people of San Francisco and go home with courses from Talk Python, + +00:29:12 check out PyBay. + +00:29:13 Please use our link. + +00:29:14 It's talkpython.fm/PyBay. + +00:29:16 The link is your podcast player show notes. + +00:29:19 Thanks to PyBay for supporting the show. + +00:29:21 We'll get into it. + +00:29:21 There's definitely some examples of large scale use cases, you know, high frequency use cases of these things. + +00:29:28 But going back to the timing, you know, what if your user onboarding looks more like, + +00:29:33 I guess you could be real simple. + +00:29:34 You could say in order to create a user account, I have to first have them create and enter their username and email or their email and their + +00:29:42 password. And then I'm going to send them an email and they got to take an action based on that, + +00:29:46 right? That would be pretty common, like a long running type of thing you could consider. But + +00:29:51 for some of these systems, it's like, and prove who you are and upload a document. + +00:29:57 That's a picture of your ID and somebody will look at it and go, yeah, that looks real enough. + +00:30:01 And they check a box or, you know, those kinds of much longer onboarding things, + +00:30:06 Something like that could be modeled with temporal pretty easily. + +00:30:08 It sounds like. + +00:30:09 Yeah. + +00:30:09 So long running workflows are the other, that, that was the feature I couldn't + +00:30:12 remember, which was timers. + +00:30:12 And I have no idea why that left my mind. + +00:30:14 It's one of my favorite features. + +00:30:15 long running workflows are one of the like amazing use cases of, of temporal. + +00:30:20 So because everything is maintained in this, in that state and basically crashing + +00:30:24 doesn't really matter because we can just reconstruct the state, you can have + +00:30:27 workflows or you can have your executions that can last for days, weeks, years. + +00:30:33 This is kind of what we know, we kind of known as the entity workflow pattern. + +00:30:37 So essentially like a user who's going through an onboarding process, you know, like you just + +00:30:41 said, I think the identity workflow process is actually one of our exact sample applications. + +00:30:46 So, you know, like you're right, they lay sign up and they have to upload some forms of + +00:30:49 ID. + +00:30:50 Someone has to check it. + +00:30:51 It has to go through maybe a background check process and all of that. + +00:30:54 That's a long running workflow. + +00:30:55 That could take days. + +00:30:56 That could take weeks, depending on what kind of background check you're getting done. + +00:31:00 Temporal can suspend its state for however long it wants, you know, and we can guarantee that it will come back online. + +00:31:07 So the interesting thing, and whenever you ever, if you ever see Temporal at a booth at a conference, we were at PyCon this year, for example. + +00:31:13 Our code sample on our booth has a really interesting statement that always catches people's eye, and it's a little mini workflow, and it's sending emails. + +00:31:21 And what it does is it says sleep for 30 days. + +00:31:24 And nobody in their right mind would actually write a sleep for 30 days in code and expect it to actually function. + +00:31:30 100% works exactly the way you would expect it to in Temporal. + +00:31:33 And we can guarantee that it works because of the way that Temporal is architected. + +00:31:37 Those timers basically exist on the servers on this on this temporal service side. + +00:31:41 And they just get basically your workflow just gets scheduled to resume after the timer has fired. + +00:31:46 So you can guarantee that long running workflows will complete exactly the way that you expect them to. + +00:31:52 So, yeah, long running workflows, amazing use case for Temporal. + +00:31:55 Yeah, it sounds incredible. + +00:31:56 Now, I do want to dive actually in. + +00:31:57 It's super interesting how y'all made this happen in Python. + +00:32:00 But I do want to just maybe talk about the scale. + +00:32:03 Like if I were to run Temporal, now you guys have a cloud. + +00:32:07 So I guess stepping just a bit back. + +00:32:09 This is MIT licensed. + +00:32:11 Yes. + +00:32:11 But you also have pricing. + +00:32:13 Yes. + +00:32:14 Before we go into what I was going to talk about, let's talk about that difference there, that contrast. + +00:32:18 Yeah. + +00:32:19 What's the story? + +00:32:20 So Temporal is 100% open source. + +00:32:22 MIT licensed. + +00:32:23 And so there's the temporal service and there's the temporal SDKs. + +00:32:26 Every single one of our temporal SDKs are and the service and everything is MIT licensed + +00:32:32 forever and always that our founders are big, big fans of that. + +00:32:36 The only SDK that is not is the Java SDK, which is Apache 2. + +00:32:40 And if you know anything about open source licensing, there is basically a dependency + +00:32:44 further up the tree that was Apache 2. + +00:32:47 And you're not allowed to downgrade licensing if you ever make a derivative. + +00:32:50 So that's the only reason that one is. + +00:32:52 but every one of our other licenses is MIT licensed. + +00:32:55 So you could run Temporal open source and be fine. + +00:32:58 The thing that we have found is that at scale, the service itself is challenging + +00:33:03 and requires large SRE teams to run. + +00:33:06 Because essentially what we're doing is we're offering distributed systems as a service. + +00:33:10 We're offering reliability as a service. + +00:33:12 Just because we have abstracted these problems away from someone does not mean that the price + +00:33:17 does not have to be paid by someone. + +00:33:18 And I'm talking about the metaphorical price, not the dollar price. + +00:33:22 someone still has to ensure that that database stays up essentially and that your things are + +00:33:27 still getting scheduled and that the network is up and all of these. So it's super complex to do + +00:33:33 that. And you can run it. So you can run the temporal service locally. You can run the temporal + +00:33:37 workers locally. Everything you run is still local. The pricing model for temporal is just the temporal + +00:33:42 service part, which is run in temporal cloud. So there's a weird misnomer around cloud, which is + +00:33:47 like cloud always assumes that we run everything for you. Temporal cloud is different. Temporal + +00:33:51 cloud only runs the temporal service, your workers, where your code executes is always run, + +00:33:57 at least by for now until who knows if another product will ever come out. I don't know. + +00:34:02 Is run by you on you in your data center on your machines. So your code, your execution run by you, + +00:34:07 that service, that brain, the orchestrator, that's the part you could pay temporal cloud for. + +00:34:12 Right. That's the part that does the orchestration, the part that handles the + +00:34:15 the failure and retries and that kind of stuff, right? + +00:34:18 Yeah. Well, Ted, it's the part that actually the failure retries is handled by the state + +00:34:22 machines that are built in the SDK. It's the part that basically maintains the event history. It + +00:34:26 maintains the communication mechanisms and it is the orchestrator. Yeah. So, but. + +00:34:32 So if we use the cloud, I could like reboot my local data center machine or even my local data + +00:34:37 center. I mean, my version of the cloud, you know, wherever digital ocean or whatever. And when it + +00:34:43 comes back it'll sort of resume along like you guys will see that it's back and then yeah start + +00:34:49 running work on something like that that's what i was getting at technically yes so fun fun + +00:34:53 fun fact about it temporal the again the architecture of this is so brilliant and we could + +00:34:58 get so touched in the weeds about the temporal service does not actually know about any of the + +00:35:02 workers that are running it's always a call out model so your your your machines would come back + +00:35:07 online know that they have things they need to do they would basically start listening back + +00:35:11 Everything that happens in Temporal listens across task queues. + +00:35:14 So they would all come back online. + +00:35:15 They would start listening again. + +00:35:17 And then they would see that there's still work in the task queues, which the service + +00:35:19 maintains. + +00:35:20 But all the service does, it goes, oh, someone requested that I do something. + +00:35:24 And then it puts it on the task queue. + +00:35:25 And then the workers handle it. + +00:35:26 The true magic of Temporal lies within the state machines that are built within the SDKs. + +00:35:31 But they cannot function without that orchestrator service. + +00:35:34 I remember when I first started working on the courses. + +00:35:38 like my primary role is writing courses for Temporal. + +00:35:42 And if you go on to our Temporal Learn site, you can find a lot of our, I think it's learn.temporal.io. + +00:35:48 You can find a lot of our courses. + +00:35:50 I wrote the Java and the Python ones. + +00:35:53 I remember when I was first writing them, I kept asking the SDK engineers like, + +00:35:56 well, how does the server do this? + +00:35:57 How does the server do this? + +00:35:58 And one of the engineers was like, Mason, you're giving the server too much credit. + +00:36:00 It's not that smart. + +00:36:02 It's really the state machines within the workers that are doing all the heavy lifting + +00:36:06 and the service just maintains it. + +00:36:08 but with the long way around, what the cloud service does is it maintains that database. + +00:36:12 So the history, without that event history, that is the magic piece. + +00:36:16 The event history is the single source of truth of all things that have happened, + +00:36:19 and that's what the service provides and maintains. + +00:36:22 So whether you use the cloud version of that or you self-host that, that is up to you. + +00:36:26 Now, again, if you self-host it, you are now responsible for maintaining it. + +00:36:28 You are now responsible for upgrading it, security patches, all of that. + +00:36:33 And there are multiple case studies. + +00:36:34 There are multiple YouTube videos on our channel of people who have gone the self-hosted route + +00:36:39 and have found that the cloud route is easier once they've reached a certain scale. + +00:36:44 So yeah, it's pretty neat. + +00:36:45 - Yeah, yeah, cool. + +00:36:46 More and more, that's the business side of what works for open source, right? + +00:36:51 - Yeah. + +00:36:51 - 'Cause there's something that you wanna do, and then there's another requirement + +00:36:55 that you're probably not good at, and you guys would be really good at, right? + +00:36:58 Like for example, making sure that Temporal itself is reliable and fast and keeps running + +00:37:04 and that kind of thing, right? + +00:37:05 - Exactly, yes. + +00:37:06 And I think it does also come down to, and this is one of the interesting lessons that + +00:37:11 I've learned throughout my career, is that like, what is your core business? + +00:37:14 Is your core business to run and maintain a temporal service or is your core business + +00:37:18 to provide a service to your customers? + +00:37:20 And whenever the price of running your own services outweighs what you could have just + +00:37:24 paid someone else to do it for, then at that point, you have to take a look at something + +00:37:28 and go, maybe I should just pay the provider who has the expertise, who literally all they + +00:37:33 do all day long. + +00:37:34 Because usually someone's SRE team doesn't doesn't full time dedicate to learning the ins and outs of a single product. + +00:37:39 Like I was an SRE. + +00:37:40 I managed a cloud platform for VRBO for Virgo. + +00:37:45 I knew there was like 12 different applications in that stack. + +00:37:48 And the way that I learned about each one was the one that went down that day. + +00:37:51 So you're constantly like figuring out with like you're learning about it as it's on fire. + +00:37:55 It's a terrible way to learn. + +00:37:58 And that's not a great way to run it to to like live your life. + +00:38:01 i found right why i moved into developer education um exactly there's fewer uh midnight calls in + +00:38:08 developer education that is the primary reason why i do this now is i i like there is a point when + +00:38:16 the p when pager duty goes off like one too many times and you become a developer advocate and i'm + +00:38:20 living proof of that you just know yeah you take it off a space style you just take it out put on + +00:38:25 some uh gangster music just exactly right exactly if you don't know the reference you need to make + +00:38:33 sure you put watching office space yeah the movie right i've driven by the list i've driven by the + +00:38:38 area where it was filmed it was filmed out here in austin where i live was it incredible yes + +00:38:42 incredible okay so why did i sort of start in that path what i want to talk about is what is + +00:38:46 so you guys have a cloud version which obviously does stuff in massive scale you talk about a fleet + +00:38:51 of these things. And by the way, at the execution level, we're probably mostly talking Go and Rust, + +00:38:57 just so people know, it's kind of like fast, fast innards. But Rakesh asks, like, it seems to be + +00:39:03 resource insensitive. Is there a lightweight option? Like what is a, what is a minimal temporal + +00:39:08 rather than a, you know, what Uber does or what, you know, ChatGPT does? Because those things, + +00:39:14 you just can't compare them, right? It's like, you're not Microsoft, you're not Google, you're + +00:39:18 not LinkedIn, you're not Netflix. Don't try to be them most. Yeah, it depends on what you're + +00:39:24 trying to accomplish. So I mean, like, I think when it comes to so there's, there's always there's + +00:39:27 the service and then there's the worker fleet. So let's talk. I'll talk about the service first, + +00:39:30 and then we'll talk about the workers. So the service comes in a handful of different flavored + +00:39:34 offerings. There's a local development binary that is meant to be used for development. And + +00:39:40 that's what I use on my local laptop all the time. It's also more than suitable for home lab stuff. + +00:39:46 So if you're wanting to play with this in your home lab, use the use that single binary. + +00:39:50 It can it's got an in memory data store, but you can have it persist to it to a SQLite database + +00:39:55 will get you very, very far on home lab stuff. Non prod use cases. Totally fine. + +00:40:00 Speaker 1: SQLite is underappreciated. People see it as like a toy thing or like I can use this while + +00:40:06 I'm developing like a sample, but then I don't really like you can put millions of records in + +00:40:10 SQLite. You can go a very long way. Speaker 1: Oh yeah. Yeah, it is. Yeah, + +00:40:14 it is an amazing tool it's one of my favorite tools um and then there so in reality what the + +00:40:19 way that the temporal service is actually built is it's actually just a single binary um but the + +00:40:23 thing is there are the temporal service is a conglomerate when i say the service i mean like + +00:40:27 all of the things together it's a conglomerate of multiple microservices that when you put them + +00:40:31 together they they connect with they interact with each other and there's like four back-end services + +00:40:36 no three services and a front-end service then there's like a web ui and that's the service but + +00:40:40 but then you also have to build in a data storage. + +00:40:44 So for when you get to that point, that's when you either need MySQL, Postgres, or Cassandra. + +00:40:48 You could probably get away with SQLite on that one, but I've never actually tried it. + +00:40:52 But we recommend MySQL, Postgres, or Cassandra. + +00:40:55 And then you can add in other things like Elastic for visibility. + +00:40:59 You can add Prometheus and Grafana for being able to see. + +00:41:02 But this is when you're starting to scale up. + +00:41:03 So if you were doing this on small, itty-bitty scale, you could probably deploy it onto, say, + +00:41:08 a DigitalOcean droplet and be fine with a single binary. + +00:41:11 We have tutorials on our learn site on exactly how to do this. + +00:41:14 And then it scales. + +00:41:15 So like there's a Docker Compose file. + +00:41:17 So like once you want to see what all these microservices are doing, like in multiple areas, + +00:41:21 like you can see how we've deployed all of them and you can play it. + +00:41:24 There's multiple different Docker Compose files to be like, oh, I want it with this database + +00:41:28 and these options. + +00:41:29 And that allows you to kind of like tune it and tweak it to your liking. + +00:41:32 And then once you get to prod scale, we have Helm charts that we support + +00:41:36 and you can deploy it directly into Kubernetes. + +00:41:38 Now, if you're self-hosting in production, Kubernetes tends to be what we see in the wild + +00:41:42 as the most popular deployment. + +00:41:45 Now, again, these are single binaries. + +00:41:46 You can deploy it however you want. + +00:41:48 So it has a, from development to prod, it has a very nice scaling story. + +00:41:53 Or the other option is just go to Temporal Cloud. + +00:41:55 The thing is Temporal Cloud has a minimum pricing structure for it, which we're constantly updating. + +00:42:02 And that's really useful once you get to actual production use cases and you have people paying you + +00:42:08 use that money to pay for the temporal cloud. + +00:42:10 It's not a lot. + +00:42:10 The temporal cloud pricing is super, I think it's really super interestingly well done + +00:42:16 because it's based on what we call actions. + +00:42:18 So like anything that basically causes a right to that durable database, + +00:42:21 you get billed on like fractions of a penny. + +00:42:24 So it's consumption based. + +00:42:26 The main thing is making sure you have enough traffic to cover, basically there's a minimum support cost + +00:42:31 that you have to pay for like a minimum bill requirement and then you get billed on the actions + +00:42:35 and making sure you have enough traffic and workload to handle that. + +00:42:38 I know, and then you get a lot of other cool features in cloud that, I have to make sure I say this carefully, + +00:42:44 that you don't get on the open source version, but those are all deployment options. + +00:42:49 So when it comes to feature to feature option, as right now, it's a one-to-one comparison. + +00:42:54 What runs in open source is what is deployed in cloud. + +00:42:58 Like, if I built all of my stuff on the open source stuff, I deployed to cloud on occasion to test it, + +00:43:03 but I'm constantly building on the open source version And all of my stuff runs in cloud. + +00:43:07 What you get when you go to cloud is you get those production features. + +00:43:10 You get single sign-on. + +00:43:11 You get role-based access control. + +00:43:13 You get more advanced metrics and things of that nature. + +00:43:17 You get all of these things that really matter at a large-scale production, + +00:43:20 things that large enterprises care about. + +00:43:23 And there are ways you can get those in the open source as well, but they come as first-class citizens in the cloud product. + +00:43:31 Maybe you got to run your own single sign-on identity server or something like that yeah yeah um so over here if i go to the temporal github repo there's docker + +00:43:44 which has got a docker compose think something yeah there's actually a docker repo repo yeah + +00:43:51 is there okay so i could even i think it would be like temporal and then doc like in the search i + +00:43:54 would just search or in the org i would search for docker compose yeah it might be there somewhere + +00:43:59 uh there we go are you right yeah i'll put that in the links as well i'm a huge fan of docker + +00:44:04 compose. I think it opens up a bunch of great options. And if you go in here, you can see that + +00:44:08 there's like the doctor compose my sequel to Docker compose postgres, and then just a bare bones one, + +00:44:15 right? So if you wanted to self host it, is this a pretty good way to go? + +00:44:19 Yeah, I think it's a great place to start. I think it always depends on your scale. + +00:44:22 Like if you if you are good at Docker compose, and you know, you think like, the thing with Docker + +00:44:28 compose is like, and I believe this is all going to deploy it on a single service. So is like our + +00:44:31 single server. So like we always talk about like, you know, depends on the level of reliability you + +00:44:36 need of the service because if the database of the service goes away, then your reliability goes away. + +00:44:41 Like, you know, but that's kind of like the truth here is with that database being the single source + +00:44:45 of truth, if that database magically disappears, you don't have it. Now you could, you know, what + +00:44:50 you could do like an RDS or like a digital ocean managed database and connect that in this. + +00:44:55 Yeah, run just basically set a room, an off server database connection. + +00:44:59 string. Yeah, exactly. Right. Of any form that's then, then it's matching the reliability of that + +00:45:04 database. Yeah, exactly. Yeah. Yeah. So another thing, whenever I hear durable, and actually I see + +00:45:09 a bit about this in the show notes as well, and you say retry, like if something goes wrong, + +00:45:13 which we tried and it'll probably resume. Like sometimes it won't. Sometimes there's a poison, + +00:45:18 what's called a poison message. Like it's cursed to always fail for whatever reason. Like it's + +00:45:23 trying to get to a service. The domain is gone or it's not coming back. Right. Yep. One of those + +00:45:28 sorts of things. Three months ago, it was there. Then you said sleep for three months and it woke + +00:45:32 up and said, wait, it's like Planet of the Apes. You're like, yeah, what have they done? You wake + +00:45:37 up, you're like, the world is not what I thought, right? How do you deal with that kind of stuff? + +00:45:42 Yeah. So in the retry policy, so the default retry policy, what is, is it, its default is just retry + +00:45:49 on a specific time limit forever until it's either canceled or it succeeds. Now, if you expect that + +00:45:55 something could happen like that. Essentially what you would do is there's something what's + +00:45:59 known or non-retriable error types. So certain HTTP error codes, you know, might, you might would be + +00:46:05 like, Hey, you know, this is just not going to come back. 500 is fine, but maybe 404 is like + +00:46:11 404 might never come back, but 500 very likely could or cannot connect. I don't know. Is that + +00:46:17 404? I don't think so. I think it does. I don't remember. Yeah. I remember, but there's probably + +00:46:20 a code that like i i can't connect to the server versus like i got there and it said not here yeah + +00:46:26 so there's non-retriable error codes and i mean like what we do with the core temporal primitives + +00:46:30 is more of like um i often tell people it's like yeah like things that can retry let them retry but + +00:46:34 like say i do a divide by zero error no amount of retries is going to change the laws of physics + +00:46:40 um you know it's like holding your breath until something changes you're just going to pass out + +00:46:44 um same thing with these retries so you have to have what are called non-retriable errors and you + +00:46:47 essentially say, hey, whenever you experience this error, you should just fail. And basically, + +00:46:51 you would bubble it up and then you have to let the next layer of execution handle it. + +00:46:55 So yeah, totally a good story for that. Okay. Yeah. But you kind of got to think a little bit + +00:46:59 about it, right? Just yes. Yes. You still have to think it doesn't take away all the thinking for + +00:47:03 when it comes to like, what potentially could go wrong, but at least like with all of like the weird, + +00:47:08 like, you know, okay, this service might go down DNS, someone messed with DNS. It's always DNS. + +00:47:13 I had I had an it's always DNS moment the other day. And I'm like, I need a sticker for my laptop + +00:47:17 says this because it got me again. + +00:47:19 I can't remember what took me out. + +00:47:21 I suffered something with that as well. + +00:47:23 And it wasn't even directly me, but it was something that I was subjected to. + +00:47:27 Yeah. + +00:47:27 It is always DNS. + +00:47:28 It's always DNS. + +00:47:29 Except for when it's the database. + +00:47:31 Yeah. + +00:47:32 Or except for when you accidentally deploy the walrus operator on Python 3.7 on the server. + +00:47:37 That also is not good. + +00:47:38 And it's not doing, which I took down Talk Python for 20 minutes. + +00:47:42 I'm like, what? + +00:47:43 Why won't it run? + +00:47:43 It was running perfect. + +00:47:45 Oh, this was like, you know, years ago when those things were like new, right? + +00:47:48 Yeah. + +00:47:49 Oh my goodness. + +00:47:50 So one thing I want to talk about here, let me get back to, that's the right spot. + +00:47:54 Let's talk about the programming execution model. + +00:47:58 Let's see, is there a quick start? + +00:48:01 Let's maybe, let's talk through the quick start. + +00:48:02 And I think there's just some super interesting modern Python ideas that you got here, right? + +00:48:09 So maybe talk us through like, how do we get started bringing this in? + +00:48:13 Does it have to be a from scratch? + +00:48:15 Or can I take some part of my application, like a particular API endpoint and go, this + +00:48:19 thing needs, needs some help. + +00:48:21 And Temporal can help it. + +00:48:22 Let's just plug it in on this one bit. + +00:48:24 That's actually how we recommend a lot of people get started with Temporal is like, we + +00:48:28 don't, we never tell people like do like a whole ground up rewrite of like everything + +00:48:31 you have. + +00:48:32 We, I've often told people find something that, like find, find a service that annoys + +00:48:38 you, that pages you because it's constantly going down because it's unreliable and maybe + +00:48:43 do a like it's a small service and do a small rewrite of that in temporal and just see how your + +00:48:47 life makes uh makes a difference so the way that you do it with temporal is like you have to use + +00:48:51 the temporal sdks so the way that you build temporal applications is you build them using sdks + +00:48:56 um typically with other workflow-esque engines or other durable execution engines um some of the + +00:49:01 more modern durable execution engines have kind of followed suit with us but some of the earlier ones + +00:49:05 didn't we're code-based um we're not dag based we're not yaml based we don't have our own structured + +00:49:11 DSL, we are 100% code based. And there's a lot of advantages to that. So not XML based. No, + +00:49:18 no XML. So yeah, so we build you build basically what's called a workflow. Workflow is you can kind + +00:49:26 of think of a workflow is like your application. It's it's the it's the blueprint of your + +00:49:29 entire application. And what we'd say about workflows is that they have to be deterministic, + +00:49:34 like the code within a workflow must be deterministic, and you execute it line by line going + +00:49:38 down. And then anything within your workflow that potentially could be non-deterministic or + +00:49:43 could potentially fail. So calling out to a microservice is both of those. It's non-deterministic + +00:49:49 because you don't know what you're going to get back. And it's potentially going to fail because + +00:49:52 the service could be down. That goes in what's called an activity. An activity is that thing that + +00:49:56 automatically gets the retries. Now you implement activities as functions or as methods in Python. + +00:50:02 You can actually do them as both. And as you were saying in the readme, yeah, we use a lot of + +00:50:08 Python tooling. So that's actually something that I think that our SDK engineers, we have an entire + +00:50:13 team whose entire job is to build and maintain our SDKs. They're super proud of. It's one of the + +00:50:17 things I love talking about. Temporal is not built off of what you would say like an open API spec + +00:50:23 for a good reason. So it's built basically, I mean, Temporal itself is built in Go. So as you + +00:50:28 can imagine, everything's built in protobufs. There is a spec on what we build off of, but open API + +00:50:33 specs generate stuck stub client libraries. And like I've worked with a handful of them. They don't, + +00:50:38 they're not very idiomatic to the language. Like it kind of looks like someone bolted a C library + +00:50:42 on top of Python. Like it works, but it doesn't feel like Python. Our SDK engineers spend months + +00:50:49 studying the programming language, learning out what is idiomatic of the language, what actually + +00:50:53 makes sense. And then they build into it. So the interesting thing about this is like, you can see + +00:50:58 here when we define our decorate, our workflows, we define them by decorating classes. And then we + +00:51:03 the entry point by decorating it with an at workflow.run. So that's how we know where the + +00:51:07 entry point is. We're using asyncio inside of this to do it. Our activities, when you want to + +00:51:13 turn a method into an activity, you are a function in an activity, you just decorate it at activity.defin. + +00:51:19 And now you have an activity. We've done a lot of those kinds of things. We're using context + +00:51:25 managers very, very a lot or, you know, rigorously within the code base. The really interesting thing + +00:51:32 is, and I can only talk a teensy bit about this because it's super complex, but there's a really + +00:51:36 great talk track from it at the PyTexas conference from this year. We built a custom async event loop + +00:51:41 for this. This runs in a durable asynchronous event loop. And the SDK engineer who built it + +00:51:46 gave a talk at PyTexas this year, and it's on the PyTexas 2025 website. And I can provide a link to + +00:51:51 that later. And it's really neat because essentially we had to build a custom event loop to handle + +00:51:57 all of the way that we expect Temporal to work. + +00:52:01 And I don't think that building a custom event loop is a very common thing for people to do. + +00:52:05 So there's a lot of lessons learned there. + +00:52:07 I think there should be more of it though. + +00:52:08 I think it's a really interesting story. + +00:52:11 Yeah, I'll put this in the show notes. + +00:52:13 Yeah, that's Chad's talk. + +00:52:14 It's amazing. + +00:52:15 So like he talks about all the things that he had to do when he was building out a custom event loop + +00:52:22 for Temporal's AsyncIO event loop. + +00:52:24 Yeah, so for people for whom this is super new, this idea. You need places in your code to execute to say, okay, we can stop, maybe save some state. + +00:52:33 And then you talked about like replaying behaviors and so on. So you can write code like await sleep + +00:52:39 a day, you know, asyncio dot sleep one day. And that goes into this, as you said, durable asyncio + +00:52:46 execution. So instead of just saying, well, we're going to let the thread do other stuff. It's like, + +00:52:51 no, save it and then resume it. Right. That's pretty wild. It's pretty great actually. Yeah. + +00:52:56 And I mean, those, those, and that's one of the great things about the workflows and like + +00:53:00 that, those you'd have to definitely write as workflows, but yeah. + +00:53:02 And it takes up no, no energy or it doesn't jam up a CPU thread by sleeping. + +00:53:06 Like it's usually if you sleep something, that thread is kind of sitting there and it's + +00:53:09 stuck. + +00:53:09 this a hundred percent because it's all event sourced under the hood, essentially. + +00:53:14 the event, basically the timer started event gets recorded into the service and then + +00:53:19 it gets descheduled. + +00:53:20 And then that worker, that, that executor can continue performing other tasks. + +00:53:23 And then once that timer fires, the next task for it to continue execution gets put on the task queue. + +00:53:28 The worker consumes it, knows, oh, I need to resume. + +00:53:31 And it resumes. + +00:53:31 And that can happen whenever. + +00:53:33 That can happen, you know, a day from now, three days from now, three months from now. + +00:53:36 It doesn't matter. + +00:53:36 Eventually it gets put on the queue and it gets executed as if nothing happened. + +00:53:39 Yeah, that's wild. + +00:53:40 So interesting question out of the audience. + +00:53:42 If people went with your cloud option, where does that run? + +00:53:45 Is that AWS? + +00:53:46 Is that DigitalOcean? + +00:53:48 Currently it's in AWS and GCP. + +00:53:51 Okay. + +00:53:51 So those are the two clouds. + +00:53:52 Pick different availability zones. + +00:53:54 Like if I were in Virginia, US East one or whatever, I could say I want to use that one. + +00:53:59 Exactly. + +00:53:59 There's different availability zones and we even have multi-region failover. + +00:54:03 So if you needed multi-region availability for super, super high availability, + +00:54:07 which we do have definitely have large customers who need that level of availability. + +00:54:11 And you can totally do that. + +00:54:13 Cool. + +00:54:13 And that doesn't mean you got to be in AWS. + +00:54:15 No, no. + +00:54:16 We have people who are running on their own private, they were running on their own infrastructure + +00:54:20 on their on-prem and they're calling into it. + +00:54:22 So that's the niftiest thing about temporal cloud is like the security model is super simple because temporal cloud or the temporal service. + +00:54:30 It doesn't have to be temporal cloud. + +00:54:31 It's the service, whether you self-host it or whether it's on the cloud. + +00:54:34 It never connects to you. + +00:54:36 The workers always do an outbound connection into the temporal service. + +00:54:39 So if you're using temporal cloud, the only firewall rule you have to allow is a single outbound connection off of a single port. + +00:54:45 That's it. + +00:54:46 So it's really awesome. + +00:54:47 Yeah, I'm impressed with a lot of this. + +00:54:49 So let's sort of kind of get into that. + +00:54:51 And I want to maybe close out with two topics, testing and what's next, or like sort of also maybe multi-language stuff + +00:54:59 we should touch on as well. + +00:55:00 But let's talk testing first. + +00:55:02 So there's a couple of interesting challenges here. + +00:55:05 Like first, if I write code like this and I want to have a unit test, no longer am I like, well, this is complicated. + +00:55:11 I just have to use pytestAsyncIO. + +00:55:12 It's like more than that, right? + +00:55:14 So what's the testing story? + +00:55:16 Yeah, so the cool thing about the testing story with this is it is technically still pytestAsyncIO. + +00:55:21 because we're code-based, this is one of the things that I always have to harp, to, to remind people on my courses and everyone's always like, oh, that's so cool. + +00:55:29 you, because we're code, you have to give up none of your tooling. + +00:55:33 You don't have to get like, how do you like to package it? + +00:55:35 Are you a poetry person? + +00:55:36 Are you a B person? + +00:55:36 Are you a Pippin person? + +00:55:38 Whatever you want to use, use it for your testing. + +00:55:40 Do you want to use a Po the poet? + +00:55:42 Are you a pie test person? + +00:55:44 Like, what are you using? + +00:55:45 Use it. + +00:55:45 It doesn't matter. + +00:55:46 Temporal does provide, obviously because these are more complex workflows and stuff. + +00:55:50 they require a little bit. Every single temporal SDK does provide a testing framework that is built + +00:55:56 into it. And these integrate natively with all of your testing frameworks. So you can use, you use + +00:56:02 pytest. I use pytest. Now you could use something else, but all of our courses, I think the temporal + +00:56:07 102 course, yeah, I wrote that one, has a whole chapter on testing temporal workflows and activities + +00:56:13 and mocking activities because you have to be able to call them independently because you, + +00:56:16 Otherwise that's an integration test. + +00:56:17 It's not a unit test. + +00:56:18 It is important. + +00:56:19 I always feel bad about mocking the code. + +00:56:20 It doesn't seem nice, but you got to. + +00:56:22 Yeah, it's super easy. + +00:56:23 It's so nice. + +00:56:25 The you you basically just read you basically redefine the activity and I just put it in the mocking story in Python is so nice. + +00:56:32 But you just use our testing framework. + +00:56:34 And what that does is it basically the testing framework will automatically + +00:56:36 spin up a temporal service and a worker for you. + +00:56:39 So because as you can imagine, like the execution part of this is like, you have to have a service running, you have to have a worker + +00:56:44 and then you have to send like basically use a client to send a request to execute the workflow and it will be executed. The temporal testing service + +00:56:51 does all of that for you. And it also has basically time skipping. So what you can do is, you know, + +00:56:56 if I have to test, if I have to test something that sleeps for 30 days, I would actually prefer + +00:56:59 it to not actually sleep for 30 days. So I can test it. Well, I see I take so long. Yeah. So + +00:57:05 time skipping is in there and there's a lot of other really neat features in the testing. So + +00:57:08 every single temporal Python or sorry, sorry, every single temporal SDK has a testing framework + +00:57:14 built into it that really well enables testing. + +00:57:17 And it all works natively with the tools that you are already used to using as a developer + +00:57:22 of whatever language you're already using. + +00:57:23 Yeah, that sounds great. + +00:57:24 And you can say things like run this and wait for a week and then resume. + +00:57:28 And you can just say, and now a week has resumed. + +00:57:30 You know, a week has passed. + +00:57:31 Now I see what's happened, right? + +00:57:33 Like it'll just zip right ahead. + +00:57:35 Yep. + +00:57:35 Skips right ahead. + +00:57:36 Doesn't even, doesn't even worry about it. + +00:57:37 That's always a fun testing trick. + +00:57:39 So the other one I want to talk about is when I'm over here, into the GitHub repo, I can scroll down. + +00:57:45 If I look at the repositories, it's Java SDK, Python SDK, Go SDK,.NET SDK, Ruby SDK, et cetera, et cetera. + +00:57:52 There's a bunch of interesting things here. + +00:57:55 Like I can write part of my app in Java, part of the workflow in Java or the workflow items in Java, + +00:58:01 the queues, and I can do part of it in Python or.NET. + +00:58:04 And the other part I think is interesting is Python,.NET, Ruby, et cetera, + +00:58:08 all share a common Rust base. + +00:58:10 And so it's kind of like they all go in lockstep. + +00:58:13 Yeah. Yeah. Yeah. So yeah, two great topics there. So the first one I'll start off with is the is the + +00:58:19 polyglot stuff, because I think it's one of my favorite things. And we don't I don't I don't ever + +00:58:23 get to talk about it enough. So I'm glad you asked. Underneath the hood, the way that all of this + +00:58:27 communication happens is it's happening via essentially protobuffs across task cubes to the + +00:58:32 temporal service back and forth. One of the things that you'll find if you dig into temporal is that + +00:58:36 we require your input, the inputs and outputs of your functions to be serializable to basically + +00:58:43 protobufs. Now, if you have something that's not serializable, we obviously, because it's code, + +00:58:47 we provide you the way to extend the serializer. So as long as you can serialize it, again, + +00:58:54 it's code, you can do whatever you want with it. But because of that, because everything speaks + +00:58:58 protobuf, all of these languages can natively speak to each other. So you're right, I can write + +00:59:03 workflows written in Python and call and have three different activities in that workflow, + +00:59:08 one written in TypeScript, one written in Java and one written in.NET. And I can call them + +00:59:12 seamlessly, like basically just by calling execute activity, giving it the name of the function. + +00:59:17 And then I could actually still then pass in a data class of the data that I have as that + +00:59:21 parameter, because it's still getting serialized down into a protobuf. It will get deserialized + +00:59:25 into the other language, execute it and pass back data that I can resume. And then I could call it + +00:59:30 technically from a client that's written in Go. So this enables Polyglot across all of these + +00:59:37 languages. And it's amazing. So if you have legacy systems or you have stuff where like you really + +00:59:41 need something to be written in a whole bunch of different languages, it just gives you this out + +00:59:46 of the box for free. And I think it's one of the neatest features. And one of the other part that + +00:59:50 it really does need about this is it's not just like the fact that it can call it, but it also + +00:59:56 preserves the stack traces as well. So one of the other courses that I developed are Crafting the + +01:00:00 handling strategy course. There's a demo in there where I am showing like basically that exact + +01:00:07 workflow, a Go client that's calling a Java workflow that's calling a Python activity. + +01:00:12 So three different languages. And then I intentionally throw an error in the Python + +01:00:15 activity and I tell it, do not handle it, do not retry it, let it bubble up. And when I get back to + +01:00:21 the Go client, I can see the different stack traces in the different languages all the way through. + +01:00:25 So I get a go basically panic that contains a Java stack trace that contains a, that contains a Python stack trace. + +01:00:32 And I can, I can, I can see all of this across the line. + +01:00:34 So not, and, and also the thing, and just to do it for fun, because I like showing off, I have all of these workers running on different machines. + +01:00:42 So I am, I am running on different, I am crossing process boundaries. + +01:00:46 I'm crossing literally across the network IP boundaries, and then I'm crossing language boundaries and it happens seamlessly and you'd never know that it happened. + +01:00:53 So the orchestration layer is the wildest thing ever. + +01:00:57 And then you asked about the Rust SDK. + +01:00:59 That's a fun one. + +01:01:00 So that kind of goes back into the history a little bit of how Temporal was built. + +01:01:04 And for a crash course in this within two minutes or less, essentially our founders started at AWS together and built out what would become, + +01:01:15 they built the foundations for SQS and what would become simple workflow service. + +01:01:19 Then one of the founders left, went to Azure and helped build the Azure Durable Task Framework at Microsoft. + +01:01:24 They met up back together at Uber and built Uber's Cadence. + +01:01:28 Cadence was then basically like battle tested for four years, open sourced, and then they got permission to fork it and build Temporal. + +01:01:34 So Temporal, the company is six years old, but it was a four year old open source project prior. + +01:01:39 So it's a 10 year old open source project, essentially. + +01:01:41 But because of that, what happened at Cadence was I think they wrote the Go and the Java SDKs there. + +01:01:46 So those are very uniquely themselves because they were written independently. + +01:01:49 And then the PHP SDK is its own story. + +01:01:52 Someone wrote that in the community because they really wanted it. + +01:01:55 And it kind of follows its own rules. + +01:01:57 But when they started building the temporal SDKs, TypeScript was the first one and then Python, if I remember correctly. + +01:02:03 They wanted a common core, like basically re-implementing this. + +01:02:06 Because in these SDKs, there are very complex state machines that maintain all of this state of what's going on. + +01:02:13 And they did not want to keep re-implementing this every single time. + +01:02:16 So they built a Rust core SDK, or it's not even an SDK. It's not an SDK. It's just the core. And + +01:02:22 all of the, so the TypeScript, the.NET, the Python, and the Ruby SDKs all have a upper level SDK + +01:02:28 that wraps this core Rust SDK and call into it. So they share a common theme. So there definitely + +01:02:33 will sometimes be features or like things that happen in the Go or the Java SDK that you're like, + +01:02:37 that's a little different because those are not based on Rust core. But yeah, that's how they all + +01:02:42 call in. So like their PIO3 is basically being used here. We're calling into pot with PIO3 into + +01:02:47 a rust binding. and that's like a Pydantic and others. Yeah, exactly. Yeah. So, and it's, + +01:02:53 it's really cool. And that, that makes, adding new SDKs a lot easier because really, + +01:02:57 and truly the hardest part of building the SDKs is, was like those state machines used to take + +01:03:01 a long time. And once they got it figured out on the rust core side, it made adding new languages + +01:03:05 easier. the Ruby SDK is in public preview and will be going generally available here soon. Um, + +01:03:11 And there may be one or two more SDKs coming out within the future. + +01:03:14 If you guessed really hard, you could figure out what it is. + +01:03:17 There's an SDK. + +01:03:17 There's a core that doesn't have an SDK. + +01:03:22 There's no secret. + +01:03:23 There's no secret about that. + +01:03:25 Yeah, of course. + +01:03:25 People have been begging for that for years and it's obvious. + +01:03:28 So yeah, may involve crates. + +01:03:30 Okay. + +01:03:30 So what's, what is the, what's the future? + +01:03:33 Like anything that's worth giving a shout out that's coming or that kind of stuff? + +01:03:38 Yeah. + +01:03:39 I mean, I think that like a lot of times people often ask me like, what is Temporal used for? + +01:03:44 And I would say Temporal is used for anything that like you don't want your code to fail. + +01:03:48 It like it's it's really interesting to help educate people and work on a product that really does affect nearly every single part of the application of the software development lifecycle and every single part of it, of, of, of the industry. + +01:04:03 You know, I was used to working on other products that like, yeah, like I worked at a synthetic data company for a little bit. + +01:04:08 And that had a very niche area. + +01:04:09 And then I worked at DigitalOcean, which was cloud products, which is still awesome, + +01:04:12 but like doesn't affect everything. + +01:04:14 Temporal really does like, are you doing finance? + +01:04:17 Temporal is great for long running transactions. + +01:04:20 Are you doing food delivery? + +01:04:21 Like, are you a fast food industry? + +01:04:23 Are you doing groceries? + +01:04:24 Are you doing, what are you doing? + +01:04:26 Temporal can benefit from it. + +01:04:27 So there's a lot of really cool things. + +01:04:29 You can use it for anything. + +01:04:30 And what we're seeing right now, specifically, and this kind of alludes back + +01:04:33 to your open AI thing earlier, is that we're seeing a lot of AI companies + +01:04:38 of value out of this because like when it becomes time to take your agents to production, there's + +01:04:44 a handful of decent production stories out there, but it turns out these are AI agents in production. + +01:04:50 This is microservices in production with a fancy label on top of it. These are just distributed + +01:04:54 systems. Not only are they microservices, they're very slow, long running microservices, + +01:05:00 which make it harder. Yeah. Yeah. That's exactly what Temporal's model is. Do you have a slow + +01:05:05 running microservice that can sometimes fail. Great. We have a product that makes all those + +01:05:09 problems go away. So, you know, like I'll, I'm working on a lot of content right now around + +01:05:14 showing the benefit of temporal in AI. and we have, we have a handful of customers, who I + +01:05:19 can't talk about, that are, using us for lots of different AI related things. but there, + +01:05:25 I mean, you can look on our blog or anything. You can see tons of people that are using it. + +01:05:28 and it's a really cool thing. So I would definitely say like, if you're, if you're trying + +01:05:31 to take AI to production, you should be looking into temporal. it's not an AI tool, you know, + +01:05:36 like we're not, we're not going to like, we're not going to do the thing that every company did is + +01:05:39 we're not going to suddenly pivot and become an AI tool. because we're not, we just, yeah, + +01:05:45 we solve everything. And AI is one of the great things we solve. So that's awesome. Yeah. You're + +01:05:50 not going to vibe code with temporal. Maybe you vibe code, temporal code, but not with temporal. + +01:05:55 No, I've, I've, I've actually vibe coded a handful of temporal things. And it's interesting because + +01:05:59 like i'm super picky uh about what what people's temporal code looks like as because i've been + +01:06:04 teaching people best practices for three years almost three years now and i i'm like no vibe + +01:06:09 coding that's no claude that's wrong like no no no cursor that's wrong like you can't do that so + +01:06:14 and the interesting thing about that is the way that i'm looking at it's like oh i need to make + +01:06:18 more content about this because yes because the llms are not like it's actually funny every now + +01:06:23 and then the llms spit out some of my content um and i can tell when it's my content because i know + +01:06:27 So I write comments in a very particular way. + +01:06:30 And I'm like, oh, okay. + +01:06:32 So what that ends up telling me is, oh, I need to make more content around this + +01:06:36 because we're still not vibe coding at 100% capacity. + +01:06:39 Yeah. + +01:06:40 Yeah. + +01:06:40 Yeah. + +01:06:41 That's a whole discussion we could go down. + +01:06:44 You know, I was totally wrong when we started. + +01:06:46 I said we could talk for two hours. + +01:06:47 I think it's three to four. + +01:06:49 Yeah. + +01:06:49 We got so much more we could talk about, but there's only so much time we can dedicate to each episode. + +01:06:55 So let's go ahead and call it. + +01:06:57 I say, you know, thanks for being here. + +01:06:59 The more I looked into this, this is a super interesting product. + +01:07:02 And there's a lot of neat Python integrations like you program it with async and await + +01:07:08 rather than some funky SDK bolt-on thing. + +01:07:12 So people should definitely check it out. + +01:07:13 Final call to action. + +01:07:14 They're interested. + +01:07:15 What do you tell them? + +01:07:16 Check out, just check out the website. + +01:07:18 Check out temporal.io or the learn site, learn.temporal.io. + +01:07:21 It's a great place to get started. + +01:07:23 You can install Temporal by just running brew install temporal on your Mac, + +01:07:26 or there's commands for Windows and Linux as well. + +01:07:29 Curl commands for that. + +01:07:31 Or Docker compose up. + +01:07:32 Or Docker compose up. + +01:07:33 If you want to do that, totally can do that. + +01:07:35 Just try it out, build a workflow. + +01:07:37 And then what I tell people is try to break it. + +01:07:39 Like start a workflow, kill the worker, bring it back online. + +01:07:42 Like I think it's really magical when you first actually try to like actually break the software and stuff. + +01:07:49 We've had multiple people that have taken jobs here who have said, I started playing with it. + +01:07:53 I tried to break it. + +01:07:54 And when I couldn't, I decided to apply for a job here. + +01:07:57 So try to break it. + +01:07:59 See what you can do. + +01:08:00 And you'll be amazed by it. + +01:08:02 Just a personal anecdote. + +01:08:04 I remember when I applied here, I was reading through their docs. + +01:08:06 And I told myself, I was like, if they can do half of the things they claim they can do in the docs, this is revolutionary. + +01:08:12 I've never seen anything like this. + +01:08:13 And it turns out we do all the things we say in our docs. + +01:08:16 It's probably the most interesting tech product I've ever worked with in my career. + +01:08:20 And I know that I will be working with it probably for the rest of my career. + +01:08:23 It fascinates me and I love playing with it. + +01:08:26 Like I build temporal applications at home for fun just because it's like, oh, look, + +01:08:30 I don't have to worry about someone's API going down anymore. + +01:08:33 Yay. + +01:08:33 Yeah. + +01:08:34 It's awesome. + +01:08:35 So I hope you enjoy it as much as I do. + +01:08:37 I'm pretty impressed. + +01:08:39 All right. + +01:08:39 Well, thanks for being on the show. + +01:08:41 Thanks for coming on and sharing everything. + +01:08:42 Yeah, it was great. + +01:08:43 Great to talk with you. + +01:08:44 Yeah, you as well. + +01:08:45 Bye bye. + +01:08:46 This has been another episode of Talk Python To Me. + +01:08:50 Thank you to our sponsors. + +01:08:51 Be sure to check out what they're offering. + +01:08:52 It really helps support the show. + +01:08:54 This episode is sponsored by Posit Connect from the makers of Shiny. + +01:08:58 Publish, share, and deploy all of your data projects that you're creating using Python. + +01:09:03 Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards, and APIs. + +01:09:10 Posit Connect supports all of them. + +01:09:12 Try Posit Connect for free by going to talkpython.fm/posit, P-O-S-I-T. + +01:09:18 The PyBay Conference is returning to the UCSF Mission Bay Conference Center in San Francisco, California on October 18th, 2025. + +01:09:26 Get your ticket and pick up a free conference course bundle from Talk Python. + +01:09:31 Get started at talkpython.fm/pybay. + +01:09:34 Want to level up your Python? + +01:09:35 We have one of the largest catalogs of Python video courses over at Talk Python. + +01:09:39 Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:09:44 And best of all, there's not a subscription in sight. + +01:09:47 Check it out for yourself at training.talkpython.fm. + +01:09:50 Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:09:55 We should be right at the top. + +01:09:56 You can also find the iTunes feed at /itunes, the Google Play feed at /play, + +01:10:01 and the direct RSS feed at /rss on talkpython.fm. + +01:10:06 We're live streaming most of our recordings these days. + +01:10:08 If you want to be part of the show and have your comments featured on the air, + +01:10:12 be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:10:17 This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. + +01:10:21 Now get out there and write some Python code. + diff --git a/transcripts/515-temporal.vtt b/transcripts/515-temporal.vtt new file mode 100644 index 0000000..04302ae --- /dev/null +++ b/transcripts/515-temporal.vtt @@ -0,0 +1,3863 @@ +WEBVTT + +00:00:00.020 --> 00:00:04.740 +What if your code was crash-proof? That's the value prop for a framework called Temporal. + +00:00:05.260 --> 00:00:09.000 +Temporal is a durable execution platform that enables developers to build + +00:00:09.440 --> 00:00:13.300 +scalable applications without sacrificing productivity or reliability. + +00:00:14.130 --> 00:00:19.460 +The Temporal server executes units of application logic called workflows in a resilient manner that + +00:00:19.760 --> 00:00:25.340 +automatically handles intermittent failures and retries failed operations. We have Mason Egger + +00:00:25.360 --> 00:00:32.080 +from Temporal on to dive into durable execution in Python. This is Talk Python To Me, episode 515, + +00:00:32.470 --> 00:00:53.480 +recorded June 19th, 2025. Welcome to Talk Python To Me, a weekly podcast on Python. + +00:00:54.080 --> 00:00:59.320 +This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the + +00:00:59.510 --> 00:01:05.620 +podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and + +00:01:05.630 --> 00:01:11.100 +listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, + +00:01:11.470 --> 00:01:15.780 +you can find the live streams over on YouTube. Subscribe to our YouTube channel over at + +00:01:15.920 --> 00:01:21.899 +Talk Python.fm slash YouTube and get notified about upcoming shows. This episode is sponsored by + +00:01:21.980 --> 00:01:27.040 +Posit Connect from the makers of Shiny. Publish, share, and deploy all of your data projects that + +00:01:27.120 --> 00:01:33.560 +you're creating using Python. Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, + +00:01:33.880 --> 00:01:39.800 +Dashboards, and APIs. Posit Connect supports all of them. Try Posit Connect for free by going to + +00:01:39.820 --> 00:01:46.779 +talkpython.fm/posit, P-O-S-I-T. The PyBay Conference is returning to the UCSF Mission Bay + +00:01:46.800 --> 00:01:53.560 +Conference Center in San Francisco, California on October 18th, 2025. Get your ticket and pick up a + +00:01:53.650 --> 00:01:59.000 +free conference course bundle from Talk Python. Get started at talkpython.fm/PyBay. + +00:02:00.070 --> 00:02:03.560 +Big announcement, we have a brand new course for you all data science types out there. + +00:02:04.180 --> 00:02:09.539 +Just Enough Python for Data Scientists. Data scientists get things done in notebooks, + +00:02:09.979 --> 00:02:14.899 +but production quality work needs more than ad hoc scripts. Just Enough Python for Data + +00:02:14.920 --> 00:02:19.960 +scientist gives you the essential Python and software engineering habits to level up your + +00:02:20.140 --> 00:02:24.820 +analysis without drowning in theory. In a few focused hours, you'll tighten up your core Python, + +00:02:25.320 --> 00:02:30.520 +write clean and reasonable functions, organize code into importable modules, track work with + +00:02:30.550 --> 00:02:36.160 +Git and GitHub, debug confidently, and make your results reproducible with pinned environments and + +00:02:36.300 --> 00:02:42.940 +Docker. You'll also see how modern agentic AI tools can accelerate data exploration, bug detection, + +00:02:43.220 --> 00:02:48.600 +refactoring, and documentation. The outcome is simple. You keep your notebook speed while gaining + +00:02:48.630 --> 00:02:54.920 +the reliability, collaboration, and professionalism your projects deserve. Just visit talkpython.fm + +00:02:55.100 --> 00:02:59.260 +and click on courses in the nav bar. The link to the course is also in your podcast player show + +00:02:59.400 --> 00:03:05.060 +notes. And if you'd rather focus purely on building with LLMs, check out Vincent Warmerdam's + +00:03:05.300 --> 00:03:10.520 +LLM building blocks for Python course we just recently released as well. Now on to that interview. + +00:03:11.480 --> 00:03:14.160 +Mason, welcome to Talk Python To Me. Fantastic to have you here. + +00:03:14.450 --> 00:03:15.800 +It's great to be here. Long time listener. + +00:03:16.200 --> 00:03:19.980 +Oh, wonderful. I'm definitely a fan of stuff I've seen you doing online as well. + +00:03:20.090 --> 00:03:24.280 +And it's super cool to get together here and, you know, share it with a couple of people. + +00:03:24.720 --> 00:03:25.940 +Yeah, definitely excited. + +00:03:26.420 --> 00:03:28.860 +Temporal. Wow, what a cool topic. Durable execution. + +00:03:29.620 --> 00:03:33.860 +What a neat idea that I've seen in other places, but I've seen less of it in Python. + +00:03:34.130 --> 00:03:35.840 +So I'm real excited to dive into this. + +00:03:35.960 --> 00:03:37.760 +This is something I learned about recently. + +00:03:37.930 --> 00:03:41.260 +We'll go into that in a bit, but there's a lot here. + +00:03:41.340 --> 00:03:44.920 +Let's just leave it like this could be a two hour show and we'd still be going easy. + +00:03:45.520 --> 00:03:45.800 +Definitely. + +00:03:46.040 --> 00:03:46.100 +Yeah. + +00:03:46.220 --> 00:03:49.400 +I, I spend a lot of my time educating people about durable execution. + +00:03:49.730 --> 00:03:51.000 +that's, that's what my role is. + +00:03:51.060 --> 00:03:52.880 +I'm a develop, one of the developer educators at temporal. + +00:03:53.620 --> 00:03:57.700 +and definitely you could spend hours on this and we would, we could be going forever. + +00:03:58.060 --> 00:03:58.520 +So yeah. + +00:03:59.000 --> 00:03:59.140 +Yeah. + +00:03:59.310 --> 00:04:00.100 +And we definitely could. + +00:04:00.400 --> 00:04:03.020 +Well, before we do go on for hours, who are you? + +00:04:03.300 --> 00:04:03.660 +It was Mason. + +00:04:04.200 --> 00:04:04.480 +Oh yeah. + +00:04:04.840 --> 00:04:05.960 +my name is Mason Egger. + +00:04:06.180 --> 00:04:08.900 +I am a developer educator at temporal, as I mentioned. + +00:04:09.060 --> 00:04:12.280 +And I also help run the PyTexas conference. + +00:04:13.440 --> 00:04:18.799 +So PyTexas is one of the oldest, actually the oldest regional Python conference in the + +00:04:18.880 --> 00:04:19.920 +world that we know of. + +00:04:20.780 --> 00:04:22.480 +No one else has come to claim that spot yet. + +00:04:22.540 --> 00:04:28.140 +We started in the late fall of 2007, like right after PyCon US would have started. + +00:04:28.300 --> 00:04:31.620 +We'll be experiencing our 20th year this upcoming year, which is exciting. + +00:04:31.980 --> 00:04:34.360 +And I also am the president of the PyTexas Foundation. + +00:04:34.600 --> 00:04:46.200 +So PyTexas has its own 501c3 organization that is used to basically act as a sheltering organization for all of the Python meetups and events that go on within the state of Texas, which is really nice. + +00:04:46.680 --> 00:04:49.860 +So I was elected president of that back in 2002. + +00:04:50.540 --> 00:04:53.520 +I've been helping run and build the Python community there ever since. + +00:04:53.560 --> 00:04:55.340 +And it's a lot of work, but it's a lot of fun. + +00:04:55.360 --> 00:04:56.940 +And I really enjoy my community work. + +00:04:57.320 --> 00:05:00.020 +Yeah, it sounds extremely fun, although also challenging. + +00:05:00.140 --> 00:05:04.780 +You say for the entire state of Texas, Texas is a big place. + +00:05:05.110 --> 00:05:08.020 +It's like countries as well, if you look at the size of Texas. + +00:05:09.060 --> 00:05:09.740 +It really is. + +00:05:09.860 --> 00:05:15.320 +Texas has some unique issues with it when it comes to running community stuff. + +00:05:17.040 --> 00:05:22.280 +We founded our own virtual meetup for basically anyone in Texas, and anyone else can join, + +00:05:22.500 --> 00:05:22.800 +obviously, too. + +00:05:22.880 --> 00:05:25.080 +We don't say, oh, no, no Texans can join it. + +00:05:25.120 --> 00:05:37.520 +But we did it because the amount of feedback we would get, it's like, oh, I live in, say, the DFW area, the Dallas-Fort Worth area, and I would love to go to the meetup, but I live two hours away from the meetup because of how large the Texas cities are. + +00:05:37.720 --> 00:05:39.900 +Texas cities are sprawling metroplexes. + +00:05:40.340 --> 00:05:47.220 +I grew up in the Houston area, and the joke when you grew up in Houston is you're always three hours away by car from every other part of Houston. + +00:05:48.000 --> 00:05:48.960 +And that really is the case. + +00:05:49.240 --> 00:05:55.400 +So it does have its own set of unique challenges, trying to coordinate four major cities across, + +00:05:55.960 --> 00:05:57.640 +you know, technically two time zones. + +00:05:57.720 --> 00:06:03.600 +If we can, if we include El Paso, the very, very edge, the very west part of Texas is + +00:06:03.620 --> 00:06:04.380 +in the mountain time zone. + +00:06:05.539 --> 00:06:07.580 +So it's an interesting bit of work. + +00:06:07.580 --> 00:06:11.500 +And we've been doing a really good job of expanding and like adding more and more offerings + +00:06:11.660 --> 00:06:13.800 +that we can offer to the community year over year. + +00:06:13.900 --> 00:06:16.140 +And it's been a lot of fun and I really enjoy it. + +00:06:16.320 --> 00:06:16.920 +It keeps me busy. + +00:06:17.080 --> 00:06:17.480 +That's for sure. + +00:06:17.860 --> 00:06:19.040 +Yeah, I could say. + +00:06:19.640 --> 00:06:22.780 +is that why people drive so fast in Texas? They've got so far to go. + +00:06:23.460 --> 00:06:28.420 +Exactly. Yes. I mean, the fastest speed limit in the United States is on the highway next to my + +00:06:28.530 --> 00:06:32.540 +house. It's 85 miles an hour and it's on the toll road next to my house because you're right. Like + +00:06:32.580 --> 00:06:36.700 +if you're trying to get from Austin to San Antonio, that's, I think it's about a hundred miles + +00:06:37.240 --> 00:06:41.120 +down. And like, if you're going 55, you'll get there in, you know, two and a half hours, + +00:06:41.210 --> 00:06:44.960 +but I can make it in an hour and a half hour and 15 minutes. If you just let me fly, + +00:06:45.160 --> 00:06:49.520 +I had no idea that the speed limit were that high. + +00:06:49.550 --> 00:06:50.980 +I knew they were high, but not that high in Texas. + +00:06:51.620 --> 00:06:51.700 +Yep. + +00:06:52.020 --> 00:06:52.980 +Wild, you know. + +00:06:53.240 --> 00:06:54.740 +You guys in Germany, come on. + +00:06:55.060 --> 00:06:58.400 +Yeah, it is the Southern Autobahn, basically. + +00:06:59.340 --> 00:06:59.700 +Fantastic. + +00:07:00.090 --> 00:07:03.480 +So you've had a bit of a tougher challenge, I suppose, + +00:07:04.060 --> 00:07:07.540 +or more challenges than a lot of people with this whole Pi Texas thing, + +00:07:07.880 --> 00:07:09.260 +which I've always thought was super cool. + +00:07:09.650 --> 00:07:10.460 +I'd love to go sometime. + +00:07:10.860 --> 00:07:12.780 +It's far from Oregon, but still would be really lovely. + +00:07:13.240 --> 00:07:16.920 +That said, you started working on this COVID time, right? + +00:07:17.060 --> 00:07:19.520 +Which for conferences is the kiss of death. + +00:07:19.890 --> 00:07:21.420 +You want to talk us through how that went? + +00:07:21.700 --> 00:07:24.180 +Yeah, I could definitely talk about that for a while. + +00:07:24.230 --> 00:07:29.840 +So I got involved in conference organizing, as most people do, by first speaking at the + +00:07:29.990 --> 00:07:30.100 +conference. + +00:07:30.330 --> 00:07:31.880 +So I spoke at the conference in 2019. + +00:07:32.750 --> 00:07:35.060 +I had been a PyCon attendee prior to that. + +00:07:35.370 --> 00:07:41.280 +My first PyCon attendance was the last Portland one, which I believe was 2017. + +00:07:42.820 --> 00:07:47.160 +and then spoke at Pitexas in 2019 and volunteered to help for the 2020 conference. + +00:07:47.360 --> 00:07:50.620 +And that was really interesting because, you know, we had planned to do it in person. + +00:07:50.620 --> 00:07:55.420 +I think everybody had planned, was planning in person 2020, you know, because you, for + +00:07:55.500 --> 00:07:59.220 +those that don't know, when you start planning a conference, you start planning about eight, + +00:07:59.580 --> 00:08:00.320 +10 months prior. + +00:08:00.840 --> 00:08:05.600 +So if you're a spring conference as Pitexas is, we were planning, you know, Pitexas 2020 + +00:08:05.980 --> 00:08:07.360 +in about summer of 2019. + +00:08:07.860 --> 00:08:08.860 +No one knew what was coming. + +00:08:09.580 --> 00:08:11.540 +We were so innocent then. + +00:08:11.720 --> 00:08:11.840 +Yes. + +00:08:12.720 --> 00:08:17.660 +We really were. So we were planning that. And then, you know, the events that we know of happened + +00:08:18.060 --> 00:08:21.720 +and we kept chasing it. We kept pushing it back. I think a lot of conferences at that time kept + +00:08:21.920 --> 00:08:25.380 +pushing back thinking, Oh, we'll be out of this in two weeks. We'll be out of this in three months. + +00:08:26.300 --> 00:08:30.520 +We ended up going virtual. And then in 2021, we made the conscious decision to just not have a + +00:08:30.690 --> 00:08:36.419 +conference. The virtual conference for us in 2020 wasn't overly successful. And we've, at that time, + +00:08:36.430 --> 00:08:39.719 +we were feeling there was just a lot of virtual conference fatigue and we didn't have a really + +00:08:39.740 --> 00:08:47.820 +good story on how to like make it work and dedicate the time. Everybody was also struggling + +00:08:48.000 --> 00:08:53.260 +to handle the world at that time. So being able to put more resources into it was difficult. + +00:08:53.950 --> 00:08:58.220 +So we pushed that off and then we came back in 2022. We made a very conscious decision about it. + +00:08:58.320 --> 00:09:01.460 +Like we were like, we're going to come back in the safest way possible. PyCon US had announced + +00:09:01.490 --> 00:09:05.359 +they were coming back. We decided we were going to come back. We had, we have a venue that has + +00:09:05.580 --> 00:09:10.260 +custom air filtration. We instilled mask mandates and vaccine mandates and all of that stuff. + +00:09:10.600 --> 00:09:15.420 +And we had 76 people return. But we knew that if we didn't come back, if we let this kind of like + +00:09:16.040 --> 00:09:20.120 +continue on, that the likelihood of getting the community to come back, that the memory + +00:09:20.820 --> 00:09:26.620 +is short, that if we didn't come back, we might lose it forever. And having run this for 18 years, + +00:09:26.700 --> 00:09:30.479 +we were concerned about losing it at that point. I think that's totally valid. A lot of this + +00:09:30.560 --> 00:09:35.940 +conference attending is it's habit. You form, you've gone a bunch of times. You really enjoy + +00:09:36.000 --> 00:09:38.780 +it. Like, yeah, of course I'm going to go. You don't even question like, well, when is it? I'm + +00:09:38.780 --> 00:09:43.300 +just going to sign up when it comes out. But you know, if you stop going, then I think that does + +00:09:43.520 --> 00:09:48.620 +make a big change. So it definitely does. So in 22, we, we did it and then we kept coming back. + +00:09:48.780 --> 00:09:53.400 +And then, you know, every year prior after that, we, we continue to grow. it took us three + +00:09:53.560 --> 00:09:59.380 +years to get back to above, what, what I would consider pre-pandemic normals. and the, + +00:09:59.460 --> 00:10:03.600 +the fun fact about that is, and the thing that I think that really kind of helped us out is like, + +00:10:03.660 --> 00:10:07.940 +we didn't really start seeing the growth back in the conference attendance until us as the + +00:10:08.120 --> 00:10:15.100 +foundation turned our inwards local or sorry, our vision local or our site to start focusing on the + +00:10:15.120 --> 00:10:19.840 +local meetups, because the local meetup scene had not returned. And that was the vast majority of + +00:10:19.960 --> 00:10:24.940 +our of our of our marketing was that we would send out these emails and stuff or the local meetups + +00:10:24.990 --> 00:10:29.420 +would promote us. And these are huge meetups. I mean, the Python, the some of the Python meetups + +00:10:29.440 --> 00:10:33.540 +are some of the oldest meetups that I know of. I mean, like they started in the early, + +00:10:33.960 --> 00:10:38.480 +in the late aughts in the early teens. Yeah, I would say they probably were not even called + +00:10:38.680 --> 00:10:42.900 +meetups when they started, right? The user groups or something. They predate meetup. Yeah, + +00:10:43.040 --> 00:10:47.400 +meetup.com and all that. Yeah, I think the PyHouston meetup group is called, + +00:10:47.740 --> 00:10:55.200 +like their tag on meetup.com is Python-14. So I'm assuming it was the 14th Python meetup when it was + +00:10:55.200 --> 00:11:01.040 +created in the world on meetup.com. so, and you know, large groups, but these meetups had + +00:11:01.050 --> 00:11:05.680 +gone dormant. So what I did as, as foundation president at that time was I was like, okay, + +00:11:05.800 --> 00:11:09.040 +I'm going to spend all of my effort while my organizers are helping get the conference going. + +00:11:09.160 --> 00:11:12.600 +I'm going to spend all of my effort finding organizers and rebuilding these meetups in + +00:11:12.600 --> 00:11:16.700 +these areas. so I would connect with people that I knew in the area. I would reach out to + +00:11:16.920 --> 00:11:22.220 +friends. I would put out all calls and after time of rebuilding the ecosystem, then we basically + +00:11:22.240 --> 00:11:25.760 +everything came back to life. And that's kind of where a little, you know, the things that I have + +00:11:25.880 --> 00:11:30.060 +learned is that if your, if your meetup ecosystem is not healthy, then your regional Python conference + +00:11:30.240 --> 00:11:36.060 +ecosystem will not be healthy because it has to feed up into it. Yeah. Yeah. I think it's, + +00:11:36.160 --> 00:11:42.940 +it's such a challenge to run like a conference or podcast or whatever to draw in new people. + +00:11:43.270 --> 00:11:46.840 +I mean, you can do all the amazing mailing lists and announcements and everything, + +00:11:47.120 --> 00:11:51.360 +but you're speaking to your existing people that know about you and finding new ones is really + +00:11:51.360 --> 00:11:56.200 +interesting and this cultivating like user groups to sort of be the foundation. It's a very + +00:11:56.380 --> 00:12:00.760 +interesting idea. Definitely. User groups work. I think we also started the PyTexas meetup, + +00:12:01.120 --> 00:12:05.380 +which was a virtual meetup that we do monthly that we allow anyone to join. And we cultivated + +00:12:05.660 --> 00:12:10.220 +amongst our network of friends and social media and word of mouth. And then we all know what + +00:12:10.400 --> 00:12:14.520 +happened with the social media situation, which really did not help. I mean, there was a lot of + +00:12:14.540 --> 00:12:20.079 +just a lot of things that have happened with like the dissolution of Twitter really did not help + +00:12:20.160 --> 00:12:24.220 +the conference and the tech ecosystem. Basically, we've all fractured now. We're all in like six + +00:12:24.340 --> 00:12:30.120 +different places. Like the move to Mastodon is not complete. Blue Sky had its moment, but I honestly + +00:12:30.320 --> 00:12:36.040 +get less engagement on Blue Sky than anything else. LinkedIn has surprisingly been our most successful + +00:12:36.580 --> 00:12:40.640 +social media platform. It seems like a lot of people have moved there for some reason or another. + +00:12:41.899 --> 00:12:46.459 +But basically, it means that you just have to reapply your marketing strategies. And the fun + +00:12:46.480 --> 00:12:51.180 +thing that I've had the benefit of is that as my work as a developer advocate, all the roles that + +00:12:51.180 --> 00:12:55.360 +I have done, they tend to sit in marketing. Developer advocacy tends to either sit in + +00:12:55.500 --> 00:12:59.480 +product or marketing. All the roles that I have taken sit in marketing. And I've had the benefit + +00:12:59.600 --> 00:13:04.520 +of like, whenever this has started going weird, I can ask all of my marketing friends, hey, what + +00:13:04.640 --> 00:13:08.020 +would you do in this situation? How would you approach this? So I've got to learn through + +00:13:08.240 --> 00:13:12.720 +osmosis kind of how marketing stuff works and being able to apply that to conference organizing + +00:13:12.780 --> 00:13:16.280 +and meetup organizing has actually been substantially beneficial to us. + +00:13:16.580 --> 00:13:16.680 +Yeah. + +00:13:17.180 --> 00:13:22.180 +Certainly, I think companies that are turned on enough to have developer advocates have + +00:13:22.779 --> 00:13:25.360 +developer intelligent marketing teams, and that's a real resource. + +00:13:25.920 --> 00:13:26.100 +Definitely. + +00:13:26.340 --> 00:13:26.480 +Yeah. + +00:13:26.500 --> 00:13:29.220 +It's been really useful to be able to get other people's opinions. + +00:13:29.520 --> 00:13:32.080 +And then, you know, just ask other conference organizers, what are they doing? + +00:13:32.160 --> 00:13:36.720 +I think that, you know, finding out what works and telling other people about it. + +00:13:36.780 --> 00:13:40.180 +I mean, we, I haven't had a chance to write this year's blog post yet, unfortunately, + +00:13:40.260 --> 00:13:40.880 +for this year's conference. + +00:13:41.120 --> 00:13:46.880 +But whenever I became conference chair and then president, I was like, we're going to be as transparent as possible. + +00:13:47.100 --> 00:13:49.560 +Every misstep we make, we're going to blog about it. + +00:13:50.220 --> 00:13:51.120 +We talk about our budget. + +00:13:51.200 --> 00:13:53.900 +I mean, we're also a 501c3, so that's kind of like part of our bylaws. + +00:13:54.440 --> 00:13:56.640 +But at the same time, it's like this is everything that we do. + +00:13:57.000 --> 00:13:58.140 +You can find it on all of our websites. + +00:13:59.380 --> 00:14:00.080 +And this is what worked. + +00:14:00.220 --> 00:14:00.720 +This is what didn't. + +00:14:00.860 --> 00:14:07.780 +Because there's so many first-time conference organizers who may want to start a conference who don't know how to achieve these goals or whatever they're trying to do. + +00:14:08.840 --> 00:14:11.180 +And we have 20 years of experience doing it. + +00:14:11.500 --> 00:14:15.300 +And like it, we need to help each other out and make sure that we distill this, this knowledge + +00:14:15.660 --> 00:14:15.780 +outward. + +00:14:16.260 --> 00:14:17.500 +I don't have 20 years of experience. + +00:14:17.680 --> 00:14:20.620 +So I'm only doing it for four, but you know, institutional knowledge. + +00:14:20.750 --> 00:14:24.640 +There are, there are Google docs with like years of back data that I can go and look + +00:14:24.780 --> 00:14:30.480 +at and be like, oh yeah, in 2009, we ordered way too many meals because we, you know, didn't + +00:14:30.630 --> 00:14:32.400 +charge enough for, we didn't charge anything for tickets. + +00:14:32.890 --> 00:14:34.240 +Like PipeSex used to be a free conference. + +00:14:34.820 --> 00:14:37.759 +And basically when you don't charge anything for tickets, one of the lessons that we learned + +00:14:37.780 --> 00:14:40.080 +is that people will just sign up for a ticket and then not show up. + +00:14:40.080 --> 00:14:42.300 +And then your catering gets all kind of out of whack. + +00:14:42.780 --> 00:14:45.000 +So even charging just a little bit of money, like five bucks, + +00:14:45.000 --> 00:14:47.080 +I think we charge like five dollars for our network event. + +00:14:47.340 --> 00:14:48.640 +It's not because I need the five dollars. + +00:14:48.760 --> 00:14:50.800 +I mean, I've told I've told my we spend, I think, + +00:14:50.920 --> 00:14:52.200 +thirty dollars per person on the food. + +00:14:52.440 --> 00:14:55.880 +It's like it's to make sure that you have a little bit of like skin in the game + +00:14:55.920 --> 00:14:58.240 +to make sure that you show up so I can get an accurate headcount + +00:14:58.420 --> 00:14:59.680 +for the for the for the catering. + +00:15:00.040 --> 00:15:01.960 +So we don't blow the budget by ten thousand dollars. + +00:15:02.200 --> 00:15:03.720 +Like I'm not envious of that. + +00:15:04.320 --> 00:15:06.760 +I certainly it's easy to just check a checkbox. + +00:15:07.180 --> 00:15:07.980 +Yeah, I'm interested in food. + +00:15:07.980 --> 00:15:08.360 +Why not? + +00:15:08.560 --> 00:15:09.020 +Yeah. + +00:15:09.340 --> 00:15:10.140 +If I come. + +00:15:10.480 --> 00:15:10.540 +Yeah. + +00:15:10.960 --> 00:15:11.100 +Exactly. + +00:15:13.120 --> 00:15:15.920 +This portion of Talk Python To Me is brought to you by the folks at Posit. + +00:15:16.500 --> 00:15:19.500 +Posit has made a huge investment in the Python community lately. + +00:15:20.100 --> 00:15:25.540 +Known originally for RStudio, they've been building out a suite of tools and services for Team Python. + +00:15:26.760 --> 00:15:31.160 +Today, I want to focus on hosting your Python-based data science workloads. + +00:15:31.560 --> 00:15:37.860 +This includes dashboards, reports, plots, interactive web apps, all the way to custom Flask and Django apps. + +00:15:38.500 --> 00:15:39.780 +Their service is Posit Connect. + +00:15:40.600 --> 00:15:45.220 +Posit Connect makes it easy for data scientists to share work built with Python code. + +00:15:46.040 --> 00:15:52.760 +If you have a streamlet app, Dash, dashboard, Plotly interactive plot, a FastAPI service, or even a Quarto report, + +00:15:53.380 --> 00:15:58.760 +just give Posit Connect the code it needs to maintain the asset and Connect automatically does the rest. + +00:15:59.320 --> 00:16:02.760 +Connect will manage your APIs and serve your interactive apps for you. + +00:16:03.280 --> 00:16:07.340 +And if you want, you can update your reports and dashboards on a scheduled basis. + +00:16:07.820 --> 00:16:08.260 +That's right. + +00:16:08.600 --> 00:16:13.500 +No need to explain to the stakeholders why that dashboard or plot stopped updating last week. + +00:16:14.260 --> 00:16:18.900 +You get a focus on your data science and leveraging your skill set while Connect makes you look good, + +00:16:19.380 --> 00:16:20.820 +keeping your code running and private. + +00:16:21.640 --> 00:16:24.660 +With Connect, you get a private URL on your Connect server, + +00:16:25.220 --> 00:16:28.200 +ensuring that your asset is continuously available to your shareholders. + +00:16:29.080 --> 00:16:32.280 +And you can control which users have access to the asset. + +00:16:32.860 --> 00:16:36.740 +Let Posit Connect handle the delivery and DevOps involved in sharing your work. + +00:16:37.180 --> 00:16:38.560 +You focus on what you do best. + +00:16:39.420 --> 00:16:44.600 +So if you work on a data science team, you owe it to you and your org to check out Posit Connect. + +00:16:45.200 --> 00:16:51.140 +Visit talkpython.fm/connect today and get a three-month free trial to see if it's a good fit. + +00:16:51.660 --> 00:16:53.660 +That's talkpython.fm/connect. + +00:16:54.140 --> 00:16:55.860 +The link is in your podcast player's show notes. + +00:16:56.560 --> 00:16:58.820 +Thank you to Posit for supporting Talk Python To Me. + +00:16:59.960 --> 00:17:00.480 +out in the audience. + +00:17:01.060 --> 00:17:01.820 +Toon Army says, + +00:17:02.360 --> 00:17:02.440 +yay, + +00:17:02.620 --> 00:17:03.380 +PyTexas Foundation. + +00:17:03.620 --> 00:17:04.240 +That's pretty awesome. + +00:17:04.640 --> 00:17:04.780 +Yeah. + +00:17:05.120 --> 00:17:05.380 +All right. + +00:17:05.740 --> 00:17:05.900 +Well, + +00:17:06.300 --> 00:17:07.040 +let's talk. + +00:17:07.480 --> 00:17:07.839 +Emporal. + +00:17:08.199 --> 00:17:08.360 +Now, + +00:17:08.660 --> 00:17:09.480 +I came across this + +00:17:09.490 --> 00:17:10.140 +a few weeks ago + +00:17:10.720 --> 00:17:11.120 +from this, + +00:17:11.900 --> 00:17:12.040 +you know, + +00:17:12.140 --> 00:17:12.520 +I think this, + +00:17:12.699 --> 00:17:12.880 +ironically, + +00:17:12.930 --> 00:17:13.600 +I think this might be + +00:17:13.699 --> 00:17:14.120 +the blue sky + +00:17:14.270 --> 00:17:15.319 +that I came across it on. + +00:17:15.740 --> 00:17:16.100 +Interesting. + +00:17:17.680 --> 00:17:17.880 +No. + +00:17:18.160 --> 00:17:18.280 +Okay. + +00:17:18.380 --> 00:17:18.819 +It was on X. + +00:17:19.120 --> 00:17:19.839 +There's still apparently + +00:17:19.949 --> 00:17:20.740 +stuff that happens on there. + +00:17:22.100 --> 00:17:23.100 +I miss Twitter. + +00:17:23.500 --> 00:17:23.839 +I do too. + +00:17:23.920 --> 00:17:24.160 +And, you know, + +00:17:24.439 --> 00:17:25.740 +people like post weird + +00:17:26.579 --> 00:17:27.280 +comments on, + +00:17:27.530 --> 00:17:28.860 +like reviews + +00:17:28.880 --> 00:17:35.640 +like oh you know they said they don't like twitter so they must be whatever like no just that used to + +00:17:35.640 --> 00:17:39.780 +be so active and it's not so active like put put aside everything else it used to be like you could + +00:17:39.780 --> 00:17:43.940 +have great conversation you still can but it's so much less than it used to it's an easy it's an easy + +00:17:44.140 --> 00:17:49.000 +explanation the signal to noise ratio is completely messed up now yes it used to be such a much better + +00:17:49.140 --> 00:17:52.740 +place because everybody was posting there and there wasn't as much noise and now the signal to noise + +00:17:52.880 --> 00:17:57.440 +ratio makes it almost unintelligible to be able to find anything of any use and that's the number + +00:17:57.460 --> 00:18:01.120 +one problem with it. And you pay for attention and there's six other places like you said. So anyway, + +00:18:01.190 --> 00:18:08.000 +so I found this post from, we'll work it backwards, from Pietro. It says, people aren't talking enough + +00:18:08.220 --> 00:18:14.160 +about how most of OpenAI's tech stack runs on Python, which I thought was a super cool post. + +00:18:14.190 --> 00:18:20.520 +It comes from the Pragmatic Engineer newsletter and it talks about how most of the product's code + +00:18:20.520 --> 00:18:26.280 +is written in Python, uses FastAPI, C for certain parts. And then, so all that stuff made, I'm like, + +00:18:26.460 --> 00:18:32.180 +yep, yep. Temporal, use for asynchronous workflows. I'm like, what is Temporal? And then I think you + +00:18:32.320 --> 00:18:36.320 +sent me a message and said, hey, I would love to talk to you about Temporal. And I looked at him, + +00:18:36.440 --> 00:18:42.160 +yeah, this is super cool. So that's how I learned about you guys, which I thought was pretty neat. + +00:18:42.680 --> 00:18:51.240 +And yeah, let's talk about it. What is Temporal? Yeah. So Temporal is essentially what we call a + +00:18:51.260 --> 00:18:57.440 +durable execution platform. And it's durable execution is kind of a new term or a new like, + +00:18:57.620 --> 00:19:03.160 +like field within the zeitgeist, as you would call it. And we're kind of going with like what + +00:19:03.220 --> 00:19:07.720 +we're calling crash proof execution. And the way that we kind of talk about it now is like, + +00:19:08.160 --> 00:19:13.660 +essentially, it handles the failure state of your of your applications and ensures that your code + +00:19:13.660 --> 00:19:20.340 +will continue execution regardless of the failures that it encounters. So say, for example, you have + +00:19:20.360 --> 00:19:25.620 +a application that's making a couple calls out to a microservice. Okay. And that microservice goes + +00:19:25.800 --> 00:19:30.780 +temporarily down. You as the developer would have to traditionally write a whole bunch of logic + +00:19:31.380 --> 00:19:34.860 +around handling that failure. Okay. So we have to detect that the failure has happened. + +00:19:35.460 --> 00:19:41.100 +What type of error did we receive? Now, what do we do? Do we back off and retry? Do we decide that + +00:19:41.140 --> 00:19:44.520 +this failure is non-retriable and we just don't do it? Like the difference between we're not + +00:19:44.760 --> 00:19:49.720 +authenticated versus a 404, those are completely different failure handling modes. And then, + +00:19:49.860 --> 00:19:52.680 +So there's all of this kind of logic that you have to build in to handle failure. + +00:19:53.660 --> 00:19:56.340 +Temporal basically abstracts this away from you. + +00:19:56.340 --> 00:19:58.920 +So whenever you have like a function call or a method call in Python, + +00:19:59.440 --> 00:20:04.120 +when implemented with Temporal, you automatically get like retries, for example, by default. + +00:20:04.400 --> 00:20:08.580 +So you get this like declarative policy that happens as a default policy, + +00:20:08.860 --> 00:20:13.880 +and it will automatically retry your application or your call until eventually it comes back online. + +00:20:14.100 --> 00:20:16.800 +Because let's be honest, in most distributed systems like this, + +00:20:17.160 --> 00:20:18.900 +those are most of the time intermittent failures. + +00:20:19.440 --> 00:20:22.300 +Like a microservice going offline, are you being rate limited? + +00:20:22.800 --> 00:20:27.040 +Those are usually fixable with a little bit of time and some retries. + +00:20:27.440 --> 00:20:30.400 +Now, there are other cases where they're not, but that's like the default policy. + +00:20:30.600 --> 00:20:34.240 +That's the default use case or the default for retries alone. + +00:20:34.680 --> 00:20:35.800 +And then there's a lot of other cases. + +00:20:36.000 --> 00:20:41.100 +So Temporal maintains the application state of your application. + +00:20:41.360 --> 00:20:46.360 +And so say you have like a Python application that has 10 steps in it and you get through + +00:20:46.340 --> 00:20:51.000 +step five and then the python application were to crash for some reason out of memory uh maybe + +00:20:51.460 --> 00:20:55.500 +your kubernetes pod got descheduled something like this and production happens happens all the time + +00:20:55.640 --> 00:21:02.280 +this is as a former sre this was this was the life that i lived for years um yeah even hey + +00:21:02.640 --> 00:21:07.000 +there's a security patch for the Linux server that's the yeah running this and we got to restart + +00:21:07.240 --> 00:21:13.940 +it and like uh we have to reboot yeah and now yes it's not wrong but like stuff is going on exactly + +00:21:14.080 --> 00:21:18.740 +So now when you have to deal with that, you're like, okay, so do we let everything get through and risk the attack surface? + +00:21:19.270 --> 00:21:21.780 +Or then you have to make the calculus. + +00:21:21.870 --> 00:21:26.480 +Do we just kill all the processes and then wait for, and then restart them? + +00:21:26.600 --> 00:21:28.340 +What's the cost on the restart? + +00:21:28.430 --> 00:21:29.860 +And then redoing all of that logic. + +00:21:29.910 --> 00:21:31.340 +Did they make any rights to the database? + +00:21:31.700 --> 00:21:33.480 +Do we have to worry about cleaning all that? + +00:21:33.680 --> 00:21:35.140 +There's all of this conversation. + +00:21:35.480 --> 00:21:39.760 +Since Temporal maintains that application state, that becomes a lot simpler. + +00:21:39.890 --> 00:21:42.560 +Because as we, I think we kind of alluded to it, but I'll state it outright. + +00:21:42.960 --> 00:21:47.600 +when that when you when that application crashes, and in a typical application, you have to start + +00:21:47.670 --> 00:21:51.920 +back over from the beginning. And you have to replay every you basically re execute everything + +00:21:51.930 --> 00:21:56.400 +you did unless you were have some like advanced sort of like event sourcing style model where you + +00:21:56.460 --> 00:22:00.960 +are keeping track of it. Temporal does this by default. So it maintains the event history and + +00:22:01.140 --> 00:22:05.820 +what's called an event history of your application. And what that does is every time a basically + +00:22:06.020 --> 00:22:10.920 +creates checkpoints, every time a function executes, it stores the result of it. And if that application + +00:22:10.940 --> 00:22:14.940 +were to crash, it will reschedule it onto another worker. That's what they're called in temporal. + +00:22:15.040 --> 00:22:19.920 +And we'll get into that in a minute within your fleet, re reconstruct the state of that application + +00:22:20.320 --> 00:22:24.340 +up into that point of failure, and then continue onward as if the failure had never happened. + +00:22:24.740 --> 00:22:29.140 +And more often than not, this happens without the developer even being aware of it. Like no, + +00:22:29.340 --> 00:22:33.180 +no alarms will go off unless you personally do them, because this is the design of the system. + +00:22:33.400 --> 00:22:37.260 +It's to ensure that in spite of failure, your application will continue executing. + +00:22:37.640 --> 00:22:42.120 +Yeah, that's awesome. And there's a bunch of neat ways in which that happens, which we'll talk about. + +00:22:42.520 --> 00:22:53.780 +But I think one mental model people should maybe think about when they consider these is it centralizes a lot of the error handling and the retry code and all of that stuff. + +00:22:54.080 --> 00:22:57.980 +So a lot of your application becomes simpler, not more complex. + +00:22:58.640 --> 00:23:03.680 +Exactly. Yes. That's actually one of the main feedback things that we get from people is, you know, all of this. + +00:23:04.220 --> 00:23:04.980 +There's so many mechanisms. + +00:23:05.200 --> 00:23:06.720 +There's so many actually like patterns + +00:23:06.950 --> 00:23:10.280 +that we have now built to handle all of these kind of things. + +00:23:11.100 --> 00:23:12.700 +Event sourcing, event-driven architecture, + +00:23:12.960 --> 00:23:15.360 +saga pattern, all of these different like CQRS, + +00:23:15.520 --> 00:23:17.140 +like all of these different distributed systems patterns + +00:23:17.740 --> 00:23:19.840 +that exist to handle all of these things. + +00:23:20.080 --> 00:23:22.760 +And Temporal basically abstracts away a lot of them + +00:23:23.100 --> 00:23:24.400 +and you get them basically for free + +00:23:24.410 --> 00:23:25.320 +out of the box with the project. + +00:23:25.800 --> 00:23:27.620 +And it does make your code a lot simpler. + +00:23:27.730 --> 00:23:29.860 +It makes it a lot more contained + +00:23:30.280 --> 00:23:32.100 +so you can walk through it straight down. + +00:23:32.780 --> 00:23:37.220 +Basically, as like almost as like it's a workflow, you know, the core primitive of temporal is called a workflow. + +00:23:37.430 --> 00:23:41.400 +We don't tend to like to refer to ourself as a workflow engine, but some people will say that. + +00:23:41.450 --> 00:23:43.460 +And I typically don't correct them on it. + +00:23:44.080 --> 00:23:46.800 +If that's if that's what it takes for people to understand, that's what it is. + +00:23:46.940 --> 00:23:48.920 +Fine. But durable execution is the actual phrase. + +00:23:50.220 --> 00:23:54.260 +It's a meaty topic. It's a it's kind of hard for people to like kind of wrap their heads around it. + +00:23:54.400 --> 00:23:59.140 +So I'm like, let's get you to understanding it and then we'll correct the little the little tidbits here and there. + +00:23:59.560 --> 00:23:59.960 +Yeah, excellent. + +00:24:00.360 --> 00:24:07.060 +So, you know, people are probably familiar with Hennick's stamina or with tenacity, which + +00:24:07.060 --> 00:24:12.320 +is interesting, where you might put a decorator onto a function and say, hey, if this fails, + +00:24:12.760 --> 00:24:14.540 +instead of just crashing, try it again. + +00:24:15.180 --> 00:24:15.360 +Yeah. + +00:24:15.580 --> 00:24:18.240 +Maybe a few times, maybe with exponential back off. + +00:24:18.620 --> 00:24:25.100 +And this is kind of a single thread of execution type of form of durability a little bit, right? + +00:24:25.240 --> 00:24:36.360 +Where it's like this function now becomes somewhat more reliable based on like maybe the service comes back or maybe your network comes back or whatever it happened to be that caused this particular example. + +00:24:36.860 --> 00:24:40.200 +But with temporal, it's a lot broader, right? + +00:24:40.200 --> 00:24:45.140 +Like every step, it saves the state. + +00:24:45.500 --> 00:24:48.780 +If something goes wrong, it can basically resume. + +00:24:48.910 --> 00:24:52.800 +I guess maybe this resumable idea is like one of the big difference, right? + +00:24:52.940 --> 00:24:56.300 +It's like, let's take our reboot the server example. + +00:24:56.700 --> 00:25:00.540 +Let's suppose we got some service that we work on that our app depends upon. + +00:25:00.910 --> 00:25:01.700 +We had to reboot it. + +00:25:01.960 --> 00:25:02.180 +Now what? + +00:25:02.460 --> 00:25:02.820 +It happens. + +00:25:03.500 --> 00:25:03.580 +Yeah. + +00:25:03.770 --> 00:25:06.620 +So, I mean, yeah, you have a service that our app depends on. + +00:25:06.720 --> 00:25:07.240 +We had to reboot it. + +00:25:07.250 --> 00:25:13.300 +So if the service, so we're assuming our service is on in like an external machine and it's + +00:25:13.480 --> 00:25:13.700 +calling out. + +00:25:13.700 --> 00:25:14.260 +Yeah, for some reason. + +00:25:14.680 --> 00:25:14.820 +Yeah. + +00:25:14.830 --> 00:25:18.380 +Or maybe it's a Docker container and we rebuilt the Docker container. + +00:25:18.430 --> 00:25:19.560 +It takes five seconds to start. + +00:25:19.700 --> 00:25:20.020 +Something like that. + +00:25:20.100 --> 00:25:20.200 +Yeah. + +00:25:20.430 --> 00:25:22.220 +So our service is, you know, calling out to it. + +00:25:22.300 --> 00:25:24.380 +it will basically retry until that comes back online. + +00:25:24.590 --> 00:25:27.160 +Now, if we were to reboot our service, + +00:25:27.190 --> 00:25:28.220 +say our service was running + +00:25:28.250 --> 00:25:30.080 +and we rebooted the container that contained it, + +00:25:30.810 --> 00:25:33.120 +it would basically, if we're in a Kubernetes system, + +00:25:33.190 --> 00:25:36.220 +the Kubernetes scheduler would probably reschedule it + +00:25:36.380 --> 00:25:39.020 +onto another node within the pod. + +00:25:39.840 --> 00:25:41.580 +And then it would reconstruct it. + +00:25:41.580 --> 00:25:42.600 +And what happens is there's actually, + +00:25:43.400 --> 00:25:45.360 +the Temporal exists as a kind of + +00:25:47.040 --> 00:25:49.380 +orchestrator executor model, like a service worker model. + +00:25:50.100 --> 00:25:51.480 +So the service maintains the history. + +00:25:51.940 --> 00:25:54.580 +So what will happen is whenever that new service comes online, + +00:25:55.000 --> 00:25:56.780 +or whenever the new execution comes online, + +00:25:57.760 --> 00:25:59.960 +we've rebooted it, it got rescheduled, it comes online, + +00:26:00.080 --> 00:26:02.480 +it will stream the old history from the service + +00:26:02.940 --> 00:26:03.860 +and then reconstruct it, + +00:26:04.140 --> 00:26:05.520 +basically going step by step through it. + +00:26:06.840 --> 00:26:10.240 +And so like the function A that was executed, + +00:26:10.640 --> 00:26:13.140 +the input and output values are stored in that event history. + +00:26:13.400 --> 00:26:14.480 +So the output value, it's like, + +00:26:14.480 --> 00:26:17.060 +oh, we've successfully completed execution of function A, + +00:26:17.480 --> 00:26:20.560 +store that in variable foo, and then continue onward. + +00:26:20.640 --> 00:26:27.740 +And then we can continue reconstructing that service up until it gets to the point where it's like, okay, we have no more events in the history about what happened. + +00:26:27.990 --> 00:26:31.040 +So now we know we've, you know, we have reconstructed successfully. + +00:26:31.410 --> 00:26:35.100 +Now we continue forward and execute as if nothing had happened. + +00:26:35.420 --> 00:26:35.500 +Yeah. + +00:26:35.790 --> 00:26:42.520 +So in your non-durable execution code, you might have try, accept, do something else. + +00:26:42.930 --> 00:26:48.380 +You might have stuff like stamina or tenacity where you're like, okay, we're going to try this again. + +00:26:48.840 --> 00:26:51.080 +There's a lot of that kind of code that you write. + +00:26:51.460 --> 00:26:53.760 +And in this world, you could just say like top to bottom, + +00:26:54.380 --> 00:26:57.100 +write the happy path, which is great. + +00:26:57.180 --> 00:26:59.420 +And then what happens is temporal says, + +00:26:59.540 --> 00:27:01.040 +okay, I tried to run this, it failed. + +00:27:01.160 --> 00:27:03.640 +So we'll reschedule it with a back off + +00:27:03.940 --> 00:27:05.780 +or something along those lines, right? + +00:27:06.060 --> 00:27:09.120 +So you can basically not deal with a lot of this + +00:27:09.240 --> 00:27:11.560 +and get visibility into the errors and the state of flow + +00:27:12.000 --> 00:27:14.200 +by letting the orchestrator manage that, right? + +00:27:14.540 --> 00:27:14.840 +Yes. + +00:27:15.100 --> 00:27:19.540 +And I'm covering like the top 10% of all the different features. + +00:27:19.780 --> 00:27:24.780 +There are so many interesting other features that Temporal provides as an ecosystem. + +00:27:25.250 --> 00:27:28.580 +So like one of the other really neat ones is that we provide a really simple way for + +00:27:28.630 --> 00:27:31.000 +things to do a human in the loop interaction. + +00:27:31.470 --> 00:27:33.780 +So you can very easily send in what's called a signal. + +00:27:34.170 --> 00:27:38.860 +So basically sending in data into a running execution and then basically doing some processing + +00:27:38.930 --> 00:27:39.200 +on it. + +00:27:39.200 --> 00:27:43.660 +So you're waiting on a confirmed signal from someone like your application is doing something + +00:27:43.740 --> 00:27:44.440 +you're waiting on to confirm. + +00:27:44.820 --> 00:27:47.120 +you can send that in directly into the workflow. + +00:27:47.490 --> 00:27:48.820 +And then that will basically be, + +00:27:49.180 --> 00:27:50.760 +that's persisted again within the events, + +00:27:50.950 --> 00:27:52.060 +within the event history. + +00:27:52.560 --> 00:27:54.540 +So if it crashes after that confirmation, + +00:27:54.820 --> 00:27:57.000 +that confirmation is stored as well. + +00:27:57.280 --> 00:27:58.140 +So you have that, + +00:27:58.170 --> 00:28:00.020 +you have the ability to do long running schedules. + +00:28:00.800 --> 00:28:02.280 +So there's some cron syntax, + +00:28:02.500 --> 00:28:04.180 +like what are called schedules in Temporal. + +00:28:04.320 --> 00:28:06.380 +There's so many different features in Temporal + +00:28:07.180 --> 00:28:08.340 +that are just really neat + +00:28:09.200 --> 00:28:10.640 +and can solve a lot of the problems + +00:28:11.100 --> 00:28:12.040 +that you're trying to do. + +00:28:12.440 --> 00:28:13.940 +And scaling becomes super easy as well. + +00:28:14.080 --> 00:28:16.340 +So like you want to scale, just add more workers to the fleet. + +00:28:16.820 --> 00:28:21.780 +That's the easiest thing you can do is just add more of these workers and they basically + +00:28:21.890 --> 00:28:23.940 +can be executed across all of your different fleets. + +00:28:23.940 --> 00:28:27.040 +So the scaling story is super awesome as well. + +00:28:28.440 --> 00:28:30.940 +This portion of Talk Python To Me is brought to you by PyBay. + +00:28:31.420 --> 00:28:36.160 +PyBay is an annual conference gathering of Pythonistas put on by the Bay Area Python + +00:28:36.480 --> 00:28:36.900 +Association. + +00:28:37.440 --> 00:28:42.400 +This year is returning to the UCSF Mission Bay Conference Center in San Francisco, California. + +00:28:43.000 --> 00:28:46.860 +It's a one-day conference on October 18th, 2025. + +00:28:47.580 --> 00:28:50.040 +I've spoken there previously and had a great time attending. + +00:28:50.740 --> 00:28:51.660 +And there's a bonus. + +00:28:52.340 --> 00:28:54.160 +Talk Python is sponsoring the conference. + +00:28:54.660 --> 00:29:00.060 +Every attendee gets a special conference bundle of paid courses for free as a conference gift. + +00:29:00.640 --> 00:29:06.060 +Plus, we'll be giving away a complete set of training materials for a dev team of some lucky attendees. + +00:29:06.660 --> 00:29:09.159 +So if you want to connect with the Python people of San Francisco + +00:29:09.160 --> 00:29:12.120 +and go home with courses from Talk Python, + +00:29:12.440 --> 00:29:13.060 +check out PyBay. + +00:29:13.840 --> 00:29:14.620 +Please use our link. + +00:29:14.800 --> 00:29:16.780 +It's talkpython.fm/PyBay. + +00:29:16.980 --> 00:29:18.720 +The link is your podcast player show notes. + +00:29:19.040 --> 00:29:20.540 +Thanks to PyBay for supporting the show. + +00:29:21.380 --> 00:29:21.860 +We'll get into it. + +00:29:21.920 --> 00:29:25.080 +There's definitely some examples of large scale use cases, + +00:29:25.400 --> 00:29:28.200 +you know, high frequency use cases of these things. + +00:29:28.640 --> 00:29:29.980 +But going back to the timing, + +00:29:30.420 --> 00:29:33.540 +you know, what if your user onboarding looks more like, + +00:29:33.800 --> 00:29:34.820 +I guess you could be real simple. + +00:29:34.840 --> 00:29:36.760 +You could say in order to create a user account, + +00:29:36.960 --> 00:29:42.200 +I have to first have them create and enter their username and email or their email and their + +00:29:42.370 --> 00:29:46.640 +password. And then I'm going to send them an email and they got to take an action based on that, + +00:29:46.860 --> 00:29:51.440 +right? That would be pretty common, like a long running type of thing you could consider. But + +00:29:51.720 --> 00:29:56.540 +for some of these systems, it's like, and prove who you are and upload a document. + +00:29:57.660 --> 00:30:01.120 +That's a picture of your ID and somebody will look at it and go, yeah, that looks real enough. + +00:30:01.720 --> 00:30:05.940 +And they check a box or, you know, those kinds of much longer onboarding things, + +00:30:06.120 --> 00:30:08.120 +Something like that could be modeled with temporal pretty easily. + +00:30:08.150 --> 00:30:08.700 +It sounds like. + +00:30:09.020 --> 00:30:09.120 +Yeah. + +00:30:09.230 --> 00:30:12.200 +So long running workflows are the other, that, that was the feature I couldn't + +00:30:12.210 --> 00:30:12.920 +remember, which was timers. + +00:30:12.950 --> 00:30:14.300 +And I have no idea why that left my mind. + +00:30:14.360 --> 00:30:15.200 +It's one of my favorite features. + +00:30:15.650 --> 00:30:19.980 +long running workflows are one of the like amazing use cases of, of temporal. + +00:30:20.050 --> 00:30:24.420 +So because everything is maintained in this, in that state and basically crashing + +00:30:24.660 --> 00:30:27.540 +doesn't really matter because we can just reconstruct the state, you can have + +00:30:27.860 --> 00:30:32.960 +workflows or you can have your executions that can last for days, weeks, years. + +00:30:33.500 --> 00:30:36.840 +This is kind of what we know, we kind of known as the entity workflow pattern. + +00:30:37.030 --> 00:30:41.060 +So essentially like a user who's going through an onboarding process, you know, like you just + +00:30:41.060 --> 00:30:45.800 +said, I think the identity workflow process is actually one of our exact sample applications. + +00:30:46.280 --> 00:30:49.680 +So, you know, like you're right, they lay sign up and they have to upload some forms of + +00:30:49.790 --> 00:30:49.880 +ID. + +00:30:50.100 --> 00:30:51.160 +Someone has to check it. + +00:30:51.210 --> 00:30:53.640 +It has to go through maybe a background check process and all of that. + +00:30:54.640 --> 00:30:55.860 +That's a long running workflow. + +00:30:55.890 --> 00:30:56.760 +That could take days. + +00:30:56.890 --> 00:30:59.680 +That could take weeks, depending on what kind of background check you're getting done. + +00:31:00.500 --> 00:31:07.020 +Temporal can suspend its state for however long it wants, you know, and we can guarantee that it will come back online. + +00:31:07.250 --> 00:31:13.020 +So the interesting thing, and whenever you ever, if you ever see Temporal at a booth at a conference, we were at PyCon this year, for example. + +00:31:13.310 --> 00:31:21.320 +Our code sample on our booth has a really interesting statement that always catches people's eye, and it's a little mini workflow, and it's sending emails. + +00:31:21.570 --> 00:31:23.500 +And what it does is it says sleep for 30 days. + +00:31:24.040 --> 00:31:30.320 +And nobody in their right mind would actually write a sleep for 30 days in code and expect it to actually function. + +00:31:30.660 --> 00:31:33.740 +100% works exactly the way you would expect it to in Temporal. + +00:31:33.740 --> 00:31:37.080 +And we can guarantee that it works because of the way that Temporal is architected. + +00:31:37.160 --> 00:31:41.360 +Those timers basically exist on the servers on this on this temporal service side. + +00:31:41.660 --> 00:31:46.260 +And they just get basically your workflow just gets scheduled to resume after the timer has fired. + +00:31:46.700 --> 00:31:51.440 +So you can guarantee that long running workflows will complete exactly the way that you expect them to. + +00:31:52.160 --> 00:31:54.740 +So, yeah, long running workflows, amazing use case for Temporal. + +00:31:55.020 --> 00:31:55.760 +Yeah, it sounds incredible. + +00:31:56.080 --> 00:31:57.540 +Now, I do want to dive actually in. + +00:31:57.660 --> 00:32:00.360 +It's super interesting how y'all made this happen in Python. + +00:32:00.470 --> 00:32:03.400 +But I do want to just maybe talk about the scale. + +00:32:03.980 --> 00:32:07.200 +Like if I were to run Temporal, now you guys have a cloud. + +00:32:07.380 --> 00:32:09.420 +So I guess stepping just a bit back. + +00:32:09.720 --> 00:32:10.880 +This is MIT licensed. + +00:32:11.340 --> 00:32:11.420 +Yes. + +00:32:11.500 --> 00:32:13.300 +But you also have pricing. + +00:32:13.840 --> 00:32:14.060 +Yes. + +00:32:14.400 --> 00:32:18.820 +Before we go into what I was going to talk about, let's talk about that difference there, that contrast. + +00:32:18.940 --> 00:32:19.060 +Yeah. + +00:32:19.200 --> 00:32:19.720 +What's the story? + +00:32:20.200 --> 00:32:21.840 +So Temporal is 100% open source. + +00:32:22.660 --> 00:32:23.120 +MIT licensed. + +00:32:23.200 --> 00:32:26.100 +And so there's the temporal service and there's the temporal SDKs. + +00:32:26.380 --> 00:32:31.460 +Every single one of our temporal SDKs are and the service and everything is MIT licensed + +00:32:32.420 --> 00:32:36.580 +forever and always that our founders are big, big fans of that. + +00:32:36.780 --> 00:32:40.560 +The only SDK that is not is the Java SDK, which is Apache 2. + +00:32:40.960 --> 00:32:44.500 +And if you know anything about open source licensing, there is basically a dependency + +00:32:44.940 --> 00:32:46.880 +further up the tree that was Apache 2. + +00:32:47.000 --> 00:32:50.560 +And you're not allowed to downgrade licensing if you ever make a derivative. + +00:32:50.820 --> 00:32:52.200 +So that's the only reason that one is. + +00:32:52.240 --> 00:32:54.880 +but every one of our other licenses is MIT licensed. + +00:32:55.020 --> 00:32:57.940 +So you could run Temporal open source and be fine. + +00:32:58.120 --> 00:33:00.480 +The thing that we have found is that at scale, + +00:33:00.700 --> 00:33:03.380 +the service itself is challenging + +00:33:03.440 --> 00:33:05.460 +and requires large SRE teams to run. + +00:33:06.800 --> 00:33:07.880 +Because essentially what we're doing + +00:33:08.000 --> 00:33:10.480 +is we're offering distributed systems as a service. + +00:33:10.720 --> 00:33:12.360 +We're offering reliability as a service. + +00:33:12.820 --> 00:33:14.100 +Just because we have abstracted + +00:33:14.240 --> 00:33:16.640 +these problems away from someone + +00:33:16.860 --> 00:33:17.640 +does not mean that the price + +00:33:17.740 --> 00:33:18.880 +does not have to be paid by someone. + +00:33:18.920 --> 00:33:20.200 +And I'm talking about the metaphorical price, + +00:33:20.340 --> 00:33:21.000 +not the dollar price. + +00:33:22.440 --> 00:33:27.360 +someone still has to ensure that that database stays up essentially and that your things are + +00:33:27.390 --> 00:33:32.940 +still getting scheduled and that the network is up and all of these. So it's super complex to do + +00:33:33.080 --> 00:33:37.340 +that. And you can run it. So you can run the temporal service locally. You can run the temporal + +00:33:37.480 --> 00:33:41.960 +workers locally. Everything you run is still local. The pricing model for temporal is just the temporal + +00:33:42.200 --> 00:33:47.580 +service part, which is run in temporal cloud. So there's a weird misnomer around cloud, which is + +00:33:47.560 --> 00:33:51.340 +like cloud always assumes that we run everything for you. Temporal cloud is different. Temporal + +00:33:51.620 --> 00:33:57.040 +cloud only runs the temporal service, your workers, where your code executes is always run, + +00:33:57.500 --> 00:34:01.280 +at least by for now until who knows if another product will ever come out. I don't know. + +00:34:02.120 --> 00:34:06.960 +Is run by you on you in your data center on your machines. So your code, your execution run by you, + +00:34:07.300 --> 00:34:11.460 +that service, that brain, the orchestrator, that's the part you could pay temporal cloud for. + +00:34:12.260 --> 00:34:15.439 +Right. That's the part that does the orchestration, the part that handles the + +00:34:15.460 --> 00:34:18.780 +the failure and retries and that kind of stuff, right? + +00:34:18.909 --> 00:34:22.260 +Yeah. Well, Ted, it's the part that actually the failure retries is handled by the state + +00:34:22.370 --> 00:34:26.080 +machines that are built in the SDK. It's the part that basically maintains the event history. It + +00:34:26.139 --> 00:34:31.960 +maintains the communication mechanisms and it is the orchestrator. Yeah. So, but. + +00:34:32.399 --> 00:34:37.780 +So if we use the cloud, I could like reboot my local data center machine or even my local data + +00:34:37.870 --> 00:34:43.760 +center. I mean, my version of the cloud, you know, wherever digital ocean or whatever. And when it + +00:34:43.620 --> 00:34:49.220 +comes back it'll sort of resume along like you guys will see that it's back and then yeah start + +00:34:49.379 --> 00:34:53.500 +running work on something like that that's what i was getting at technically yes so fun fun + +00:34:53.879 --> 00:34:58.380 +fun fact about it temporal the again the architecture of this is so brilliant and we could + +00:34:58.380 --> 00:35:02.220 +get so touched in the weeds about the temporal service does not actually know about any of the + +00:35:02.300 --> 00:35:07.100 +workers that are running it's always a call out model so your your your machines would come back + +00:35:07.300 --> 00:35:11.359 +online know that they have things they need to do they would basically start listening back + +00:35:11.380 --> 00:35:13.800 +Everything that happens in Temporal listens across task queues. + +00:35:14.180 --> 00:35:15.260 +So they would all come back online. + +00:35:15.480 --> 00:35:16.540 +They would start listening again. + +00:35:17.000 --> 00:35:19.660 +And then they would see that there's still work in the task queues, which the service + +00:35:19.940 --> 00:35:20.040 +maintains. + +00:35:20.320 --> 00:35:23.940 +But all the service does, it goes, oh, someone requested that I do something. + +00:35:24.020 --> 00:35:25.220 +And then it puts it on the task queue. + +00:35:25.300 --> 00:35:26.320 +And then the workers handle it. + +00:35:26.640 --> 00:35:31.480 +The true magic of Temporal lies within the state machines that are built within the SDKs. + +00:35:31.840 --> 00:35:33.900 +But they cannot function without that orchestrator service. + +00:35:34.720 --> 00:35:38.440 +I remember when I first started working on the courses. + +00:35:38.720 --> 00:35:42.540 +like my primary role is writing courses for Temporal. + +00:35:42.540 --> 00:35:44.440 +And if you go on to our Temporal Learn site, + +00:35:44.550 --> 00:35:46.200 +you can find a lot of our, + +00:35:46.940 --> 00:35:48.360 +I think it's learn.temporal.io. + +00:35:48.800 --> 00:35:49.920 +You can find a lot of our courses. + +00:35:50.720 --> 00:35:52.020 +I wrote the Java and the Python ones. + +00:35:53.480 --> 00:35:54.920 +I remember when I was first writing them, + +00:35:54.930 --> 00:35:56.440 +I kept asking the SDK engineers like, + +00:35:56.440 --> 00:35:57.520 +well, how does the server do this? + +00:35:57.520 --> 00:35:58.260 +How does the server do this? + +00:35:58.260 --> 00:35:59.260 +And one of the engineers was like, + +00:35:59.280 --> 00:36:00.740 +Mason, you're giving the server too much credit. + +00:36:00.900 --> 00:36:01.560 +It's not that smart. + +00:36:02.660 --> 00:36:05.220 +It's really the state machines within the workers + +00:36:05.680 --> 00:36:06.760 +that are doing all the heavy lifting + +00:36:06.870 --> 00:36:08.060 +and the service just maintains it. + +00:36:08.140 --> 00:36:10.160 +but with the long way around, + +00:36:10.300 --> 00:36:12.200 +what the cloud service does is it maintains that database. + +00:36:12.680 --> 00:36:14.900 +So the history, without that event history, + +00:36:15.220 --> 00:36:16.120 +that is the magic piece. + +00:36:16.760 --> 00:36:18.400 +The event history is the single source of truth + +00:36:18.480 --> 00:36:19.580 +of all things that have happened, + +00:36:19.820 --> 00:36:21.660 +and that's what the service provides and maintains. + +00:36:22.080 --> 00:36:23.920 +So whether you use the cloud version of that + +00:36:24.060 --> 00:36:26.020 +or you self-host that, that is up to you. + +00:36:26.160 --> 00:36:27.180 +Now, again, if you self-host it, + +00:36:27.240 --> 00:36:28.780 +you are now responsible for maintaining it. + +00:36:28.960 --> 00:36:30.820 +You are now responsible for upgrading it, + +00:36:30.860 --> 00:36:32.220 +security patches, all of that. + +00:36:33.200 --> 00:36:34.640 +And there are multiple case studies. + +00:36:34.800 --> 00:36:37.359 +There are multiple YouTube videos on our channel + +00:36:37.380 --> 00:36:39.180 +of people who have gone the self-hosted route + +00:36:39.300 --> 00:36:41.000 +and have found that the cloud route is easier + +00:36:42.040 --> 00:36:43.100 +once they've reached a certain scale. + +00:36:44.120 --> 00:36:45.240 +So yeah, it's pretty neat. + +00:36:45.500 --> 00:36:46.300 +- Yeah, yeah, cool. + +00:36:46.740 --> 00:36:49.380 +More and more, that's the business side + +00:36:49.410 --> 00:36:51.360 +of what works for open source, right? + +00:36:51.660 --> 00:36:51.740 +- Yeah. + +00:36:51.900 --> 00:36:53.540 +- 'Cause there's something that you wanna do, + +00:36:53.880 --> 00:36:55.240 +and then there's another requirement + +00:36:55.520 --> 00:36:56.680 +that you're probably not good at, + +00:36:57.300 --> 00:36:58.600 +and you guys would be really good at, right? + +00:36:58.680 --> 00:37:00.720 +Like for example, making sure that Temporal itself + +00:37:01.599 --> 00:37:04.400 +is reliable and fast and keeps running + +00:37:04.510 --> 00:37:05.540 +and that kind of thing, right? + +00:37:05.980 --> 00:37:06.700 +- Exactly, yes. + +00:37:06.940 --> 00:37:11.120 +And I think it does also come down to, and this is one of the interesting lessons that + +00:37:11.200 --> 00:37:14.480 +I've learned throughout my career, is that like, what is your core business? + +00:37:14.720 --> 00:37:18.580 +Is your core business to run and maintain a temporal service or is your core business + +00:37:18.740 --> 00:37:20.140 +to provide a service to your customers? + +00:37:20.860 --> 00:37:24.440 +And whenever the price of running your own services outweighs what you could have just + +00:37:24.560 --> 00:37:28.840 +paid someone else to do it for, then at that point, you have to take a look at something + +00:37:28.890 --> 00:37:33.240 +and go, maybe I should just pay the provider who has the expertise, who literally all they + +00:37:33.250 --> 00:37:33.780 +do all day long. + +00:37:34.200 --> 00:37:39.400 +Because usually someone's SRE team doesn't doesn't full time dedicate to learning the ins and outs of a single product. + +00:37:39.760 --> 00:37:40.540 +Like I was an SRE. + +00:37:40.610 --> 00:37:45.060 +I managed a cloud platform for VRBO for Virgo. + +00:37:45.840 --> 00:37:48.280 +I knew there was like 12 different applications in that stack. + +00:37:48.390 --> 00:37:51.260 +And the way that I learned about each one was the one that went down that day. + +00:37:51.620 --> 00:37:55.680 +So you're constantly like figuring out with like you're learning about it as it's on fire. + +00:37:55.800 --> 00:37:56.640 +It's a terrible way to learn. + +00:37:58.060 --> 00:38:01.580 +And that's not a great way to run it to to like live your life. + +00:38:01.980 --> 00:38:08.280 +i found right why i moved into developer education um exactly there's fewer uh midnight calls in + +00:38:08.360 --> 00:38:16.180 +developer education that is the primary reason why i do this now is i i like there is a point when + +00:38:16.180 --> 00:38:20.360 +the p when pager duty goes off like one too many times and you become a developer advocate and i'm + +00:38:20.400 --> 00:38:25.800 +living proof of that you just know yeah you take it off a space style you just take it out put on + +00:38:25.780 --> 00:38:33.400 +some uh gangster music just exactly right exactly if you don't know the reference you need to make + +00:38:33.500 --> 00:38:38.200 +sure you put watching office space yeah the movie right i've driven by the list i've driven by the + +00:38:38.240 --> 00:38:41.860 +area where it was filmed it was filmed out here in austin where i live was it incredible yes + +00:38:42.400 --> 00:38:46.080 +incredible okay so why did i sort of start in that path what i want to talk about is what is + +00:38:46.520 --> 00:38:51.439 +so you guys have a cloud version which obviously does stuff in massive scale you talk about a fleet + +00:38:51.440 --> 00:38:57.780 +of these things. And by the way, at the execution level, we're probably mostly talking Go and Rust, + +00:38:57.970 --> 00:39:03.420 +just so people know, it's kind of like fast, fast innards. But Rakesh asks, like, it seems to be + +00:39:03.610 --> 00:39:07.940 +resource insensitive. Is there a lightweight option? Like what is a, what is a minimal temporal + +00:39:08.330 --> 00:39:14.400 +rather than a, you know, what Uber does or what, you know, ChatGPT does? Because those things, + +00:39:14.550 --> 00:39:18.359 +you just can't compare them, right? It's like, you're not Microsoft, you're not Google, you're + +00:39:18.280 --> 00:39:24.160 +not LinkedIn, you're not Netflix. Don't try to be them most. Yeah, it depends on what you're + +00:39:24.360 --> 00:39:27.260 +trying to accomplish. So I mean, like, I think when it comes to so there's, there's always there's + +00:39:27.280 --> 00:39:30.520 +the service and then there's the worker fleet. So let's talk. I'll talk about the service first, + +00:39:30.590 --> 00:39:33.940 +and then we'll talk about the workers. So the service comes in a handful of different flavored + +00:39:34.120 --> 00:39:40.340 +offerings. There's a local development binary that is meant to be used for development. And + +00:39:40.500 --> 00:39:46.260 +that's what I use on my local laptop all the time. It's also more than suitable for home lab stuff. + +00:39:46.400 --> 00:39:50.340 +So if you're wanting to play with this in your home lab, use the use that single binary. + +00:39:50.760 --> 00:39:54.800 +It can it's got an in memory data store, but you can have it persist to it to a SQLite database + +00:39:55.340 --> 00:39:59.860 +will get you very, very far on home lab stuff. Non prod use cases. Totally fine. + +00:40:00.040 --> 00:40:06.440 +Speaker 1: SQLite is underappreciated. People see it as like a toy thing or like I can use this while + +00:40:06.480 --> 00:40:10.760 +I'm developing like a sample, but then I don't really like you can put millions of records in + +00:40:10.860 --> 00:40:14.880 +SQLite. You can go a very long way. Speaker 1: Oh yeah. Yeah, it is. Yeah, + +00:40:14.940 --> 00:40:19.860 +it is an amazing tool it's one of my favorite tools um and then there so in reality what the + +00:40:19.900 --> 00:40:23.340 +way that the temporal service is actually built is it's actually just a single binary um but the + +00:40:23.420 --> 00:40:27.480 +thing is there are the temporal service is a conglomerate when i say the service i mean like + +00:40:27.640 --> 00:40:31.800 +all of the things together it's a conglomerate of multiple microservices that when you put them + +00:40:31.940 --> 00:40:36.100 +together they they connect with they interact with each other and there's like four back-end services + +00:40:36.720 --> 00:40:40.839 +no three services and a front-end service then there's like a web ui and that's the service but + +00:40:40.860 --> 00:40:44.000 +but then you also have to build in a data storage. + +00:40:44.420 --> 00:40:45.860 +So for when you get to that point, + +00:40:46.020 --> 00:40:48.020 +that's when you either need MySQL, Postgres, or Cassandra. + +00:40:48.320 --> 00:40:49.920 +You could probably get away with SQLite on that one, + +00:40:50.000 --> 00:40:51.320 +but I've never actually tried it. + +00:40:52.420 --> 00:40:55.200 +But we recommend MySQL, Postgres, or Cassandra. + +00:40:55.580 --> 00:40:57.040 +And then you can add in other things + +00:40:57.240 --> 00:40:59.240 +like Elastic for visibility. + +00:40:59.300 --> 00:41:01.660 +You can add Prometheus and Grafana for being able to see. + +00:41:02.020 --> 00:41:03.060 +But this is when you're starting to scale up. + +00:41:03.100 --> 00:41:07.060 +So if you were doing this on small, itty-bitty scale, + +00:41:07.200 --> 00:41:08.480 +you could probably deploy it onto, say, + +00:41:08.520 --> 00:41:09.459 +a DigitalOcean droplet + +00:41:09.780 --> 00:41:11.040 +and be fine with a single binary. + +00:41:11.220 --> 00:41:12.500 +We have tutorials on our learn site + +00:41:12.520 --> 00:41:13.620 +on exactly how to do this. + +00:41:14.960 --> 00:41:15.600 +And then it scales. + +00:41:15.840 --> 00:41:17.500 +So like there's a Docker Compose file. + +00:41:17.520 --> 00:41:18.380 +So like once you want to see + +00:41:18.440 --> 00:41:19.780 +what all these microservices are doing, + +00:41:20.000 --> 00:41:21.440 +like in multiple areas, + +00:41:21.840 --> 00:41:23.620 +like you can see how we've deployed all of them + +00:41:23.680 --> 00:41:24.280 +and you can play it. + +00:41:24.380 --> 00:41:26.840 +There's multiple different Docker Compose files + +00:41:26.960 --> 00:41:28.440 +to be like, oh, I want it with this database + +00:41:28.660 --> 00:41:29.200 +and these options. + +00:41:29.380 --> 00:41:31.200 +And that allows you to kind of like tune it + +00:41:31.200 --> 00:41:32.380 +and tweak it to your liking. + +00:41:32.840 --> 00:41:34.200 +And then once you get to prod scale, + +00:41:34.420 --> 00:41:35.860 +we have Helm charts that we support + +00:41:36.620 --> 00:41:37.940 +and you can deploy it directly into Kubernetes. + +00:41:38.360 --> 00:41:39.880 +Now, if you're self-hosting in production, + +00:41:40.360 --> 00:41:42.540 +Kubernetes tends to be what we see in the wild + +00:41:42.720 --> 00:41:44.820 +as the most popular deployment. + +00:41:45.180 --> 00:41:46.540 +Now, again, these are single binaries. + +00:41:46.880 --> 00:41:48.120 +You can deploy it however you want. + +00:41:48.360 --> 00:41:50.840 +So it has a, from development to prod, + +00:41:50.840 --> 00:41:52.600 +it has a very nice scaling story. + +00:41:53.640 --> 00:41:55.240 +Or the other option is just go to Temporal Cloud. + +00:41:55.960 --> 00:41:59.380 +The thing is Temporal Cloud has a minimum pricing structure + +00:41:59.740 --> 00:42:01.380 +for it, which we're constantly updating. + +00:42:02.840 --> 00:42:06.120 +And that's really useful once you get to actual production + +00:42:06.200 --> 00:42:08.320 +use cases and you have people paying you + +00:42:08.340 --> 00:42:09.740 +use that money to pay for the temporal cloud. + +00:42:10.060 --> 00:42:10.600 +It's not a lot. + +00:42:10.700 --> 00:42:12.100 +The temporal cloud pricing is super, + +00:42:13.740 --> 00:42:16.020 +I think it's really super interestingly well done + +00:42:16.200 --> 00:42:17.660 +because it's based on what we call actions. + +00:42:18.090 --> 00:42:20.360 +So like anything that basically causes a right + +00:42:20.440 --> 00:42:21.220 +to that durable database, + +00:42:21.640 --> 00:42:23.880 +you get billed on like fractions of a penny. + +00:42:24.190 --> 00:42:25.180 +So it's consumption based. + +00:42:26.340 --> 00:42:28.160 +The main thing is making sure you have enough traffic + +00:42:28.360 --> 00:42:30.600 +to cover, basically there's a minimum support cost + +00:42:31.600 --> 00:42:33.720 +that you have to pay for like a minimum bill requirement + +00:42:33.840 --> 00:42:34.780 +and then you get billed on the actions + +00:42:35.520 --> 00:42:36.780 +and making sure you have enough traffic + +00:42:36.800 --> 00:42:38.000 +and workload to handle that. + +00:42:38.780 --> 00:42:40.400 +I know, and then you get a lot of other cool features + +00:42:40.520 --> 00:42:43.860 +in cloud that, I have to make sure I say this carefully, + +00:42:44.240 --> 00:42:47.540 +that you don't get on the open source version, + +00:42:47.640 --> 00:42:49.300 +but those are all deployment options. + +00:42:49.880 --> 00:42:52.200 +So when it comes to feature to feature option, + +00:42:52.400 --> 00:42:54.220 +as right now, it's a one-to-one comparison. + +00:42:54.580 --> 00:42:57.880 +What runs in open source is what is deployed in cloud. + +00:42:58.100 --> 00:43:01.480 +Like, if I built all of my stuff on the open source stuff, + +00:43:02.060 --> 00:43:03.580 +I deployed to cloud on occasion to test it, + +00:43:03.800 --> 00:43:05.980 +but I'm constantly building on the open source version + +00:43:06.000 --> 00:43:07.000 +And all of my stuff runs in cloud. + +00:43:07.380 --> 00:43:10.100 +What you get when you go to cloud is you get those production features. + +00:43:10.480 --> 00:43:11.640 +You get single sign-on. + +00:43:11.900 --> 00:43:13.360 +You get role-based access control. + +00:43:13.860 --> 00:43:17.160 +You get more advanced metrics and things of that nature. + +00:43:17.220 --> 00:43:20.480 +You get all of these things that really matter at a large-scale production, + +00:43:20.760 --> 00:43:22.420 +things that large enterprises care about. + +00:43:23.320 --> 00:43:27.340 +And there are ways you can get those in the open source as well, + +00:43:27.940 --> 00:43:31.600 +but they come as first-class citizens in the cloud product. + +00:43:31.700 --> 00:43:35.359 +Maybe you got to run your own single sign-on identity server + +00:43:35.380 --> 00:43:43.920 +or something like that yeah yeah um so over here if i go to the temporal github repo there's docker + +00:43:44.360 --> 00:43:50.620 +which has got a docker compose think something yeah there's actually a docker repo repo yeah + +00:43:51.100 --> 00:43:54.920 +is there okay so i could even i think it would be like temporal and then doc like in the search i + +00:43:54.920 --> 00:43:59.260 +would just search or in the org i would search for docker compose yeah it might be there somewhere + +00:43:59.820 --> 00:44:04.040 +uh there we go are you right yeah i'll put that in the links as well i'm a huge fan of docker + +00:44:04.060 --> 00:44:08.460 +compose. I think it opens up a bunch of great options. And if you go in here, you can see that + +00:44:08.560 --> 00:44:15.220 +there's like the doctor compose my sequel to Docker compose postgres, and then just a bare bones one, + +00:44:15.540 --> 00:44:18.760 +right? So if you wanted to self host it, is this a pretty good way to go? + +00:44:19.120 --> 00:44:21.560 +Yeah, I think it's a great place to start. I think it always depends on your scale. + +00:44:22.600 --> 00:44:27.840 +Like if you if you are good at Docker compose, and you know, you think like, the thing with Docker + +00:44:28.020 --> 00:44:31.599 +compose is like, and I believe this is all going to deploy it on a single service. So is like our + +00:44:31.580 --> 00:44:36.400 +single server. So like we always talk about like, you know, depends on the level of reliability you + +00:44:36.520 --> 00:44:40.860 +need of the service because if the database of the service goes away, then your reliability goes away. + +00:44:41.180 --> 00:44:45.560 +Like, you know, but that's kind of like the truth here is with that database being the single source + +00:44:45.640 --> 00:44:50.680 +of truth, if that database magically disappears, you don't have it. Now you could, you know, what + +00:44:50.840 --> 00:44:54.840 +you could do like an RDS or like a digital ocean managed database and connect that in this. + +00:44:55.180 --> 00:44:59.120 +Yeah, run just basically set a room, an off server database connection. + +00:44:59.180 --> 00:45:04.640 +string. Yeah, exactly. Right. Of any form that's then, then it's matching the reliability of that + +00:45:04.780 --> 00:45:09.280 +database. Yeah, exactly. Yeah. Yeah. So another thing, whenever I hear durable, and actually I see + +00:45:09.620 --> 00:45:13.780 +a bit about this in the show notes as well, and you say retry, like if something goes wrong, + +00:45:13.860 --> 00:45:17.940 +which we tried and it'll probably resume. Like sometimes it won't. Sometimes there's a poison, + +00:45:18.260 --> 00:45:23.600 +what's called a poison message. Like it's cursed to always fail for whatever reason. Like it's + +00:45:23.720 --> 00:45:28.559 +trying to get to a service. The domain is gone or it's not coming back. Right. Yep. One of those + +00:45:28.580 --> 00:45:32.620 +sorts of things. Three months ago, it was there. Then you said sleep for three months and it woke + +00:45:32.720 --> 00:45:37.480 +up and said, wait, it's like Planet of the Apes. You're like, yeah, what have they done? You wake + +00:45:37.560 --> 00:45:41.380 +up, you're like, the world is not what I thought, right? How do you deal with that kind of stuff? + +00:45:42.320 --> 00:45:49.180 +Yeah. So in the retry policy, so the default retry policy, what is, is it, its default is just retry + +00:45:49.660 --> 00:45:55.160 +on a specific time limit forever until it's either canceled or it succeeds. Now, if you expect that + +00:45:55.180 --> 00:45:59.620 +something could happen like that. Essentially what you would do is there's something what's + +00:45:59.620 --> 00:46:05.500 +known or non-retriable error types. So certain HTTP error codes, you know, might, you might would be + +00:46:05.600 --> 00:46:10.900 +like, Hey, you know, this is just not going to come back. 500 is fine, but maybe 404 is like + +00:46:11.200 --> 00:46:16.940 +404 might never come back, but 500 very likely could or cannot connect. I don't know. Is that + +00:46:17.080 --> 00:46:20.800 +404? I don't think so. I think it does. I don't remember. Yeah. I remember, but there's probably + +00:46:20.820 --> 00:46:26.120 +a code that like i i can't connect to the server versus like i got there and it said not here yeah + +00:46:26.230 --> 00:46:30.260 +so there's non-retriable error codes and i mean like what we do with the core temporal primitives + +00:46:30.400 --> 00:46:34.700 +is more of like um i often tell people it's like yeah like things that can retry let them retry but + +00:46:34.760 --> 00:46:39.700 +like say i do a divide by zero error no amount of retries is going to change the laws of physics + +00:46:40.380 --> 00:46:44.180 +um you know it's like holding your breath until something changes you're just going to pass out + +00:46:44.540 --> 00:46:47.900 +um same thing with these retries so you have to have what are called non-retriable errors and you + +00:46:47.860 --> 00:46:51.400 +essentially say, hey, whenever you experience this error, you should just fail. And basically, + +00:46:51.600 --> 00:46:55.200 +you would bubble it up and then you have to let the next layer of execution handle it. + +00:46:55.440 --> 00:46:59.140 +So yeah, totally a good story for that. Okay. Yeah. But you kind of got to think a little bit + +00:46:59.200 --> 00:47:03.580 +about it, right? Just yes. Yes. You still have to think it doesn't take away all the thinking for + +00:47:03.960 --> 00:47:08.480 +when it comes to like, what potentially could go wrong, but at least like with all of like the weird, + +00:47:08.800 --> 00:47:12.620 +like, you know, okay, this service might go down DNS, someone messed with DNS. It's always DNS. + +00:47:13.040 --> 00:47:17.820 +I had I had an it's always DNS moment the other day. And I'm like, I need a sticker for my laptop + +00:47:17.840 --> 00:47:19.260 +says this because it got me again. + +00:47:19.760 --> 00:47:20.920 +I can't remember what took me out. + +00:47:21.620 --> 00:47:23.480 +I suffered something with that as well. + +00:47:23.540 --> 00:47:24.740 +And it wasn't even directly me, + +00:47:24.820 --> 00:47:26.880 +but it was something that I was subjected to. + +00:47:27.000 --> 00:47:27.080 +Yeah. + +00:47:27.340 --> 00:47:28.040 +It is always DNS. + +00:47:28.600 --> 00:47:29.340 +It's always DNS. + +00:47:29.820 --> 00:47:30.940 +Except for when it's the database. + +00:47:31.440 --> 00:47:31.540 +Yeah. + +00:47:32.040 --> 00:47:34.560 +Or except for when you accidentally deploy + +00:47:34.600 --> 00:47:37.240 +the walrus operator on Python 3.7 on the server. + +00:47:37.420 --> 00:47:38.440 +That also is not good. + +00:47:38.720 --> 00:47:39.260 +And it's not doing, + +00:47:39.380 --> 00:47:42.020 +which I took down Talk Python for 20 minutes. + +00:47:42.080 --> 00:47:42.740 +I'm like, what? + +00:47:43.060 --> 00:47:43.880 +Why won't it run? + +00:47:43.880 --> 00:47:44.620 +It was running perfect. + +00:47:45.180 --> 00:47:46.780 +Oh, this was like, you know, years ago + +00:47:46.800 --> 00:47:48.320 +when those things were like new, right? + +00:47:48.720 --> 00:47:48.880 +Yeah. + +00:47:49.380 --> 00:47:49.940 +Oh my goodness. + +00:47:50.920 --> 00:47:52.660 +So one thing I want to talk about here, + +00:47:52.720 --> 00:47:54.480 +let me get back to, that's the right spot. + +00:47:54.820 --> 00:47:58.200 +Let's talk about the programming execution model. + +00:47:58.800 --> 00:48:00.940 +Let's see, is there a quick start? + +00:48:01.100 --> 00:48:02.820 +Let's maybe, let's talk through the quick start. + +00:48:02.860 --> 00:48:05.580 +And I think there's just some super interesting + +00:48:06.240 --> 00:48:08.840 +modern Python ideas that you got here, right? + +00:48:09.000 --> 00:48:10.620 +So maybe talk us through like, + +00:48:10.920 --> 00:48:12.880 +how do we get started bringing this in? + +00:48:13.200 --> 00:48:14.700 +Does it have to be a from scratch? + +00:48:15.020 --> 00:48:19.680 +Or can I take some part of my application, like a particular API endpoint and go, this + +00:48:19.860 --> 00:48:21.100 +thing needs, needs some help. + +00:48:21.540 --> 00:48:22.460 +And Temporal can help it. + +00:48:22.720 --> 00:48:24.180 +Let's just plug it in on this one bit. + +00:48:24.460 --> 00:48:27.940 +That's actually how we recommend a lot of people get started with Temporal is like, we + +00:48:28.040 --> 00:48:31.280 +don't, we never tell people like do like a whole ground up rewrite of like everything + +00:48:31.320 --> 00:48:31.720 +you have. + +00:48:32.240 --> 00:48:38.700 +We, I've often told people find something that, like find, find a service that annoys + +00:48:38.740 --> 00:48:43.180 +you, that pages you because it's constantly going down because it's unreliable and maybe + +00:48:43.200 --> 00:48:47.160 +do a like it's a small service and do a small rewrite of that in temporal and just see how your + +00:48:47.360 --> 00:48:51.580 +life makes uh makes a difference so the way that you do it with temporal is like you have to use + +00:48:51.680 --> 00:48:55.960 +the temporal sdks so the way that you build temporal applications is you build them using sdks + +00:48:56.230 --> 00:49:01.520 +um typically with other workflow-esque engines or other durable execution engines um some of the + +00:49:01.520 --> 00:49:05.380 +more modern durable execution engines have kind of followed suit with us but some of the earlier ones + +00:49:05.620 --> 00:49:11.440 +didn't we're code-based um we're not dag based we're not yaml based we don't have our own structured + +00:49:11.460 --> 00:49:17.180 +DSL, we are 100% code based. And there's a lot of advantages to that. So not XML based. No, + +00:49:18.820 --> 00:49:26.260 +no XML. So yeah, so we build you build basically what's called a workflow. Workflow is you can kind + +00:49:26.260 --> 00:49:29.800 +of think of a workflow is like your application. It's it's the it's the blueprint of your + +00:49:29.980 --> 00:49:34.020 +entire application. And what we'd say about workflows is that they have to be deterministic, + +00:49:34.260 --> 00:49:38.560 +like the code within a workflow must be deterministic, and you execute it line by line going + +00:49:38.580 --> 00:49:43.100 +down. And then anything within your workflow that potentially could be non-deterministic or + +00:49:43.640 --> 00:49:49.020 +could potentially fail. So calling out to a microservice is both of those. It's non-deterministic + +00:49:49.240 --> 00:49:52.540 +because you don't know what you're going to get back. And it's potentially going to fail because + +00:49:52.550 --> 00:49:56.000 +the service could be down. That goes in what's called an activity. An activity is that thing that + +00:49:56.260 --> 00:50:01.720 +automatically gets the retries. Now you implement activities as functions or as methods in Python. + +00:50:02.160 --> 00:50:08.540 +You can actually do them as both. And as you were saying in the readme, yeah, we use a lot of + +00:50:08.560 --> 00:50:13.160 +Python tooling. So that's actually something that I think that our SDK engineers, we have an entire + +00:50:13.360 --> 00:50:17.920 +team whose entire job is to build and maintain our SDKs. They're super proud of. It's one of the + +00:50:17.920 --> 00:50:23.300 +things I love talking about. Temporal is not built off of what you would say like an open API spec + +00:50:23.580 --> 00:50:28.320 +for a good reason. So it's built basically, I mean, Temporal itself is built in Go. So as you + +00:50:28.320 --> 00:50:33.220 +can imagine, everything's built in protobufs. There is a spec on what we build off of, but open API + +00:50:33.240 --> 00:50:38.500 +specs generate stuck stub client libraries. And like I've worked with a handful of them. They don't, + +00:50:38.600 --> 00:50:42.200 +they're not very idiomatic to the language. Like it kind of looks like someone bolted a C library + +00:50:42.310 --> 00:50:48.940 +on top of Python. Like it works, but it doesn't feel like Python. Our SDK engineers spend months + +00:50:49.360 --> 00:50:53.400 +studying the programming language, learning out what is idiomatic of the language, what actually + +00:50:53.660 --> 00:50:58.260 +makes sense. And then they build into it. So the interesting thing about this is like, you can see + +00:50:58.340 --> 00:51:03.200 +here when we define our decorate, our workflows, we define them by decorating classes. And then we + +00:51:03.220 --> 00:51:07.000 +the entry point by decorating it with an at workflow.run. So that's how we know where the + +00:51:07.090 --> 00:51:13.480 +entry point is. We're using asyncio inside of this to do it. Our activities, when you want to + +00:51:13.710 --> 00:51:18.780 +turn a method into an activity, you are a function in an activity, you just decorate it at activity.defin. + +00:51:19.060 --> 00:51:25.140 +And now you have an activity. We've done a lot of those kinds of things. We're using context + +00:51:25.320 --> 00:51:32.060 +managers very, very a lot or, you know, rigorously within the code base. The really interesting thing + +00:51:32.080 --> 00:51:36.240 +is, and I can only talk a teensy bit about this because it's super complex, but there's a really + +00:51:36.360 --> 00:51:41.700 +great talk track from it at the PyTexas conference from this year. We built a custom async event loop + +00:51:41.920 --> 00:51:46.380 +for this. This runs in a durable asynchronous event loop. And the SDK engineer who built it + +00:51:46.740 --> 00:51:51.460 +gave a talk at PyTexas this year, and it's on the PyTexas 2025 website. And I can provide a link to + +00:51:51.580 --> 00:51:57.500 +that later. And it's really neat because essentially we had to build a custom event loop to handle + +00:51:57.680 --> 00:52:00.280 +all of the way that we expect Temporal to work. + +00:52:01.110 --> 00:52:02.640 +And I don't think that building a custom event loop + +00:52:02.840 --> 00:52:04.360 +is a very common thing for people to do. + +00:52:05.480 --> 00:52:06.740 +So there's a lot of lessons learned there. + +00:52:07.480 --> 00:52:08.640 +I think there should be more of it though. + +00:52:08.650 --> 00:52:10.780 +I think it's a really interesting story. + +00:52:11.740 --> 00:52:13.080 +Yeah, I'll put this in the show notes. + +00:52:13.480 --> 00:52:14.760 +Yeah, that's Chad's talk. + +00:52:14.900 --> 00:52:15.480 +It's amazing. + +00:52:15.550 --> 00:52:17.480 +So like he talks about all the things that he had to do + +00:52:17.620 --> 00:52:20.100 +when he was building out a custom event loop + +00:52:22.239 --> 00:52:23.800 +for Temporal's AsyncIO event loop. + +00:52:24.270 --> 00:52:27.240 +Yeah, so for people for whom this is super new, + +00:52:27.460 --> 00:52:32.820 +this idea. You need places in your code to execute to say, okay, we can stop, maybe save some state. + +00:52:33.120 --> 00:52:38.840 +And then you talked about like replaying behaviors and so on. So you can write code like await sleep + +00:52:39.140 --> 00:52:46.020 +a day, you know, asyncio dot sleep one day. And that goes into this, as you said, durable asyncio + +00:52:46.380 --> 00:52:51.360 +execution. So instead of just saying, well, we're going to let the thread do other stuff. It's like, + +00:52:51.580 --> 00:52:56.200 +no, save it and then resume it. Right. That's pretty wild. It's pretty great actually. Yeah. + +00:52:56.320 --> 00:52:59.760 +And I mean, those, those, and that's one of the great things about the workflows and like + +00:53:00.160 --> 00:53:02.440 +that, those you'd have to definitely write as workflows, but yeah. + +00:53:02.490 --> 00:53:06.760 +And it takes up no, no energy or it doesn't jam up a CPU thread by sleeping. + +00:53:06.940 --> 00:53:09.420 +Like it's usually if you sleep something, that thread is kind of sitting there and it's + +00:53:09.560 --> 00:53:09.820 +stuck. + +00:53:09.960 --> 00:53:13.960 +this a hundred percent because it's all event sourced under the hood, essentially. + +00:53:14.339 --> 00:53:18.920 +the event, basically the timer started event gets recorded into the service and then + +00:53:19.050 --> 00:53:19.900 +it gets descheduled. + +00:53:20.070 --> 00:53:23.100 +And then that worker, that, that executor can continue performing other tasks. + +00:53:23.260 --> 00:53:28.080 +And then once that timer fires, the next task for it to continue execution gets put on the task queue. + +00:53:28.380 --> 00:53:30.980 +The worker consumes it, knows, oh, I need to resume. + +00:53:31.340 --> 00:53:31.820 +And it resumes. + +00:53:31.900 --> 00:53:33.240 +And that can happen whenever. + +00:53:33.680 --> 00:53:36.280 +That can happen, you know, a day from now, three days from now, three months from now. + +00:53:36.280 --> 00:53:36.760 +It doesn't matter. + +00:53:36.940 --> 00:53:39.140 +Eventually it gets put on the queue and it gets executed as if nothing happened. + +00:53:39.360 --> 00:53:39.980 +Yeah, that's wild. + +00:53:40.400 --> 00:53:41.860 +So interesting question out of the audience. + +00:53:42.519 --> 00:53:45.660 +If people went with your cloud option, where does that run? + +00:53:45.720 --> 00:53:46.440 +Is that AWS? + +00:53:46.940 --> 00:53:47.760 +Is that DigitalOcean? + +00:53:48.499 --> 00:53:50.980 +Currently it's in AWS and GCP. + +00:53:51.260 --> 00:53:51.420 +Okay. + +00:53:51.780 --> 00:53:52.720 +So those are the two clouds. + +00:53:52.740 --> 00:53:54.120 +Pick different availability zones. + +00:53:54.310 --> 00:53:55.640 +Like if I were in Virginia, + +00:53:56.260 --> 00:53:57.300 +US East one or whatever, + +00:53:57.370 --> 00:53:58.860 +I could say I want to use that one. + +00:53:59.340 --> 00:53:59.500 +Exactly. + +00:53:59.780 --> 00:54:00.800 +There's different availability zones + +00:54:00.830 --> 00:54:02.360 +and we even have multi-region failover. + +00:54:03.060 --> 00:54:04.660 +So if you needed multi-region availability + +00:54:04.920 --> 00:54:06.160 +for super, super high availability, + +00:54:07.220 --> 00:54:09.140 +which we do have definitely have large customers + +00:54:09.230 --> 00:54:10.480 +who need that level of availability. + +00:54:11.359 --> 00:54:12.740 +And you can totally do that. + +00:54:13.020 --> 00:54:13.120 +Cool. + +00:54:13.700 --> 00:54:15.240 +And that doesn't mean you got to be in AWS. + +00:54:15.780 --> 00:54:16.260 +No, no. + +00:54:16.270 --> 00:54:18.620 +We have people who are running on their own private, + +00:54:18.760 --> 00:54:20.060 +they were running on their own infrastructure + +00:54:20.270 --> 00:54:22.460 +on their on-prem and they're calling into it. + +00:54:22.540 --> 00:54:30.280 +So that's the niftiest thing about temporal cloud is like the security model is super simple because temporal cloud or the temporal service. + +00:54:30.320 --> 00:54:31.200 +It doesn't have to be temporal cloud. + +00:54:31.260 --> 00:54:33.900 +It's the service, whether you self-host it or whether it's on the cloud. + +00:54:34.360 --> 00:54:35.780 +It never connects to you. + +00:54:36.100 --> 00:54:39.300 +The workers always do an outbound connection into the temporal service. + +00:54:39.780 --> 00:54:45.600 +So if you're using temporal cloud, the only firewall rule you have to allow is a single outbound connection off of a single port. + +00:54:45.840 --> 00:54:46.040 +That's it. + +00:54:46.340 --> 00:54:46.980 +So it's really awesome. + +00:54:47.600 --> 00:54:49.200 +Yeah, I'm impressed with a lot of this. + +00:54:49.380 --> 00:54:51.400 +So let's sort of kind of get into that. + +00:54:51.580 --> 00:54:53.000 +And I want to maybe close out with two topics, + +00:54:53.720 --> 00:54:55.260 +testing and what's next, + +00:54:55.580 --> 00:54:58.960 +or like sort of also maybe multi-language stuff + +00:54:59.100 --> 00:55:00.080 +we should touch on as well. + +00:55:00.180 --> 00:55:02.060 +But let's talk testing first. + +00:55:02.480 --> 00:55:05.060 +So there's a couple of interesting challenges here. + +00:55:05.300 --> 00:55:07.320 +Like first, if I write code like this + +00:55:07.320 --> 00:55:08.460 +and I want to have a unit test, + +00:55:08.840 --> 00:55:11.020 +no longer am I like, well, this is complicated. + +00:55:11.060 --> 00:55:12.700 +I just have to use pytestAsyncIO. + +00:55:12.920 --> 00:55:14.220 +It's like more than that, right? + +00:55:14.420 --> 00:55:15.780 +So what's the testing story? + +00:55:16.060 --> 00:55:18.120 +Yeah, so the cool thing about the testing story with this + +00:55:18.240 --> 00:55:21.100 +is it is technically still pytestAsyncIO. + +00:55:21.520 --> 00:55:28.700 +because we're code-based, this is one of the things that I always have to harp, to, to remind people on my courses and everyone's always like, oh, that's so cool. + +00:55:29.160 --> 00:55:32.940 +you, because we're code, you have to give up none of your tooling. + +00:55:33.280 --> 00:55:35.100 +You don't have to get like, how do you like to package it? + +00:55:35.120 --> 00:55:36.000 +Are you a poetry person? + +00:55:36.100 --> 00:55:36.820 +Are you a B person? + +00:55:36.980 --> 00:55:38.340 +Are you a Pippin person? + +00:55:38.720 --> 00:55:40.480 +Whatever you want to use, use it for your testing. + +00:55:40.680 --> 00:55:42.400 +Do you want to use a Po the poet? + +00:55:42.600 --> 00:55:44.080 +Are you a pie test person? + +00:55:44.260 --> 00:55:44.840 +Like, what are you using? + +00:55:45.160 --> 00:55:45.420 +Use it. + +00:55:45.420 --> 00:55:45.920 +It doesn't matter. + +00:55:46.440 --> 00:55:50.000 +Temporal does provide, obviously because these are more complex workflows and stuff. + +00:55:50.040 --> 00:55:56.100 +they require a little bit. Every single temporal SDK does provide a testing framework that is built + +00:55:56.260 --> 00:56:02.180 +into it. And these integrate natively with all of your testing frameworks. So you can use, you use + +00:56:02.280 --> 00:56:07.060 +pytest. I use pytest. Now you could use something else, but all of our courses, I think the temporal + +00:56:07.360 --> 00:56:12.920 +102 course, yeah, I wrote that one, has a whole chapter on testing temporal workflows and activities + +00:56:13.340 --> 00:56:16.760 +and mocking activities because you have to be able to call them independently because you, + +00:56:16.840 --> 00:56:17.920 +Otherwise that's an integration test. + +00:56:17.940 --> 00:56:18.620 +It's not a unit test. + +00:56:18.900 --> 00:56:19.420 +It is important. + +00:56:19.460 --> 00:56:20.900 +I always feel bad about mocking the code. + +00:56:20.900 --> 00:56:22.140 +It doesn't seem nice, but you got to. + +00:56:22.240 --> 00:56:23.420 +Yeah, it's super easy. + +00:56:23.620 --> 00:56:24.400 +It's so nice. + +00:56:25.420 --> 00:56:28.140 +The you you basically just read you basically redefine the activity + +00:56:28.280 --> 00:56:31.140 +and I just put it in the mocking story in Python is so nice. + +00:56:32.400 --> 00:56:34.100 +But you just use our testing framework. + +00:56:34.200 --> 00:56:36.640 +And what that does is it basically the testing framework will automatically + +00:56:36.920 --> 00:56:39.120 +spin up a temporal service and a worker for you. + +00:56:39.260 --> 00:56:42.280 +So because as you can imagine, like the execution part of this is like, + +00:56:42.320 --> 00:56:43.720 +you have to have a service running, you have to have a worker + +00:56:44.160 --> 00:56:46.180 +and then you have to send like basically use a client + +00:56:46.200 --> 00:56:51.060 +to send a request to execute the workflow and it will be executed. The temporal testing service + +00:56:51.130 --> 00:56:56.220 +does all of that for you. And it also has basically time skipping. So what you can do is, you know, + +00:56:56.440 --> 00:56:59.640 +if I have to test, if I have to test something that sleeps for 30 days, I would actually prefer + +00:56:59.800 --> 00:57:05.080 +it to not actually sleep for 30 days. So I can test it. Well, I see I take so long. Yeah. So + +00:57:05.280 --> 00:57:08.420 +time skipping is in there and there's a lot of other really neat features in the testing. So + +00:57:08.670 --> 00:57:14.280 +every single temporal Python or sorry, sorry, every single temporal SDK has a testing framework + +00:57:14.300 --> 00:57:17.740 +built into it that really well enables testing. + +00:57:17.980 --> 00:57:21.900 +And it all works natively with the tools that you are already used to using as a developer + +00:57:22.060 --> 00:57:23.220 +of whatever language you're already using. + +00:57:23.380 --> 00:57:24.360 +Yeah, that sounds great. + +00:57:24.430 --> 00:57:28.760 +And you can say things like run this and wait for a week and then resume. + +00:57:28.810 --> 00:57:30.640 +And you can just say, and now a week has resumed. + +00:57:30.770 --> 00:57:31.660 +You know, a week has passed. + +00:57:31.930 --> 00:57:33.240 +Now I see what's happened, right? + +00:57:33.400 --> 00:57:34.720 +Like it'll just zip right ahead. + +00:57:35.100 --> 00:57:35.240 +Yep. + +00:57:35.640 --> 00:57:36.000 +Skips right ahead. + +00:57:36.140 --> 00:57:37.100 +Doesn't even, doesn't even worry about it. + +00:57:37.320 --> 00:57:38.960 +That's always a fun testing trick. + +00:57:39.580 --> 00:57:42.740 +So the other one I want to talk about is when I'm over here, + +00:57:43.380 --> 00:57:45.480 +into the GitHub repo, I can scroll down. + +00:57:45.550 --> 00:57:49.360 +If I look at the repositories, it's Java SDK, Python SDK, + +00:57:49.570 --> 00:57:52.540 +Go SDK,.NET SDK, Ruby SDK, et cetera, et cetera. + +00:57:52.900 --> 00:57:55.180 +There's a bunch of interesting things here. + +00:57:55.320 --> 00:57:57.660 +Like I can write part of my app in Java, + +00:57:58.160 --> 00:58:01.180 +part of the workflow in Java or the workflow items in Java, + +00:58:01.540 --> 00:58:04.720 +the queues, and I can do part of it in Python or.NET. + +00:58:04.790 --> 00:58:05.940 +And the other part I think is interesting + +00:58:06.140 --> 00:58:08.280 +is Python,.NET, Ruby, et cetera, + +00:58:08.720 --> 00:58:10.860 +all share a common Rust base. + +00:58:10.930 --> 00:58:13.060 +And so it's kind of like they all go in lockstep. + +00:58:13.340 --> 00:58:19.600 +Yeah. Yeah. Yeah. So yeah, two great topics there. So the first one I'll start off with is the is the + +00:58:19.680 --> 00:58:23.040 +polyglot stuff, because I think it's one of my favorite things. And we don't I don't I don't ever + +00:58:23.050 --> 00:58:27.420 +get to talk about it enough. So I'm glad you asked. Underneath the hood, the way that all of this + +00:58:27.640 --> 00:58:32.880 +communication happens is it's happening via essentially protobuffs across task cubes to the + +00:58:32.940 --> 00:58:36.859 +temporal service back and forth. One of the things that you'll find if you dig into temporal is that + +00:58:36.820 --> 00:58:43.060 +we require your input, the inputs and outputs of your functions to be serializable to basically + +00:58:43.360 --> 00:58:47.620 +protobufs. Now, if you have something that's not serializable, we obviously, because it's code, + +00:58:47.620 --> 00:58:53.880 +we provide you the way to extend the serializer. So as long as you can serialize it, again, + +00:58:54.320 --> 00:58:58.540 +it's code, you can do whatever you want with it. But because of that, because everything speaks + +00:58:58.920 --> 00:59:03.120 +protobuf, all of these languages can natively speak to each other. So you're right, I can write + +00:59:03.140 --> 00:59:08.540 +workflows written in Python and call and have three different activities in that workflow, + +00:59:08.740 --> 00:59:12.480 +one written in TypeScript, one written in Java and one written in.NET. And I can call them + +00:59:12.720 --> 00:59:16.760 +seamlessly, like basically just by calling execute activity, giving it the name of the function. + +00:59:17.120 --> 00:59:21.540 +And then I could actually still then pass in a data class of the data that I have as that + +00:59:21.720 --> 00:59:25.660 +parameter, because it's still getting serialized down into a protobuf. It will get deserialized + +00:59:25.680 --> 00:59:30.780 +into the other language, execute it and pass back data that I can resume. And then I could call it + +00:59:30.800 --> 00:59:37.560 +technically from a client that's written in Go. So this enables Polyglot across all of these + +00:59:37.660 --> 00:59:41.280 +languages. And it's amazing. So if you have legacy systems or you have stuff where like you really + +00:59:41.320 --> 00:59:46.160 +need something to be written in a whole bunch of different languages, it just gives you this out + +00:59:46.200 --> 00:59:50.500 +of the box for free. And I think it's one of the neatest features. And one of the other part that + +00:59:50.500 --> 00:59:55.740 +it really does need about this is it's not just like the fact that it can call it, but it also + +00:59:56.000 --> 01:00:00.760 +preserves the stack traces as well. So one of the other courses that I developed are Crafting the + +01:00:00.780 --> 01:00:06.740 +handling strategy course. There's a demo in there where I am showing like basically that exact + +01:00:07.020 --> 01:00:11.740 +workflow, a Go client that's calling a Java workflow that's calling a Python activity. + +01:00:12.080 --> 01:00:15.600 +So three different languages. And then I intentionally throw an error in the Python + +01:00:15.900 --> 01:00:21.080 +activity and I tell it, do not handle it, do not retry it, let it bubble up. And when I get back to + +01:00:21.080 --> 01:00:25.400 +the Go client, I can see the different stack traces in the different languages all the way through. + +01:00:25.720 --> 01:00:31.920 +So I get a go basically panic that contains a Java stack trace that contains a, that contains a Python stack trace. + +01:00:32.220 --> 01:00:34.780 +And I can, I can, I can see all of this across the line. + +01:00:34.900 --> 01:00:41.740 +So not, and, and also the thing, and just to do it for fun, because I like showing off, I have all of these workers running on different machines. + +01:00:42.320 --> 01:00:45.760 +So I am, I am running on different, I am crossing process boundaries. + +01:00:46.060 --> 01:00:52.740 +I'm crossing literally across the network IP boundaries, and then I'm crossing language boundaries and it happens seamlessly and you'd never know that it happened. + +01:00:53.080 --> 01:00:56.640 +So the orchestration layer is the wildest thing ever. + +01:00:57.400 --> 01:00:58.940 +And then you asked about the Rust SDK. + +01:00:59.540 --> 01:01:00.200 +That's a fun one. + +01:01:00.500 --> 01:01:03.700 +So that kind of goes back into the history a little bit of how Temporal was built. + +01:01:04.160 --> 01:01:08.260 +And for a crash course in this within two minutes or less, + +01:01:09.100 --> 01:01:13.680 +essentially our founders started at AWS together and built out what would become, + +01:01:15.340 --> 01:01:19.000 +they built the foundations for SQS and what would become simple workflow service. + +01:01:19.440 --> 01:01:24.120 +Then one of the founders left, went to Azure and helped build the Azure Durable Task Framework at Microsoft. + +01:01:24.630 --> 01:01:27.520 +They met up back together at Uber and built Uber's Cadence. + +01:01:28.340 --> 01:01:34.540 +Cadence was then basically like battle tested for four years, open sourced, and then they got permission to fork it and build Temporal. + +01:01:34.570 --> 01:01:38.740 +So Temporal, the company is six years old, but it was a four year old open source project prior. + +01:01:39.050 --> 01:01:40.940 +So it's a 10 year old open source project, essentially. + +01:01:41.980 --> 01:01:46.400 +But because of that, what happened at Cadence was I think they wrote the Go and the Java SDKs there. + +01:01:46.480 --> 01:01:49.120 +So those are very uniquely themselves because they were written independently. + +01:01:49.920 --> 01:01:52.080 +And then the PHP SDK is its own story. + +01:01:52.900 --> 01:01:55.000 +Someone wrote that in the community because they really wanted it. + +01:01:55.200 --> 01:01:56.760 +And it kind of follows its own rules. + +01:01:57.020 --> 01:02:03.100 +But when they started building the temporal SDKs, TypeScript was the first one and then Python, if I remember correctly. + +01:02:03.680 --> 01:02:06.260 +They wanted a common core, like basically re-implementing this. + +01:02:06.400 --> 01:02:12.660 +Because in these SDKs, there are very complex state machines that maintain all of this state of what's going on. + +01:02:13.620 --> 01:02:15.940 +And they did not want to keep re-implementing this every single time. + +01:02:16.020 --> 01:02:21.680 +So they built a Rust core SDK, or it's not even an SDK. It's not an SDK. It's just the core. And + +01:02:22.000 --> 01:02:27.700 +all of the, so the TypeScript, the.NET, the Python, and the Ruby SDKs all have a upper level SDK + +01:02:28.260 --> 01:02:33.800 +that wraps this core Rust SDK and call into it. So they share a common theme. So there definitely + +01:02:33.940 --> 01:02:37.680 +will sometimes be features or like things that happen in the Go or the Java SDK that you're like, + +01:02:37.720 --> 01:02:42.820 +that's a little different because those are not based on Rust core. But yeah, that's how they all + +01:02:42.820 --> 01:02:47.280 +call in. So like their PIO3 is basically being used here. We're calling into pot with PIO3 into + +01:02:47.280 --> 01:02:53.640 +a rust binding. and that's like a Pydantic and others. Yeah, exactly. Yeah. So, and it's, + +01:02:53.700 --> 01:02:57.700 +it's really cool. And that, that makes, adding new SDKs a lot easier because really, + +01:02:57.780 --> 01:03:01.360 +and truly the hardest part of building the SDKs is, was like those state machines used to take + +01:03:01.360 --> 01:03:05.260 +a long time. And once they got it figured out on the rust core side, it made adding new languages + +01:03:05.500 --> 01:03:10.920 +easier. the Ruby SDK is in public preview and will be going generally available here soon. Um, + +01:03:11.120 --> 01:03:14.480 +And there may be one or two more SDKs coming out within the future. + +01:03:14.940 --> 01:03:17.080 +If you guessed really hard, you could figure out what it is. + +01:03:17.260 --> 01:03:17.720 +There's an SDK. + +01:03:17.900 --> 01:03:19.360 +There's a core that doesn't have an SDK. + +01:03:22.240 --> 01:03:22.920 +There's no secret. + +01:03:23.120 --> 01:03:24.460 +There's no secret about that. + +01:03:25.140 --> 01:03:25.560 +Yeah, of course. + +01:03:25.980 --> 01:03:28.100 +People have been begging for that for years and it's obvious. + +01:03:28.350 --> 01:03:29.860 +So yeah, may involve crates. + +01:03:30.020 --> 01:03:30.080 +Okay. + +01:03:30.320 --> 01:03:33.100 +So what's, what is the, what's the future? + +01:03:33.280 --> 01:03:38.560 +Like anything that's worth giving a shout out that's coming or that kind of stuff? + +01:03:38.980 --> 01:03:39.140 +Yeah. + +01:03:39.260 --> 01:03:44.020 +I mean, I think that like a lot of times people often ask me like, what is Temporal used for? + +01:03:44.130 --> 01:03:48.120 +And I would say Temporal is used for anything that like you don't want your code to fail. + +01:03:48.720 --> 01:04:02.340 +It like it's it's really interesting to help educate people and work on a product that really does affect nearly every single part of the application of the software development lifecycle and every single part of it, of, of, of the industry. + +01:04:03.300 --> 01:04:08.080 +You know, I was used to working on other products that like, yeah, like I worked at a synthetic data company for a little bit. + +01:04:08.200 --> 01:04:09.560 +And that had a very niche area. + +01:04:09.650 --> 01:04:10.620 +And then I worked at DigitalOcean, + +01:04:10.730 --> 01:04:11.620 +which was cloud products, + +01:04:12.030 --> 01:04:12.640 +which is still awesome, + +01:04:12.790 --> 01:04:14.140 +but like doesn't affect everything. + +01:04:14.600 --> 01:04:15.800 +Temporal really does like, + +01:04:16.340 --> 01:04:17.140 +are you doing finance? + +01:04:17.760 --> 01:04:19.960 +Temporal is great for long running transactions. + +01:04:20.140 --> 01:04:21.460 +Are you doing food delivery? + +01:04:21.840 --> 01:04:23.200 +Like, are you a fast food industry? + +01:04:23.380 --> 01:04:24.180 +Are you doing groceries? + +01:04:24.410 --> 01:04:25.900 +Are you doing, what are you doing? + +01:04:26.299 --> 01:04:27.580 +Temporal can benefit from it. + +01:04:27.740 --> 01:04:29.260 +So there's a lot of really cool things. + +01:04:29.820 --> 01:04:30.480 +You can use it for anything. + +01:04:30.610 --> 01:04:31.720 +And what we're seeing right now, + +01:04:32.440 --> 01:04:32.720 +specifically, + +01:04:33.020 --> 01:04:33.800 +and this kind of alludes back + +01:04:33.850 --> 01:04:35.360 +to your open AI thing earlier, + +01:04:35.840 --> 01:04:38.160 +is that we're seeing a lot of AI companies + +01:04:38.180 --> 01:04:44.220 +of value out of this because like when it becomes time to take your agents to production, there's + +01:04:44.600 --> 01:04:49.540 +a handful of decent production stories out there, but it turns out these are AI agents in production. + +01:04:50.140 --> 01:04:54.220 +This is microservices in production with a fancy label on top of it. These are just distributed + +01:04:54.540 --> 01:04:59.560 +systems. Not only are they microservices, they're very slow, long running microservices, + +01:05:00.060 --> 01:05:04.980 +which make it harder. Yeah. Yeah. That's exactly what Temporal's model is. Do you have a slow + +01:05:05.000 --> 01:05:08.920 +running microservice that can sometimes fail. Great. We have a product that makes all those + +01:05:09.120 --> 01:05:13.720 +problems go away. So, you know, like I'll, I'm working on a lot of content right now around + +01:05:14.060 --> 01:05:19.440 +showing the benefit of temporal in AI. and we have, we have a handful of customers, who I + +01:05:19.660 --> 01:05:25.480 +can't talk about, that are, using us for lots of different AI related things. but there, + +01:05:25.580 --> 01:05:28.500 +I mean, you can look on our blog or anything. You can see tons of people that are using it. + +01:05:28.800 --> 01:05:31.760 +and it's a really cool thing. So I would definitely say like, if you're, if you're trying + +01:05:31.780 --> 01:05:36.680 +to take AI to production, you should be looking into temporal. it's not an AI tool, you know, + +01:05:36.760 --> 01:05:39.540 +like we're not, we're not going to like, we're not going to do the thing that every company did is + +01:05:39.540 --> 01:05:45.180 +we're not going to suddenly pivot and become an AI tool. because we're not, we just, yeah, + +01:05:45.810 --> 01:05:50.360 +we solve everything. And AI is one of the great things we solve. So that's awesome. Yeah. You're + +01:05:50.380 --> 01:05:54.900 +not going to vibe code with temporal. Maybe you vibe code, temporal code, but not with temporal. + +01:05:55.260 --> 01:05:59.740 +No, I've, I've, I've actually vibe coded a handful of temporal things. And it's interesting because + +01:05:59.760 --> 01:06:04.760 +like i'm super picky uh about what what people's temporal code looks like as because i've been + +01:06:04.880 --> 01:06:08.980 +teaching people best practices for three years almost three years now and i i'm like no vibe + +01:06:09.080 --> 01:06:14.220 +coding that's no claude that's wrong like no no no cursor that's wrong like you can't do that so + +01:06:14.430 --> 01:06:18.020 +and the interesting thing about that is the way that i'm looking at it's like oh i need to make + +01:06:18.100 --> 01:06:23.040 +more content about this because yes because the llms are not like it's actually funny every now + +01:06:23.160 --> 01:06:27.420 +and then the llms spit out some of my content um and i can tell when it's my content because i know + +01:06:27.360 --> 01:06:29.360 +So I write comments in a very particular way. + +01:06:30.520 --> 01:06:32.000 +And I'm like, oh, okay. + +01:06:32.140 --> 01:06:33.880 +So what that ends up telling me is, + +01:06:34.480 --> 01:06:36.480 +oh, I need to make more content around this + +01:06:36.660 --> 01:06:39.080 +because we're still not vibe coding at 100% capacity. + +01:06:39.660 --> 01:06:39.800 +Yeah. + +01:06:40.100 --> 01:06:40.280 +Yeah. + +01:06:40.700 --> 01:06:40.820 +Yeah. + +01:06:41.460 --> 01:06:43.680 +That's a whole discussion we could go down. + +01:06:44.020 --> 01:06:46.520 +You know, I was totally wrong when we started. + +01:06:46.600 --> 01:06:47.940 +I said we could talk for two hours. + +01:06:47.960 --> 01:06:48.920 +I think it's three to four. + +01:06:49.320 --> 01:06:49.640 +Yeah. + +01:06:49.880 --> 01:06:51.160 +We got so much more we could talk about, + +01:06:51.400 --> 01:06:55.500 +but there's only so much time we can dedicate to each episode. + +01:06:55.840 --> 01:06:57.400 +So let's go ahead and call it. + +01:06:57.400 --> 01:06:59.420 +I say, you know, thanks for being here. + +01:06:59.700 --> 01:07:00.740 +The more I looked into this, + +01:07:00.760 --> 01:07:02.100 +this is a super interesting product. + +01:07:02.380 --> 01:07:04.720 +And there's a lot of neat Python integrations + +01:07:04.880 --> 01:07:08.460 +like you program it with async and await + +01:07:08.600 --> 01:07:11.760 +rather than some funky SDK bolt-on thing. + +01:07:12.040 --> 01:07:13.300 +So people should definitely check it out. + +01:07:13.980 --> 01:07:14.560 +Final call to action. + +01:07:14.720 --> 01:07:15.080 +They're interested. + +01:07:15.700 --> 01:07:16.140 +What do you tell them? + +01:07:16.820 --> 01:07:17.960 +Check out, just check out the website. + +01:07:18.160 --> 01:07:19.620 +Check out temporal.io + +01:07:19.760 --> 01:07:21.640 +or the learn site, learn.temporal.io. + +01:07:21.820 --> 01:07:22.880 +It's a great place to get started. + +01:07:23.840 --> 01:07:24.820 +You can install Temporal + +01:07:24.840 --> 01:07:26.500 +by just running brew install temporal on your Mac, + +01:07:26.760 --> 01:07:29.580 +or there's commands for Windows and Linux as well. + +01:07:29.860 --> 01:07:30.700 +Curl commands for that. + +01:07:31.140 --> 01:07:32.060 +Or Docker compose up. + +01:07:32.320 --> 01:07:33.060 +Or Docker compose up. + +01:07:33.110 --> 01:07:35.160 +If you want to do that, totally can do that. + +01:07:35.760 --> 01:07:37.020 +Just try it out, build a workflow. + +01:07:37.380 --> 01:07:39.220 +And then what I tell people is try to break it. + +01:07:39.520 --> 01:07:41.520 +Like start a workflow, kill the worker, + +01:07:41.880 --> 01:07:42.420 +bring it back online. + +01:07:42.640 --> 01:07:45.020 +Like I think it's really magical + +01:07:45.210 --> 01:07:47.120 +when you first actually try to like + +01:07:47.530 --> 01:07:49.280 +actually break the software and stuff. + +01:07:49.820 --> 01:07:52.080 +We've had multiple people that have taken jobs here + +01:07:52.090 --> 01:07:53.820 +who have said, I started playing with it. + +01:07:53.820 --> 01:07:54.620 +I tried to break it. + +01:07:54.720 --> 01:07:56.620 +And when I couldn't, I decided to apply for a job here. + +01:07:57.920 --> 01:07:59.040 +So try to break it. + +01:07:59.220 --> 01:08:00.240 +See what you can do. + +01:08:00.580 --> 01:08:02.460 +And you'll be amazed by it. + +01:08:02.940 --> 01:08:03.900 +Just a personal anecdote. + +01:08:04.000 --> 01:08:06.320 +I remember when I applied here, I was reading through their docs. + +01:08:06.580 --> 01:08:12.300 +And I told myself, I was like, if they can do half of the things they claim they can do in the docs, this is revolutionary. + +01:08:12.440 --> 01:08:13.600 +I've never seen anything like this. + +01:08:13.600 --> 01:08:15.380 +And it turns out we do all the things we say in our docs. + +01:08:16.640 --> 01:08:19.859 +It's probably the most interesting tech product I've ever worked with in my career. + +01:08:20.680 --> 01:08:23.440 +And I know that I will be working with it probably for the rest of my career. + +01:08:23.540 --> 01:08:26.660 +It fascinates me and I love playing with it. + +01:08:26.740 --> 01:08:30.440 +Like I build temporal applications at home for fun just because it's like, oh, look, + +01:08:30.440 --> 01:08:32.740 +I don't have to worry about someone's API going down anymore. + +01:08:33.100 --> 01:08:33.240 +Yay. + +01:08:33.759 --> 01:08:33.880 +Yeah. + +01:08:34.680 --> 01:08:35.040 +It's awesome. + +01:08:35.319 --> 01:08:36.900 +So I hope you enjoy it as much as I do. + +01:08:37.800 --> 01:08:38.500 +I'm pretty impressed. + +01:08:39.220 --> 01:08:39.339 +All right. + +01:08:39.390 --> 01:08:40.700 +Well, thanks for being on the show. + +01:08:41.120 --> 01:08:42.020 +Thanks for coming on and sharing everything. + +01:08:42.500 --> 01:08:42.960 +Yeah, it was great. + +01:08:43.279 --> 01:08:44.120 +Great to talk with you. + +01:08:44.380 --> 01:08:44.839 +Yeah, you as well. + +01:08:45.140 --> 01:08:45.240 +Bye bye. + +01:08:46.960 --> 01:08:49.339 +This has been another episode of Talk Python To Me. + +01:08:50.160 --> 01:08:51.080 +Thank you to our sponsors. + +01:08:51.560 --> 01:08:52.759 +Be sure to check out what they're offering. + +01:08:52.839 --> 01:08:54.200 +It really helps support the show. + +01:08:54.839 --> 01:08:58.500 +This episode is sponsored by Posit Connect from the makers of Shiny. + +01:08:58.980 --> 01:09:02.859 +Publish, share, and deploy all of your data projects that you're creating using Python. + +01:09:03.480 --> 01:09:09.580 +Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards, and APIs. + +01:09:10.420 --> 01:09:12.060 +Posit Connect supports all of them. + +01:09:12.380 --> 01:09:17.700 +Try Posit Connect for free by going to talkpython.fm/posit, P-O-S-I-T. + +01:09:18.560 --> 01:09:25.779 +The PyBay Conference is returning to the UCSF Mission Bay Conference Center in San Francisco, California on October 18th, 2025. + +01:09:26.779 --> 01:09:30.540 +Get your ticket and pick up a free conference course bundle from Talk Python. + +01:09:31.200 --> 01:09:33.740 +Get started at talkpython.fm/pybay. + +01:09:34.400 --> 01:09:35.279 +Want to level up your Python? + +01:09:35.730 --> 01:09:39.380 +We have one of the largest catalogs of Python video courses over at Talk Python. + +01:09:39.859 --> 01:09:44.540 +Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:09:44.900 --> 01:09:47.160 +And best of all, there's not a subscription in sight. + +01:09:47.240 --> 01:09:50.080 +Check it out for yourself at training.talkpython.fm. + +01:09:50.779 --> 01:09:54.960 +Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:09:55.380 --> 01:09:56.280 +We should be right at the top. + +01:09:56.780 --> 01:10:01.480 +You can also find the iTunes feed at /itunes, the Google Play feed at /play, + +01:10:01.880 --> 01:10:05.660 +and the direct RSS feed at /rss on talkpython.fm. + +01:10:06.320 --> 01:10:08.540 +We're live streaming most of our recordings these days. + +01:10:08.980 --> 01:10:12.020 +If you want to be part of the show and have your comments featured on the air, + +01:10:12.300 --> 01:10:16.400 +be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:10:17.440 --> 01:10:20.940 +This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. + +01:10:21.300 --> 01:10:22.880 +Now get out there and write some Python code. + From 7a6d5908970dc168c77f8e9d5bdeadcb2d6da2bf Mon Sep 17 00:00:00 2001 From: Michael Kennedy Date: Tue, 19 Aug 2025 07:29:30 -0700 Subject: [PATCH 2/5] transcripts --- .../476-unified-python-packaging-with-uv.vtt | 4 +- .../482-pre-commit-hooks-for-python-devs.vtt | 2 +- .../517-agentic-ai-youtube.vtt | 2 +- ...ango-s-20th-birthday-with-its-creators.vtt | 2075 +++++++++++++++++ 4 files changed, 2079 insertions(+), 4 deletions(-) create mode 100644 youtube_transcripts/518-celebrating-django-s-20th-birthday-with-its-creators.vtt diff --git a/youtube_transcripts/476-unified-python-packaging-with-uv.vtt b/youtube_transcripts/476-unified-python-packaging-with-uv.vtt index 84f77b1..2dc5d61 100644 --- a/youtube_transcripts/476-unified-python-packaging-with-uv.vtt +++ b/youtube_transcripts/476-unified-python-packaging-with-uv.vtt @@ -3418,7 +3418,7 @@ Like I now use UVX all the time when I want to run things. So I'll do like UVX rough check and that translates 01:03:20.000 --> 01:03:24.000 -to uv tool run rough, you know, run the check command. +to uv tool run ruff, you know, run the check command. 01:03:24.000 --> 01:03:26.000 Behind the scenes, what it does is it finds latest version @@ -3484,7 +3484,7 @@ or I could uv tool install. it makes a lot of sense to install them. 01:04:28.000 --> 01:04:34.000 -Also, if you have, like if you ran uv tool install rough, +Also, if you have, like if you ran uv toolinstall ruff, 01:04:34.000 --> 01:04:37.000 and then you ran like UVX rough, we would use diff --git a/youtube_transcripts/482-pre-commit-hooks-for-python-devs.vtt b/youtube_transcripts/482-pre-commit-hooks-for-python-devs.vtt index 0bb2ff6..b38be4a 100644 --- a/youtube_transcripts/482-pre-commit-hooks-for-python-devs.vtt +++ b/youtube_transcripts/482-pre-commit-hooks-for-python-devs.vtt @@ -445,7 +445,7 @@ Right. Right. Right. So if you want to run more, you basically have to, potentially, write a program, which then itself figures out all the things to do. 00:13:19.200 --> 00:13:24.800 -And then delegates to running them. Like if you want to run rough with a fixed formatting issues, +And then delegates to running them. Like if you want to run ruff with a fixed formatting issues, 00:13:24.800 --> 00:13:31.760 and you want to run the checker fixer for NumPy doc strings and all those things, diff --git a/youtube_transcripts/517-agentic-ai-youtube.vtt b/youtube_transcripts/517-agentic-ai-youtube.vtt index cbc58d8..d713ebd 100644 --- a/youtube_transcripts/517-agentic-ai-youtube.vtt +++ b/youtube_transcripts/517-agentic-ai-youtube.vtt @@ -2209,7 +2209,7 @@ That is pre, or it's not been affected by like nuclear, uh the nuclear bombs tha the mothballs out that's right yeah and we're going back into into service it's boy it's an interesting time. And there's a lot of opportunity as, as you pointed out. So I want to leave people with message of don't be scared, embrace this, see what it can do for you. There's a lot 01:15:19.740 --> 01:15:36.460 -of opportunity here, but Matt, you get the final word. Yeah. I mean, I agree. I think it's, again, it's like the, the same philosophy we started with, which is like, I don't see this as, as that big of a shift compared to like a lot of the other stuff that's happened in industry. like just, I would just learn these things. +of opportunity here, but Matt, you get the final word. Yeah. I mean, I agree. I think it's, again, it's like the same philosophy we started with, which is like, I don't see this as, as that big of a shift compared to like a lot of the other stuff that's happened in industry. like just, I would just learn these things. 01:15:36.570 --> 01:15:41.800 And frankly, like then you are empowered to say like, I'm not going to use that because you've tried it. diff --git a/youtube_transcripts/518-celebrating-django-s-20th-birthday-with-its-creators.vtt b/youtube_transcripts/518-celebrating-django-s-20th-birthday-with-its-creators.vtt new file mode 100644 index 0000000..485f222 --- /dev/null +++ b/youtube_transcripts/518-celebrating-django-s-20th-birthday-with-its-creators.vtt @@ -0,0 +1,2075 @@ +WEBVTT + +00:00:01.500 --> 00:00:04.300 +Hello, everyone. Welcome to Talk Python To Me. + +00:00:04.400 --> 00:00:08.800 +I'm so excited to be celebrating Django's 20th birthday. + +00:00:09.160 --> 00:00:15.480 +What a milestone. We have Will Vincent, we have Adrian, we have Theobald, we have Jeff, we have Simon. + +00:00:16.960 --> 00:00:21.480 +Many of the original people have the origin of Django here to talk about it. + +00:00:21.600 --> 00:00:23.120 +So welcome all of you. + +00:00:25.240 --> 00:00:25.880 +Thanks, Michael. + +00:00:26.680 --> 00:00:34.160 +Yeah, you bet. It's going to be a lot of fun to go back to some stories and maybe also see where things are going after 20 years. + +00:00:34.380 --> 00:00:36.780 +It's still going strong, to be sure. + +00:00:36.980 --> 00:00:38.560 +So what an accomplishment. + +00:00:39.940 --> 00:00:45.580 +Let's just start with a real quick, who am I, introduction from each of you. + +00:00:45.760 --> 00:00:49.440 +Just give us the elevator pitch on you and your Django life. + +00:00:49.700 --> 00:00:52.400 +And we'll go around the Brady Bunches in order, I suppose, here on the screen. + +00:00:52.700 --> 00:00:54.520 +So Will, welcome. + +00:00:54.650 --> 00:00:56.260 +Okay, so we'll go from least to most important. + +00:00:57.760 --> 00:01:02.320 +So I'm a developer advocate at JetBrains, some Django books, a podcast. + +00:01:04.879 --> 00:01:05.560 +I think that's enough. + +00:01:05.600 --> 00:01:07.700 +We have more important people to come. + +00:01:11.120 --> 00:01:11.400 +All right. + +00:01:11.860 --> 00:01:12.460 +Adrian, welcome. + +00:01:12.740 --> 00:01:13.800 +First time to have you on the show. + +00:01:14.520 --> 00:01:15.080 +Happy to have you here. + +00:01:15.140 --> 00:01:15.260 +Yeah. + +00:01:15.680 --> 00:01:15.800 +Hi. + +00:01:16.600 --> 00:01:20.480 +I'm Adrian, one of the creators of Django many years ago. + +00:01:20.900 --> 00:01:21.620 +Precisely 20. + +00:01:23.119 --> 00:01:24.320 +Long-time web developer. + +00:01:24.720 --> 00:01:26.800 +And yeah, that's basically it. + +00:01:27.860 --> 00:01:28.120 +Awesome. + +00:01:28.620 --> 00:01:29.300 +I'm Simon. + +00:01:29.500 --> 00:01:31.480 +I'm Adrian's intern, effectively. + +00:01:32.560 --> 00:01:37.880 +We built the first version of Django during my year-long paid internship at the Longest Journal World in Kansas. + +00:01:38.640 --> 00:01:42.560 +My university had a year in industry thing, which meant I could get a visa. + +00:01:42.940 --> 00:01:46.440 +We worked on what we called the CMS at the time. + +00:01:46.600 --> 00:01:48.340 +We didn't know that it was a web framework. + +00:01:48.450 --> 00:01:50.580 +We thought it was a tool for building local newspapers. + +00:01:51.500 --> 00:01:52.000 +Yeah. + +00:01:52.400 --> 00:01:54.160 +almost got named TPS, which would be amazing. + +00:01:56.840 --> 00:01:57.900 +Well, I'm on. + +00:01:58.600 --> 00:01:59.080 +Oh, go ahead. + +00:01:59.320 --> 00:02:04.060 +Sorry. Sorry. I was going to say, Simon, you're almost more well known for your AI stuff these days. + +00:02:04.380 --> 00:02:08.619 +You've been in the news so much. + +00:02:08.619 --> 00:02:09.679 +I see you everywhere now. + +00:02:09.820 --> 00:02:12.240 +It's like congratulations on like having to. + +00:02:13.180 --> 00:02:20.100 +Yeah. I mean, that was the trick there is I have a blog and not everyone else gave up on blogging, but I kept going. + +00:02:20.580 --> 00:02:27.840 +And so when AI started up, I was the only person blogging about AI, which turns out gets you invited to all of the weird Silicon Valley mansion parties and things. + +00:02:28.410 --> 00:02:31.140 +So, yeah, that's been an interesting little twist over the past couple of years. + +00:02:31.820 --> 00:02:32.060 +Yeah. + +00:02:32.660 --> 00:02:34.140 +Jeff Triplett, I'm so sorry to cut you off. + +00:02:34.380 --> 00:02:34.760 +Happy to have you. + +00:02:34.790 --> 00:02:35.160 +No, no, no. + +00:02:35.250 --> 00:02:35.520 +All good. + +00:02:36.240 --> 00:02:37.220 +I thought it was pretty free form. + +00:02:37.830 --> 00:02:44.960 +But I came to Lawrence, Kansas two years after these two had left, maybe three years after these two had left to work at the newspaper. + +00:02:45.480 --> 00:02:46.240 +I've stuck around. + +00:02:46.400 --> 00:02:59.920 +probably one of the few people left that stuck around that worked in the journal world, me and Frank Wiles, who's over here someplace. I'm on the Django Software Foundation board of directors. I've talked with DjangoCon US for, I think, 11 years now and do a newsletter with Will and a bunch of + +00:03:00.020 --> 00:03:10.800 +random Django projects. So that's me. Jeff, were you there for the whole of Media for Media, the commercial arm of the newspaper doing Django stuff? About three years of it. It just had moved + +00:03:10.840 --> 00:03:20.040 +across the street to the Little Red building. They called Little Red. Right. I think I moved, When I moved here, it was within a month or two of that starting in the building. + +00:03:21.140 --> 00:03:25.600 +Awesome. I'd love to hear little stories from that era of Django. I missed that bit entirely. + +00:03:27.900 --> 00:03:40.420 +And I'm last. I'm Thibault. I'm on the board with Jeff. I'm the president of the Django Software Foundation for 2025. I've been involved with Django for only 10 years. + +00:03:42.380 --> 00:03:52.240 +So much less than everyone else here. Yeah, outside that, I'm a dev. I'm on a core team for an open source project called Wagtail, which is a CMS as well built on Django. + +00:03:54.260 --> 00:04:04.300 +Yeah, welcome. And Wagtail is a really cool project. I think people who are looking to take the CMS idea and level it up even more. + +00:04:04.850 --> 00:04:26.220 +And you know, I've got a little bit of Wagtail related career history. Wagtail came out of a Torchbox for about six months to a year, quite early on, working on one of the first Django projects that Torchbox worked on. It was a carbon calculator that we were building. I had nothing to do with Wagtail at all. That was probably 10 years after I left Torchbox. + +00:04:26.420 --> 00:04:28.240 +But I know the Torchbox team really well. + +00:04:29.580 --> 00:04:38.820 +We should also, I mean, Jeff, amongst other things, you run Django packages.org right now. And I think Wagtail is number one or two with Django REST framework for most downloaded. + +00:04:39.480 --> 00:04:40.100 +It's something like that. + +00:04:40.130 --> 00:04:40.980 +It's one of the top three. + +00:04:41.310 --> 00:04:42.500 +I feel confident saying that. + +00:04:45.100 --> 00:04:45.740 +Maybe you don't. + +00:04:45.790 --> 00:04:46.040 +I do. + +00:04:46.919 --> 00:04:48.600 +Yeah, it's downloaded a lot. + +00:04:48.630 --> 00:04:50.920 +I don't know that it's top 25. + +00:04:51.300 --> 00:04:51.780 +Tebow would go. + +00:04:52.000 --> 00:04:53.460 +Well, we get a post, right? + +00:04:54.650 --> 00:04:55.280 +Okay, Tebow, go. + +00:04:55.900 --> 00:04:57.700 +I think it's in the top 100 for sure. + +00:04:57.830 --> 00:05:00.880 +So you're close amongst 20,000 or so plus. + +00:05:01.980 --> 00:05:03.600 +Wait, of Django ones? + +00:05:04.180 --> 00:05:05.300 +Yeah, of Django ones. + +00:05:05.800 --> 00:05:06.480 +What is that? + +00:05:06.520 --> 00:05:08.020 +There's like 4,000 now, Jeff? + +00:05:08.480 --> 00:05:27.880 +like that i think it's probably like 16 000 20 000 but i think we're tracking like okay four or five thousand it's what people uh i never wanted to add code that just pulled everything in just to have a lot of numbers so it's still like user user submitted and ran so but tiboh github stars + +00:05:28.100 --> 00:05:31.740 +that's the most important thing how many is it for wagtail it is the most important thing + +00:05:33.660 --> 00:05:48.940 +uh no it isn't but yeah i think we just like almost at 20 000. i'm pretty sure if you're on here listening to this show stars right now will be at 20 000 in like a week yeah it's really high in downloads as well but you know the django ecosystem is really big these days and cms only + +00:05:49.060 --> 00:06:06.540 +definitely in a smaller portion of it oh there you go yeah very very very popular that's amazing so let's start let's start at the beginning folks i have on the screen here a picture of of the University of Kansas, where I also have a connection to. + +00:06:06.640 --> 00:06:11.140 +I actually went here and got my undergraduate degree in math at this school. + +00:06:11.220 --> 00:06:12.780 +And I love my time in Lawrence. + +00:06:12.900 --> 00:06:20.900 +I think it's a lovely town, but it's also the home of Lawrence Journal World, right? + +00:06:21.900 --> 00:06:24.160 +Tell us who wants to start the story. + +00:06:24.300 --> 00:06:26.560 +Like what's the origin story here? + +00:06:27.580 --> 00:06:29.440 +- Probably makes sense for me to start it. + +00:06:29.640 --> 00:06:30.480 +- Yeah, I think so. + +00:06:30.740 --> 00:06:31.040 +Let's do it. + +00:06:32.180 --> 00:06:39.200 +Back in 2002/2003, early 2003, I joined that newspaper as a web developer. + +00:06:40.870 --> 00:06:46.600 +The guy who ran it was this guy called Rob Curley, who was seen as this big tech innovator in the news industry. + +00:06:46.780 --> 00:06:58.200 +He recruited me and put together a great team of awesome web developers, believe it or not, in this tiny little newspaper in the middle of Kansas. + +00:06:59.300 --> 00:07:07.180 +We were tasked with building interactive database-driven apps for the newspaper, which at that time was unheard of. + +00:07:07.290 --> 00:07:11.060 +Even these days, it's kind of weird for a newspaper to have developers on its staff. + +00:07:13.500 --> 00:07:14.960 +And at the time, I was using PHP. + +00:07:16.000 --> 00:07:24.660 +And, yeah, fast forward maybe a year into the job, we had an opening to hire another developer. + +00:07:25.520 --> 00:07:27.700 +And I posted a job entry on my blog. + +00:07:28.880 --> 00:07:31.540 +And Simon read it all the way in the UK. + +00:07:32.820 --> 00:07:37.040 +And we followed each other's blogs back when that was like the social media of the day. + +00:07:37.340 --> 00:07:37.920 +To be fair. + +00:07:38.030 --> 00:07:40.160 +Much more interesting than social media of today. + +00:07:40.740 --> 00:07:45.100 +At the time, I felt like there were about 100 people blogging about web development. + +00:07:45.350 --> 00:07:46.160 +And we were two of those. + +00:07:46.400 --> 00:07:46.560 +Yeah. + +00:07:46.670 --> 00:07:48.520 +And we all followed everyone. + +00:07:49.420 --> 00:07:50.720 +Did you guys have a blog role? + +00:07:51.360 --> 00:07:52.380 +Oh, yeah, absolutely. + +00:07:52.780 --> 00:07:53.380 +Blog role. + +00:07:53.510 --> 00:07:53.640 +Yeah. + +00:07:53.960 --> 00:07:56.520 +But I think we were both massive fans of Mark Pilgrim. + +00:07:56.720 --> 00:08:00.600 +Mark Pilgrim was the web development of Python blogger + +00:08:00.600 --> 00:08:02.240 +that we were paying attention to. + +00:08:03.480 --> 00:08:04.760 +Unfortunately, he's not around anyway. + +00:08:04.980 --> 00:08:07.960 +I mean, he's alive, but he stopped blogging many, many years ago. + +00:08:08.900 --> 00:08:15.460 +Yeah, he wrote an online book called Dive into Python, and we both read that and were like, oh, what's this Python thing? + +00:08:16.559 --> 00:08:23.980 +And thanks to the kindness or possibly ignorance of our boss, We just started using Python one day. + +00:08:24.740 --> 00:08:26.660 +We didn't really have to get much buy-in or anything. + +00:08:28.280 --> 00:08:34.520 +And we'd started making little bits and pieces in Python, never really with the intent of making a framework. + +00:08:34.860 --> 00:08:42.659 +But over time, we just, at some point, we just like decided, okay, we're switching to Python, no more PHP. + +00:08:42.770 --> 00:08:44.280 +We're just going to do everything in this new thing. + +00:08:45.280 --> 00:08:49.619 +And I remember the key thing for us is how do you deploy Python? + +00:08:50.040 --> 00:08:52.240 +Well, there was modpython, the Apache module. + +00:08:52.660 --> 00:08:56.020 +And we were just a little bit nervous about it because not many other people were using it. + +00:08:56.050 --> 00:08:59.060 +And so we're thinking, OK, well, we can try modpython, but what if it doesn't work? + +00:08:59.210 --> 00:09:07.180 +So OK, well, if we have an abstraction layer between our code and modpython, like a request and a response object, then if modpython doesn't work out, we can swap it out for something else. + +00:09:07.320 --> 00:09:07.840 +We didn't. + +00:09:07.880 --> 00:09:08.820 +Modpython worked great. + +00:09:09.450 --> 00:09:16.680 +But that abstraction layer, I think that was the origins of Django, was that tiny layer between our code and modpython, just in case. + +00:09:17.040 --> 00:09:19.280 +Classic thing you shouldn't do in programming, right? + +00:09:19.560 --> 00:09:20.280 +You're not going to need it. + +00:09:20.300 --> 00:09:23.120 +You don't need the abstraction layer, but we built one anyway. + +00:09:23.899 --> 00:09:27.400 +And fast forward 20 years, and maybe that was a good idea after all. + +00:09:28.760 --> 00:09:29.900 +Is ModPython still around? + +00:09:30.400 --> 00:09:31.140 +It's got to be, right? + +00:09:31.980 --> 00:09:32.540 +I don't know. + +00:09:32.680 --> 00:09:35.760 +I mean, ModWhiskey, it was replaced by ModWhiskey many years ago. + +00:09:37.100 --> 00:09:38.420 +But, yeah, I don't know, actually. + +00:09:38.580 --> 00:09:41.240 +I've not checked in on the ModPython project in a while. + +00:09:44.040 --> 00:09:48.380 +It's been last commit eight months ago to ModPython on GitHub. + +00:09:48.940 --> 00:09:49.340 +Yeah, yeah. + +00:09:49.440 --> 00:09:50.780 +That sounds alive. + +00:09:51.240 --> 00:09:52.340 +Yeah, yeah, it's definitely alive. + +00:09:52.660 --> 00:09:55.340 +And now we have WSGI and ASGI. + +00:09:56.480 --> 00:09:58.560 +That's sort of carrying on the same theme in a way. + +00:09:59.080 --> 00:10:15.100 +You know, WSGI kind of came out of the work we were doing in that we were working on Django, the general world, and there was a mailing list popped up, the Python Web SIG mailing list, to discuss how this stuff should work. + +00:10:16.060 --> 00:10:20.260 +And we were -- sorry about that. + +00:10:20.850 --> 00:10:22.240 +Yeah, this mailing list popped up. + +00:10:22.990 --> 00:10:27.180 +And we were very active on that initially, trying to pitch our request and response objects. + +00:10:27.520 --> 00:10:31.240 +We did not win with our response and request objects, and Whiskey came out of that instead. + +00:10:31.390 --> 00:10:38.420 +But yeah, that was definitely, Whiskey came out of that early era of what we were trying to achieve with the stuff we were doing. + +00:10:40.560 --> 00:10:51.440 +So we built up this series of libraries slash tools slash I guess it turned into a framework and we started thinking, oh, this is kind of interesting. + +00:10:52.540 --> 00:11:13.860 +Personally speaking for myself, I was like, oh, I'm probably not going to work you forever, but I would love to keep using this. So that's sort of started planting the seed about open sourcing it. But I think very few people today know that even 20 years ago, there was this meme that there are way too many Python web frameworks. So we went to PyCon US and it had to have been 2004. + +00:11:15.660 --> 00:11:24.140 +And there was a keynote where the speaker said she showed a slide that was like a tag cloud of all the Python web framework names. + +00:11:24.690 --> 00:11:25.880 +And there were like 50 of them. + +00:11:27.640 --> 00:11:30.500 +So Simon and I were sitting there like, oh, shoot. + +00:11:30.720 --> 00:11:32.260 +Like we may get another one. + +00:11:32.360 --> 00:11:32.880 +We're fools. + +00:11:33.030 --> 00:11:33.740 +We're so stupid. + +00:11:34.000 --> 00:11:34.400 +We're embarrassed. + +00:11:35.500 --> 00:11:41.620 +But I ended up doing a lightning talk at that Python and showed the, you know, what we had. + +00:11:41.620 --> 00:11:43.220 +It didn't have a name yet. + +00:11:44.240 --> 00:11:47.860 +And, you know, people came up to me afterward and said, oh, please open source. + +00:11:48.340 --> 00:11:49.380 +We're very interested in this. + +00:11:49.400 --> 00:11:53.180 +It looks very promising, open source, even though it's yet another Python web framework. + +00:11:53.440 --> 00:12:00.000 +So I guess extrapolating a lesson from that, it's never too late to make stuff. + +00:12:00.380 --> 00:12:04.660 +I think another fact there is that Ruby on Rails had come out maybe six months before that PyCon. + +00:12:04.780 --> 00:12:09.800 +Like Ruby on Rails was the absolute buzz of everything in the web development industry at the time. + +00:12:10.320 --> 00:12:14.000 +And there was like an open question, like what's Python's answer to Ruby on Rails? + +00:12:14.320 --> 00:12:18.260 +And so it's possible that the Django timing was really good on that front. + +00:12:18.420 --> 00:12:22.120 +I believe that Rails helped influence the newspaper to open source Django. + +00:12:22.280 --> 00:12:23.840 +Was that part of the conversation then? + +00:12:24.780 --> 00:12:31.520 +Well, I remember making a list of 10 to 20 reasons why we should open source it. + +00:12:32.340 --> 00:12:36.300 +And really thinking hard about it, probably lost some sleep about it. + +00:12:36.600 --> 00:12:41.720 +And we went into the meeting with Rob to make our pitch. + +00:12:41.940 --> 00:12:46.320 +and we only got to like five reasons and finally he was like, yeah, let's do it. + +00:12:46.400 --> 00:12:46.740 +Let's do it. + +00:12:47.580 --> 00:12:47.660 +Wow. + +00:12:47.760 --> 00:12:51.160 +So I mean, endless, infinite credit to him for being cool. + +00:12:52.060 --> 00:13:06.020 +And I mean, the company took advantage of open source a lot and I think the higher-ups knew it because it also owned a cable company that did a lot of stuff with Linux and all sorts of crazy cable company software that was open source. + +00:13:06.760 --> 00:13:08.000 +Which was Frank, right? + +00:13:08.160 --> 00:13:10.160 +That was Frank Wiles at the cable company + +00:13:10.500 --> 00:13:11.660 +driving all of that stuff. + +00:13:13.940 --> 00:13:27.540 +yeah you know i think it's worth pointing out as well one adrian you're making me nostalgic just to think about tag clouds okay they were such a thing back in the early 2000s such it's a usability + +00:13:27.980 --> 00:13:34.420 +nightmare i know any kind of data information out of that thing it really is but i think i use um + +00:13:35.200 --> 00:13:42.060 +oh my god delicious a lot and you'll get tag clouds out of it and all sorts of stuff But that's not why I'm bringing this up. + +00:13:42.220 --> 00:13:49.720 +More in that time frame, open sourcing something was not the clear choice it is today. + +00:13:49.980 --> 00:13:52.620 +At least it was really more of a push, I think. + +00:13:54.200 --> 00:13:56.420 +Legal liabilities, giving away our IP. + +00:13:56.710 --> 00:13:57.620 +Like it was a different time. + +00:13:57.760 --> 00:13:59.940 +There was open source, but not like today. + +00:14:01.300 --> 00:14:07.720 +Well, also the publisher of the newspaper at the time still used a mechanical typewriter to write up his things. + +00:14:07.940 --> 00:14:09.140 +He was legendary for it. + +00:14:09.200 --> 00:14:31.380 +but my understanding is that he got into it the argument that was convincing to the newspaper was we benefit from this community, we should give back to this community and that evidently was part of the pitch but yeah and then I mean it just took off right, I feel like that newspaper did very well from that one decision that they made back then + +00:14:32.380 --> 00:14:56.980 +yeah I mean here we are this many years later talking about it on a podcast right, like already a lot of people know about it and a lot of people all of you on the call but many many many others have contributed back to django to make so much better if that was just an internal cms that you guys left there how many external contributors would you have right not that many + +00:14:58.920 --> 00:15:14.620 +well one thing actually i'm curious while the two of you are here i heard a story that there were some efforts to monetize it that didn't pan out jeff maybe you were there for that is that is Were there some efforts to package it up and sell it to newspapers in some capacity as a premium product? + +00:15:18.000 --> 00:15:18.520 +I think. + +00:15:18.690 --> 00:15:19.400 +Am I making that up? + +00:15:20.580 --> 00:15:21.800 +That was after my time. + +00:15:22.410 --> 00:15:22.600 +I think. + +00:15:23.050 --> 00:15:23.800 +Are you talking about... + +00:15:23.940 --> 00:15:24.920 +Maybe Frank mentioned that. + +00:15:24.960 --> 00:15:26.860 +I thought there was something around... + +00:15:26.960 --> 00:15:27.960 +This was Ellington. + +00:15:28.440 --> 00:15:29.040 +I think we should... + +00:15:29.040 --> 00:15:29.420 +Oh, I guess, yeah. + +00:15:29.720 --> 00:15:29.780 +Sorry. + +00:15:30.980 --> 00:15:33.140 +This is kind of my era, I guess, then. + +00:15:34.040 --> 00:15:36.339 +I still feel like I'm new to Django because I felt like... + +00:15:36.360 --> 00:15:41.500 +I've been around for 18 years, but I saw these guys code so much, and then I came into that era. + +00:15:41.720 --> 00:15:44.540 +So to me, it felt like next-gen Star Trek versus the originals. + +00:15:45.500 --> 00:15:54.760 +And so, yeah, when I started, we had hundreds of newspapers were buying Ellington, which was basically like, what was it, the first or second Django app ever written? + +00:15:55.360 --> 00:15:57.880 +And they basically created a CMS to run your newspaper. + +00:15:58.350 --> 00:16:03.040 +And then they also had something called Marketplace, which was almost like an online Craigslist or business directory. + +00:16:03.840 --> 00:16:09.600 +And so I came in to kind of help manage some of the inventory and do a lot of data imports and stuff for newspapers. + +00:16:09.880 --> 00:16:12.040 +Like we had all of Massachusetts at one point. + +00:16:13.140 --> 00:16:14.180 +There was a lot of kind of that. + +00:16:14.300 --> 00:16:15.780 +They went that direction with it. + +00:16:16.300 --> 00:16:19.040 +And I left before that kind of had shuttered. + +00:16:19.080 --> 00:16:22.160 +And I think they sold it to a big company to buy it. + +00:16:22.220 --> 00:16:24.200 +And I think Ellington still runs to this day. + +00:16:25.400 --> 00:16:29.940 +At DjangoCon every two or three years, somebody will approach and say, we want to open source Ellington. + +00:16:30.120 --> 00:16:30.760 +Would you like that? + +00:16:31.100 --> 00:16:32.100 +Could we do this at DjangoCon? + +00:16:32.240 --> 00:16:49.540 +we always say, yeah, that would be really cool. And then six years has gone by and it happens every so often. I hope that code gets to see the light of day because it was hundreds of apps by the time we worked on it for hundreds of newspapers. I feel like that one, the product was + +00:16:49.580 --> 00:17:11.020 +fantastic and the market was great but it's a terrible market to be selling into. Like in the late 2000s, early 2010s, if you're selling to local newspapers which are blinking out of existence to the rate of knots. My hunch is that the marketplace challenges were down to that. It was selling into a shrinking industry, which is a difficult thing to do. + +00:17:11.880 --> 00:17:28.260 +Yeah, I think they were hitting a brick wall for sure. You mentioned Rails earlier and PHP, and two things come to mind. I have an embarrassing email I sent to Simon at one point because I followed your PHP blog and also Matt Croydon, who later became my boss at Media for Media, And I did not. + +00:17:28.339 --> 00:17:29.340 +I saw a couple of things. + +00:17:29.540 --> 00:17:32.420 +I think you were so into Django, you weren't really posting about Django. + +00:17:32.820 --> 00:17:36.060 +And so I think I emailed you once to say, have you seen this thing called Django? + +00:17:36.440 --> 00:17:37.360 +It looks pretty cool. + +00:17:37.580 --> 00:17:39.440 +Now that you're doing Python, I think you may like it. + +00:17:40.020 --> 00:17:43.200 +And so if you ever check your Gmail, it may be there. + +00:17:43.380 --> 00:17:44.760 +We don't have to do anything live on the air. + +00:17:44.980 --> 00:17:46.460 +But yeah, I did this day. + +00:17:46.720 --> 00:17:49.100 +I think, oh, man, that was kind of wild. + +00:17:50.040 --> 00:17:55.260 +What brought me to Django, though, was I did get the Rails book because they very quickly created a Ruby on Rails book. + +00:17:55.380 --> 00:17:59.760 +It was kind of the main way to learn how to use it because their website kind of sucked as far as documentation went. + +00:18:00.240 --> 00:18:05.200 +And as I was going through it, so much was boilerplate just to get anything running in Rails. + +00:18:05.780 --> 00:18:14.000 +And so the Django admin was just so beautifully designed and well done that to me it was just this win of like this looks better than anything I could do by hand. + +00:18:14.460 --> 00:18:16.820 +And it's largely stood the test of time for 20 years. + +00:18:17.800 --> 00:18:22.420 +I remember like Wilson Minor I had met a couple of times had done some of that work or most of that work. + +00:18:22.880 --> 00:18:26.960 +He did all of that design work and he designed the original Django website. + +00:18:27.180 --> 00:18:38.200 +And I think that's one of the main reasons that we got a lot of traction was because the design of the site and the documentation were so good, especially considering the context. + +00:18:38.260 --> 00:18:48.780 +I mean, back in those times, it was par for the course for an open source project to be completely unstyled, like a normal browser default H1. + +00:18:49.400 --> 00:18:49.860 +Craigslist. + +00:18:49.940 --> 00:18:50.500 +White background. + +00:18:52.000 --> 00:18:53.960 +Now, this design isn't the one that Wilson did. + +00:18:54.510 --> 00:18:55.280 +The one that's... + +00:18:55.280 --> 00:18:57.440 +Yeah, you'd have to archive.org to go back. + +00:18:57.760 --> 00:19:00.080 +But it was pretty timeless at the time, though. + +00:19:00.080 --> 00:19:00.580 +It was amazing. + +00:19:01.050 --> 00:19:05.520 +And I feel like the admin has evolved a lot, but it still has that sort of... + +00:19:05.520 --> 00:19:08.840 +It feels like the Wilson-minded design today, even 20 years later. + +00:19:11.880 --> 00:19:12.860 +Yeah, Wilson went... + +00:19:13.260 --> 00:19:15.420 +Wilson was there the same time I was at the Journal World. + +00:19:15.470 --> 00:19:18.860 +And he then went on to early Facebook, he was. + +00:19:19.040 --> 00:19:20.360 +He's bounced around Silicon Valley. + +00:19:21.300 --> 00:19:23.240 +Yeah, he designed Apple.com, I think, too. + +00:19:23.720 --> 00:19:26.420 +Back when he worked with me at every block for a few years. + +00:19:27.180 --> 00:19:28.480 +He's now working on Apple Music. + +00:19:29.740 --> 00:19:30.840 +I didn't know that. Excellent. + +00:19:31.720 --> 00:19:36.040 +As of last time I talked to him, which was recent ish. + +00:19:37.540 --> 00:19:38.160 +Yeah, that's quite neat. + +00:19:41.640 --> 00:19:43.120 +Trying to pull it up here in the Wayback Machine. + +00:19:43.300 --> 00:19:43.600 +We'll see. + +00:19:44.960 --> 00:19:45.820 +See what I can get. + +00:19:45.880 --> 00:19:46.640 +I think the other thing. + +00:19:47.140 --> 00:19:48.460 +How about this, guys? Is this the OG? + +00:19:48.600 --> 00:19:48.980 +There we go. + +00:19:49.300 --> 00:19:55.120 +Yep, yep. That's the one. That, to be fair, that looks good by today's standard. + +00:19:55.640 --> 00:20:01.940 +You know, I feel like I would not be at all upset to publish a website look like that in 2025. + +00:20:03.940 --> 00:20:05.300 +Yeah, that is pretty timeless. That's cool. + +00:20:06.080 --> 00:20:07.940 +Wait, I see the... Oh, go ahead, Simon. + +00:20:08.520 --> 00:20:13.660 +The other thing that's worth mentioning with Django, Django's documentation has always been fantastic. + +00:20:14.540 --> 00:20:20.660 +And I think that's basically because it was Jacob Kaplan Moss had a, he was a literature major. + +00:20:21.180 --> 00:20:22.740 +And you have Adrian, who's a journalism major. + +00:20:22.740 --> 00:20:29.120 +And it turns out if you put a journalist and somebody who put that talent on your documentation, you get really good documentation. + +00:20:31.700 --> 00:20:37.840 +That was a double-edged sword because it took a few years for any publisher to be interested in publishing a book about Django. + +00:20:38.680 --> 00:20:42.360 +Because the standard line was always, oh, the docs are good enough. + +00:20:42.460 --> 00:20:47.980 +everyone just there's no market for a book well but at the same time i wanted to ask i mean because + +00:20:48.180 --> 00:20:59.960 +there was the link there to the django book which i have the url now and jeff and i run it lists all the django books but i mean the two of you wrote that i thought that was a great book i think you + +00:21:00.540 --> 00:21:07.320 +it was in print right but it's mainly open source adrian yeah it was both it was free online and it was published by A Press. + +00:21:07.740 --> 00:21:09.140 +There were a couple editions, yeah. + +00:21:11.760 --> 00:21:19.480 +The interesting question for Paul Everett out there in the audience is, in the first years, did your team also have to admin the newspaper site? + +00:21:19.500 --> 00:21:21.880 +Like, did you DevOps before DevOps was a thing? + +00:21:22.620 --> 00:21:23.460 +Or how did you run it? + +00:21:23.540 --> 00:21:25.300 +The cloud stuff wasn't that big back then. + +00:21:25.800 --> 00:21:27.200 +No, we had our own physical servers. + +00:21:28.060 --> 00:21:28.700 +In the basement. + +00:21:29.240 --> 00:21:29.820 +In the basement. + +00:21:30.540 --> 00:21:31.480 +The team was in the basement. + +00:21:31.480 --> 00:21:32.580 +You could probably hear them. + +00:21:32.660 --> 00:21:41.900 +programmers and yeah there was a there was a room in the basement with with physical servers and oh i've forgotten their names now i think frank frank helped well they were named after characters + +00:21:42.100 --> 00:21:49.680 +and beatle songs one was mr mustard one was pam from polytheon pam yes um yep but yeah and and + +00:21:50.000 --> 00:22:06.640 +frank and nick from from sunflower cable helped us wire those up but yeah we it was it was a fit there was a there was a max there was an apple server in there as well that was the media server I think it was doing like imagery sizing. Yeah, it was like a 2U box or something. I forget. It + +00:22:06.680 --> 00:22:09.200 +was actually like a physical server that mounted. I'd never seen one before. + +00:22:10.980 --> 00:22:17.680 +XServe. It was an XServe. Yeah. Also, I remember that room had a wire that was the AP News wire. + +00:22:18.340 --> 00:22:23.440 +Like that I didn't realize there was a physical wire that the news came in on from the Associated + +00:22:23.580 --> 00:22:27.500 +Press. And that when you hear it came in over the wire, that was the wire? That was the wire. + +00:22:27.780 --> 00:22:33.460 +we have the actual wire in our basement yeah incredible well simon and i were lucky because + +00:22:33.540 --> 00:22:57.800 +we didn't really have any sys admin experience but there were some really experienced guys frank and nick who worked at the company and uh actually like did some uh like training with us i remember to go going into their dark room all the lights were like turned off you could only see like these unix shells i was like whoa i'm really learning the good stuff and this is the + +00:22:57.880 --> 00:23:08.620 +same frank this is frank wiles who he's been president of the django software foundation in the past and he's still very heavily involved with django to this day yeah all right let's talk about + +00:23:08.840 --> 00:23:47.439 +2008 a lot of stuff happened around 2008 that was the time frame a little bit before then django's open source but maybe you all have to tell me maybe a little bit inspired by ruby on rails like there's a lot of in ruby on rails a lot of convention over configuration a lot of magic and django used to have a little more implicit magic and there was the when django went to around when went to 1.0 there's the magic removal and it was the first django con all those things tell us about that whoever wants to take it i can talk to django con a little bit the very first + +00:23:47.460 --> 00:24:24.240 +DjangoCon. So I was working for GCAP Media, a radio station in London that was doing a big Django project and one of my co-workers there was this chap, Rob Lofthouse, who very foolishly stepped up and said, well okay, I mean everyone says there should be a conference and no one's arranging a conference. I'll arrange a conference, how hard can it be? The answer is everyone in the school understands it. Very, very, very hard and he managed to pull it off and the big thing there was Google offered to host it. So we got the venue and the catering and everything was in Google's campus in Mountain View. Was 2008, was that the year of the first DjangoCon? I think it was, right? + +00:24:24.440 --> 00:24:29.400 +I think it was. And yeah, we pulled together a conference. + +00:24:31.680 --> 00:25:01.160 +And the videos from that conference were all available on YouTube, actually. I rewatched one of the panels quite recently just out of interest. And it was really fascinating seeing There are people on those panels. Russell was on the database migrations panel. He was one of the people trying to solve database migrations back then. He's been an enormous fixture in the Python and Django community ever since. That was really fun. How many of us were there? Jeff, were you at + +00:25:01.310 --> 00:25:18.600 +the first Django conference? I was. I was barely working at the journal world like maybe six months or a year or something so yeah i just just kind of experienced it all as a nobody so it's great and i think my biggest contribution to that conference was + +00:25:19.020 --> 00:25:45.740 +so i was working on a migration system called demigrations and andrew godwin was working on i think he was calling it south back then um the like his migration and we had russell keith and i got i managed to i made the pitch for andrew who was just a student at university at the time to be flown to California to present at this conference. I think it may have been his first ever conference talk. And he has been to and spoken at every Django and Python conference since. + +00:25:45.920 --> 00:25:51.300 +So I like to think I helped just provide that little spark to get his speaking career started. + +00:25:52.740 --> 00:25:56.960 +But yeah, that was a very, very notable moment, that first DjangoCon. + +00:25:58.140 --> 00:25:59.820 +I had a mind-blowing moment there. + +00:26:01.760 --> 00:26:06.380 +A guy from, I think it was Japan, came up to me. + +00:26:06.500 --> 00:26:12.660 +He didn't speak a word of English, and he gave me his book that he had written about Django. + +00:26:13.160 --> 00:26:13.640 +Whoa. + +00:26:14.080 --> 00:26:18.200 +And he just bowed and went away. + +00:26:18.440 --> 00:26:22.340 +And I still think about that, and it still kind of brings a tear to me. + +00:26:22.680 --> 00:26:23.320 +It's like, what? + +00:26:23.980 --> 00:26:29.940 +It's mind-blowing that people all around the world that you've never met who speak languages you don't speak use your code. + +00:26:31.580 --> 00:26:32.740 +It's been 20 years. + +00:26:32.750 --> 00:26:37.280 +I still can't even 100% come to terms with it. + +00:26:37.720 --> 00:26:38.620 +It's really cool. + +00:26:40.500 --> 00:26:41.980 +See, I thought you were going to go a different direction. + +00:26:42.420 --> 00:26:47.960 +And you were going to say when Cal Henderson put that sign up of the unicorn, that was going to forever be the moment. + +00:26:48.720 --> 00:26:50.620 +I don't know if you want to expand on that. + +00:26:50.620 --> 00:26:52.740 +Well, that's the downside of that first DjangoCon. + +00:26:53.200 --> 00:27:23.160 +there was a q a with me and jacob and i think it was me i said are there any pony requests and that was intended to mean are there any future ideas sort of open suggestions and evidently that was the um the impetus for the idea of the django pony which i despise i think it's so dumb and i + +00:27:23.180 --> 00:27:23.580 +Oh, no. + +00:27:23.880 --> 00:27:27.680 +I know it's funny in a way, but it's also infuriating to me. + +00:27:27.690 --> 00:27:30.020 +I think it's so stupid and unprofessional. + +00:27:30.230 --> 00:27:32.700 +And the type of humor is just dumb. + +00:27:33.580 --> 00:27:34.540 +No wit at all. + +00:27:34.720 --> 00:27:35.660 +I'll stop my rant. + +00:27:35.880 --> 00:27:36.580 +That'll be on my own. + +00:27:36.620 --> 00:27:40.460 +Whenever I start my own podcast, I'm going to have the first episode on why the Django pony sucks. + +00:27:41.240 --> 00:27:42.320 +Wait, what should it be, though? + +00:27:42.530 --> 00:27:43.440 +What would you like it to be? + +00:27:43.610 --> 00:27:44.280 +Do you have thoughts? + +00:27:45.700 --> 00:27:46.520 +Nothing is needed. + +00:27:49.480 --> 00:27:50.520 +I think you underestimated. + +00:27:51.200 --> 00:27:52.620 +It should at least be playing a guitar. + +00:27:53.400 --> 00:27:56.140 +Yeah, I was going to say a guitar or a musical instrument or something. + +00:27:56.710 --> 00:28:01.860 +Yeah, people love the little stickers celebrating this or that event and occasion. + +00:28:03.580 --> 00:28:04.620 +Qualified people. + +00:28:05.730 --> 00:28:08.080 +Well, Thibaut, I have one of your stickers on my fridge. + +00:28:09.320 --> 00:28:09.460 +Yes. + +00:28:10.360 --> 00:28:19.320 +No, no, but yeah, that's one of the things I did for the 20th birthday is design a new pony to distribute to people around the world. + +00:28:19.630 --> 00:28:21.620 +You might have seen one at the Amsterdam event. + +00:28:23.240 --> 00:28:26.480 +A bag of the stickers was passed to me and I passed it along. + +00:28:28.400 --> 00:28:29.060 +To the trash can. + +00:28:29.500 --> 00:28:29.980 +No, just kidding. + +00:28:30.920 --> 00:28:41.480 +Another notable thing that happened in 2008, other than the housing crisis crash that wrecked a bunch of things, was the foundation, right? + +00:28:42.600 --> 00:28:43.540 +The Django Software Foundation. + +00:28:45.900 --> 00:28:48.840 +And I think more broadly, I think one of the things that makes... + +00:28:48.840 --> 00:28:58.080 +I talk to a lot of different people about different web frameworks And the thing that really makes Django stick and stand out, I think, is one, the admin thing. + +00:28:58.220 --> 00:29:00.580 +But the other is the community around Django. + +00:29:00.980 --> 00:29:02.980 +And we're talking about the Django conference, the foundation. + +00:29:03.300 --> 00:29:06.080 +It's just unlike other things. + +00:29:07.140 --> 00:29:14.060 +There's a whole community that you get into when you go into Django, not just some APIs you use. + +00:29:14.060 --> 00:29:14.600 +You know what I mean? + +00:29:16.100 --> 00:29:20.100 +So who wants to tell us about how the foundation got started? + +00:29:25.320 --> 00:29:25.340 +so + +00:29:25.660 --> 00:30:02.760 +i was there uh so the there was the question of who owns the code um it was i think technically owned by the lawrence journal world correct me if i'm wrong um and so that wasn't going to last forever so we needed to find for legal you know ownership reasons an entity to own it. We didn't want to go the route of Ruby on Rails. Copyright, David Hennemeyer Hanson. I think that's just really cringe. So it's good to have an organization as part of it. So yeah, we set up 501c3 if that's the right one that we use. I forget. + +00:30:04.620 --> 00:30:11.340 +And yeah, that's basically the origin story. From my perspective, I'm sure there's a lot more subtlety to it. + +00:30:18.080 --> 00:30:18.220 +Nice. + +00:30:18.440 --> 00:30:19.500 +Thibode, do you want to weigh in? + +00:30:19.620 --> 00:30:20.540 +You have-- Yeah, go ahead, Thibode. + +00:30:20.580 --> 00:30:21.780 +You're the president, and you have-- + +00:30:21.860 --> 00:30:22.320 +Oh, gosh. + +00:30:23.920 --> 00:30:26.200 +I was only barely started at that time. + +00:30:26.440 --> 00:30:29.660 +So I definitely relate to the community around Django. + +00:30:30.320 --> 00:30:33.000 +You were all telling why you got involved and when and how. + +00:30:33.110 --> 00:30:35.960 +And for me, it was Whacktail as a day job. + +00:30:36.480 --> 00:30:51.800 +But then what got me to stick around actually Django girls and organizing some of those local workshops. And you know, having organizations like this, like alongside the Django projects, not even the main thing is definitely only possible because we've had such a like long running open source foundation. + +00:30:53.020 --> 00:31:02.580 +There's lots of aspects of the foundation we could discuss. But definitely, you know, that kind of longevity for community run open source is a is rare and to be cherished. + +00:31:03.460 --> 00:31:17.320 +yeah maybe you could tell us a little bit like what was it when it started and what is it now like how has it changed over the time over the over the years i'm going to put in a plug for + +00:31:17.420 --> 00:31:37.280 +the django fellows program i feel like the django fellows program has been an absolutely astonishingly effective part of the mission of the foundation for people who aren't familiar with that um The Django Software Foundation employs fellows who are paid contributors who work on the bits of open source maintenance that aren't necessarily so glamorous. + +00:31:37.440 --> 00:31:43.940 +They do bug issue. They do code review and security, handle security incidents and bug triage and all of these different things. + +00:31:44.360 --> 00:31:49.100 +And thanks to them, Django releases on a has a release schedule that operates like clockwork. + +00:31:49.460 --> 00:31:52.780 +Like Django, there's a new release that comes out when they said it was going to come out. + +00:31:53.200 --> 00:31:56.180 +Code gets reviewed. Bugs get triaged. It's a wonderful, wonderful thing. + +00:31:56.640 --> 00:32:01.120 +And it's unique because most open source projects that are run like that have a company behind them. + +00:32:01.240 --> 00:32:04.860 +You know, there are people who have a full time job and there's a company that's commercially supporting that. + +00:32:05.180 --> 00:32:08.600 +Django does not have a sort of commercial sponsor in that respect. + +00:32:08.860 --> 00:32:16.240 +It has this nonprofit that people fund and that employs these fellows and the fellows keep everything ticking long. + +00:32:16.660 --> 00:32:24.960 +I have I keep on saying to other open source projects, if you can figure out a pattern to use that's similar to the Django fellows, that's absolutely worth doing. + +00:32:25.040 --> 00:33:10.480 +The Python software foundation has a developer in residence program now, or several developers in residence, which were inspired by the Django Fellows program. I think it's been really influential. It's notable because there are very few major open source projects that don't have a company behind them. The other one that comes up a lot is Postgres. Postgres is very much a community funded, a community organized program. If you look at the way Postgres works, a lot of the key maintainers have their own consultancies. So they've got a good sort of way of financially sustaining the work they're doing i mean i think the django fellows program which was introduced um when was that that was at least 10 years ago the fellows program started i think it's been an enormous success yeah absolutely you know another thing that i think is really + +00:33:10.700 --> 00:33:35.700 +interesting um it's a little bit newer but is the django knots program right and django girls as well shout out to them you know so many people i've talked to like i got into python because i wanted to do django which for you all is probably a little bit mind-blowing as well like they didn't use django because they were python but the other way around the first time i met somebody said oh + +00:33:35.780 --> 00:33:56.380 +yeah i got into python because of django and i was absolutely bowled away i i always assumed you'd have to be a proficient python program before you could start picking up something like django it turns out it's the other way around people would would get in from because partly because the django documentation was so good. But so many people got their start in the Python community as web developers who dabbled with Django and then got into Python from there. + +00:33:57.160 --> 00:34:01.020 +Yeah. Kenneth, I love it in the audience. I learned Python through Django. Very cool. + +00:34:01.260 --> 00:34:05.580 +When I chaired DjangoCon US, I'd always ask the audience. For one, I was always a nervous speaker. + +00:34:05.960 --> 00:34:41.359 +So any time you ask the audience to do something, it takes my nerves away. But I'd always ask who learned Python by learning Django. And over half the room every year would always raise their hands up so it was a heck of a gateway to learn python um also uh we just hired our third fellow jacob walls last week who i think starts this next monday so jane goes up to three fellows now um and it's just been the load is just incredible so trying to keep up um i think our existing fellows have over a thousand tickets that they're trying to sort through and you know if people are watching and want to help fund uh contact tibo and i and we will uh definitely + +00:34:41.379 --> 00:34:47.899 +help you give money to the dsf to support more fellows yeah and it makes such a difference yeah + +00:34:48.800 --> 00:34:53.340 +we'll give you our annual reports which details everything the foundation does and more on the + +00:34:53.340 --> 00:35:24.600 +other side well i just want to make one point on that because i mean t-bone and jeff you know now but the finances for a long time like when i was treasurer in 2000 or 2020 for three years the total budget was under 200,000. And that was almost all fellows. So like the total budget for everything, including grants, legal stuff, hosting, I think now maybe it's 250, 300. But it's, you know, to your point, Simon, it's not funded by a corporation. It's, it's tiny. + +00:35:26.760 --> 00:35:30.240 +And people don't realize that aside from the fellows, everything else is volunteers, right? + +00:35:30.480 --> 00:35:39.500 +Even the conferences, it's not, you know, PyCon, US, EuroPython, have a lot of volunteers, but they also have paid employees like Django cons are all volunteers. + +00:35:41.080 --> 00:36:00.640 +Yeah, they're all volunteer based. The PSF runs a staff of about, I used to be on the PSF board, Simon's on the board. So he can speak to this. Last time I saw, they probably have about what, two million in salaries a year because of so many people. You've got like five or six developer in residence, at least probably what, like 12 to 13 people, maybe 14 people on staff. That's about + +00:36:00.580 --> 00:36:16.920 +Right. Yeah. It's interesting how much the PSF is an accounting organization, because it turns out accounting, especially if you're a nonprofit with like with a whole bunch of organizations who use the PSF as their fiscal sponsor to help them with their nonprofit accounting. + +00:36:17.280 --> 00:36:29.800 +There's a lot of there's a lot of accounting and logistical works that goes into an organization like the PSF and PyCon U.S. is run by the PSF as well, which is a 2500 person conference these days. That's that gets very complicated. + +00:36:30.780 --> 00:36:59.780 +Yeah, so the DSF runs on shoestring size budget compared to the PSF. But Jango is a pretty good amount of the community, though. So like, that's kind of one of the things we're doing is like by getting that third fellow. And part of it, too, is like, we've ran not to nerd out on expenses for a while, but the DSF for a long time has ran with over 200,000 in the bank. And to me, that's just not doing much with our money. So I know that we're all, you know, we're also doing investments and stuff as well. But like, I think getting that third fellow was something that was just long overdue for the number of people that we have. + +00:37:00.100 --> 00:37:08.700 +If one fellow goes on vacation for a while and leave, it's just too much for one person to keep the foundation, keep security releases, releases going. + +00:37:09.340 --> 00:37:12.540 +So it was kind of long overdue, but I'm happy we were able to pull it off this year. + +00:37:13.320 --> 00:37:13.780 +That's great. + +00:37:14.280 --> 00:37:20.660 +My favorite talk at JangoCon last year was Jacob Kaplan Moss gave a keynote about what the DSF would do with a million dollars. + +00:37:21.020 --> 00:37:24.640 +Like if the DSF could go up to a million dollars a year, what would they spend that money on? + +00:37:24.820 --> 00:37:28.020 +As also a pitch for why you should be sponsoring the DSF. + +00:37:28.280 --> 00:37:33.880 +I recommend people watch that if you want to get a feel for how that organization was shaped at least sort of 10 months ago. + +00:37:34.820 --> 00:37:47.540 +I think there's a huge impact that just having one or two people whose full-time job is to go into the corner, the dark corners of some open source project that's just not that interesting. + +00:37:47.780 --> 00:37:48.940 +Nobody really wants to go mess with. + +00:37:49.680 --> 00:37:58.060 +But just start hacking away on those and, you know, really putting the polish on stuff that is not as exciting as other pieces. + +00:37:58.280 --> 00:38:05.540 +And so there's not, you know, you said it's not that big relative to the PSF, but put the PSF aside. + +00:38:05.920 --> 00:38:16.580 +Any other open source Python project, it's a really huge pool that you're working with relative to how many other projects even have fellows. + +00:38:19.080 --> 00:38:19.880 +Not many, if any. + +00:38:22.620 --> 00:38:24.140 +Yeah, super, super cool. + +00:38:24.620 --> 00:38:28.420 +So let's talk about some of the more recent things. + +00:38:28.720 --> 00:38:49.680 +I know that, at least personally, when the Instagram keynote came out a couple years ago at PyCon, or maybe more than a couple, 2017, I think, but around that timeframe, just realizing how big some of the deployments of Django were, how much traffic people were serving through them. + +00:38:51.800 --> 00:38:55.960 +When it first came out, you probably didn't think on that scale. + +00:38:56.940 --> 00:38:59.020 +No, Instagram, Pinterest, NASA. + +00:38:59.770 --> 00:39:03.200 +Yeah, no, that was, the goal was to build newspaper websites, right? + +00:39:03.210 --> 00:39:10.480 +We wanted to be able to build, to, I loved, Django's original tagline was web development on journalism deadlines. + +00:39:10.760 --> 00:39:13.260 +That's always been my favorite description of the project. + +00:39:15.060 --> 00:39:15.300 +Nice. + +00:39:17.900 --> 00:39:19.300 +Who else wants to chime in on this idea? + +00:39:19.430 --> 00:39:21.140 +Like, this is, this is just, it blew my mind. + +00:39:21.320 --> 00:39:21.820 +I'm like, wow, that's a big deal. + +00:39:21.820 --> 00:39:27.440 +Oh, one thing I will say is Threads, Instagram's Twitter clone, came out what, last year or the year before? + +00:39:27.680 --> 00:39:28.360 +That's a Django app. + +00:39:28.550 --> 00:39:29.540 +I met one of the engineers. + +00:39:30.140 --> 00:39:35.560 +It's just a classic Django app that Facebook launched just a few years ago. + +00:39:38.339 --> 00:39:40.400 +I mean, I believe technically they have their own fork. + +00:39:40.790 --> 00:39:44.840 +So it's not, you know, they didn't just download Django 4.0, but- + +00:39:44.840 --> 00:39:48.800 +I feel like at that scale, if you haven't got your own fork, I'd be very surprised. + +00:39:48.800 --> 00:39:50.460 +But yeah. + +00:39:50.830 --> 00:39:51.140 +Yeah, for sure. + +00:39:51.160 --> 00:40:43.960 +runs on sender maybe who knows well I mean one of the working thing as far as sorry I think I'm a brief like 500 milliseconds behind all of you no no scale is also the number of projects out there I was been doing the counting lately to figure out our impacts and I think we're at around the 500,000 live Django sites and apps out there from the numbers I could find and also like specific types of outreach so like governments for example seeing how many you know like central governments of all the countries around the world basically use it and when i say use it it's not just like the one project like they have teams whose role is just to manage 10 20 50. i think right now brazil of all countries i don't have a connection there had like 84 sites i could detect in the central government using django like just imagine that + +00:40:43.980 --> 00:40:45.220 +at times all the countries. + +00:40:46.960 --> 00:40:52.820 +You know, one of the things that's always fun for me to do is to go to the GitHub repo, right? + +00:40:52.960 --> 00:40:57.700 +And we've got Django with 85,000 stars and 32,000 forks. + +00:40:58.000 --> 00:40:58.920 +That's a really interesting number. + +00:40:59.200 --> 00:41:03.580 +Way more forks to stars than an average project, by the way. + +00:41:04.100 --> 00:41:15.140 +But if you go down the side here, you can see they now track how many contributors and use by 1.997 million projects. + +00:41:17.380 --> 00:41:18.080 +That is a lot. + +00:41:18.460 --> 00:41:22.820 +Now, that's just on GitHub, right, that are public repositories that we can know about and stuff. + +00:41:23.540 --> 00:41:27.980 +And used by could be a student project, not a government entity. + +00:41:28.540 --> 00:41:29.780 +But still, that's pretty wild. + +00:41:33.020 --> 00:41:33.540 +Very impressive. + +00:41:35.420 --> 00:41:38.400 +Let's talk about some of the more recent transitions. + +00:41:38.800 --> 00:41:44.180 +I mean, I feel like the speed of major version releases is picking up. + +00:41:44.660 --> 00:41:47.820 +You know, we had the first three years, 2005 to 2008 for one. + +00:41:48.360 --> 00:41:52.040 +Then it was like one for a while, and then two, and then three, then four, then five. + +00:41:52.940 --> 00:41:54.220 +You guys are maybe working on six. + +00:41:54.250 --> 00:41:54.620 +I don't know. + +00:41:55.720 --> 00:42:00.260 +And also adopting some of the lot of changes coming in Python, right? + +00:42:00.360 --> 00:42:02.620 +Like async and await and that kind of stuff. + +00:42:03.340 --> 00:42:04.160 +How has that been? + +00:42:08.160 --> 00:42:13.440 +I'll get it started. In some ways, it's really tricky because people really love Django as it is. + +00:42:14.140 --> 00:42:30.920 +And from that standpoint, you have to make sure that such a big group of people agree on what exactly the future of Python means. So, you know, definitely mentioned Async/Await. It's one of the big ones. API, is it in Django Core? Is it not? Mobile web, how much we get involved with that. + +00:42:31.480 --> 00:42:52.320 +These days, it's like the HTMLX that people really want to see Django support well. And I guess also AI to some extent. I don't know if we want to get there. But I guess for us, I think it's also, if it's not in core, we definitely have so many packages out there that we just try and curate a bit more these days, make sure they work well together. So that also helps a lot. + +00:42:53.220 --> 00:43:09.520 +Yeah. I think the package curation is really important. I would actually like to see Python in the standard library smaller and then recommend more stuff to keep the flexibility open so they could move faster, focus more. So what I find tricky about this one, like Django, Django has + +00:43:09.520 --> 00:43:32.520 +a bit always been a batteries included framework. And that's because in 2004, 2005, PyPI and pip didn't really exist. So if you weren't batteries included, then people had to, it was very high friction for people to glue together the pieces that they needed. And of course, today, I feel like Python packaging is incredibly good shape, like you could ship a web framework today that was entirely dependent on other things. + +00:43:32.880 --> 00:43:42.640 +So then the value from the Django thing comes down more to everything being, that there's a Django promise that the stuff in Django core will continue to work. + +00:43:42.960 --> 00:43:52.920 +Like we're not going to, you're not going to have a situation where Python 3.15 comes out and Django templates stop working, but the ORM still works because everything is maintained in the one place by that single team. + +00:43:53.340 --> 00:43:53.920 +And I love that. + +00:43:53.940 --> 00:43:59.020 +I love that you just know that all of the pieces of Django are going to continue to work together. + +00:43:59.360 --> 00:44:35.500 +And the moment you add external packages, there's that little bit of risk. Like what if that external package that you're using becomes unmaintained in the future? So I'd quite like to see just a little bit more of a sort of adoption of crucial packages under the Django software foundation, but not as part of Django itself. I want separate packages that maintain that promise to be maintained because we have fellows, which means that we can actually do that kind of work. But it's also a very big to take on. It's an interesting pattern that I'd like to see explored a little bit more, I think. + +00:44:37.480 --> 00:44:41.640 +Yeah, it's easy to forget that pip and Solved wasn't always there. Sorry, Will, go ahead. + +00:44:42.640 --> 00:45:11.280 +I was just going to ask Adrian and Simon. I mean, even today, batteries included web frameworks are not that common. It's still the trend, more micro frameworks. I'm curious what your take is on that. I mean, like FastAPI, right? That's a lightweight framework. I'm always shocked there aren't more batteries included frameworks in Pika language. Maybe it's too difficult to do, but micro frameworks seem more common. There's only one batteries included option per programming + +00:45:11.440 --> 00:45:38.980 +language. I feel like the sales pitch, like you get to pick the best pieces for the components for your project, always sounds great. Obviously, you don't want one thing that dictates what you're going to use for all of the different things. You'd much rather pick and choose the things that are talk about much is the fact that now every single one of those selections is getting you further away from that sort of community standards. There's more risk of future maintenance problems. + +00:45:39.280 --> 00:46:12.780 +There's more work that you have to do to wire things together. But I feel like the biggest innovation in the open source world in the past 15 years has been packaging. Like every programming language has a PyPI style packaging solution of some sort. You've got cargo and you've got the Ruby gems and all of those kinds of things. And that pattern mostly works really, really well. But I do, I just, I just keep on coming back to that feeling that with Django, I know it's all going to work. And I know it's going to work far into the future. And that, that feels really valuable to me. + +00:46:17.980 --> 00:46:18.380 +Go ahead, Adrian. + +00:46:18.540 --> 00:46:29.400 +For my answer, I think I'd have to evaluate it on a case by case basis, but But the way I would evaluate it is to look at the origin of the project. + +00:46:29.860 --> 00:46:32.240 +Was it extracted from something from the real world? + +00:46:32.260 --> 00:46:37.520 +Or was it built either as an academic or for fun exercise? + +00:46:37.820 --> 00:46:40.760 +Or what was the reason for its existence? + +00:46:41.720 --> 00:46:46.540 +And generally, I trust stuff that was extracted from real production sites more. + +00:46:47.020 --> 00:46:48.780 +That's just my own preference. + +00:46:49.980 --> 00:46:51.900 +And it's also sort of to flip it around. + +00:46:52.760 --> 00:46:56.920 +I never really cared whether someone used Django or not. + +00:46:56.940 --> 00:47:02.320 +I would evaluate you on your product, not based on like the web framework you use. + +00:47:02.320 --> 00:47:04.500 +Who cares really what web framework you use? + +00:47:05.200 --> 00:47:12.320 +And I think that kind of ethos may be seeped into Django at some point unintentionally. + +00:47:12.760 --> 00:47:19.960 +I mean, I never intentionally sort of made that, had that philosophy communicated, but yeah. + +00:47:21.120 --> 00:47:26.680 +I always felt like one of the most interesting differences between Rails and Django is that Rails was extracted from Basecamp. + +00:47:26.760 --> 00:47:28.680 +They built Basecamp and they pulled the framework out of it. + +00:47:28.960 --> 00:47:34.800 +I feel like Django was kind of reverse extracted from Lawrence.com because we had Lawrence.com. + +00:47:34.840 --> 00:47:35.840 +It was PHP and MySQL. + +00:47:36.060 --> 00:47:41.420 +And the goal with Django was always to get it to a point where it was good enough that it could replace the PHP in Lawrence.com. + +00:47:41.880 --> 00:47:47.420 +So everything that we were building was always like, what would Lawrence.com need in this new thing for us to replace it? + +00:47:47.520 --> 00:47:50.820 +And the end result was a framework that you could build Lawrence.com in. + +00:47:50.920 --> 00:47:52.960 +Lawrence.com was the entertainment website for the newspaper. + +00:47:53.600 --> 00:47:57.520 +It's the best local entertainment website for any city that has ever existed. + +00:47:58.300 --> 00:48:00.240 +I will very confidently state that. + +00:48:00.400 --> 00:48:06.840 +I've never lived in a city with a website as good for figuring out what the things to do in a town are. + +00:48:07.050 --> 00:48:15.020 +And that was Adrian's biggest project, Lawrence Journal World, was always Lawrence.com and evolving Django up to the point where it could power that site. + +00:48:16.460 --> 00:48:46.500 +it's what got me here like i saw that site living 200 miles away and went to san francisco and kind of looked at y combinator and was like you know this is only 200 miles like they've got something good going here and that side especially um i think to a django's batteries um django i think 20 years ago django there's a lot of things that you had to do the django way and i think there's been so much concerted effort especially the first 12 to 14 years that so much is pluggable that like If you don't like the batteries, I don't know what you're missing. + +00:48:46.900 --> 00:48:48.300 +Django is still a really good core. + +00:48:49.040 --> 00:48:50.440 +Don't use the admin if you don't want it. + +00:48:51.380 --> 00:49:04.000 +I think most of the problems I see, the foundation and the community, I think there's a lot of lore that comes with Django, and it's lore of maybe it's harder to change certain things on the website than it really needs to be or is. + +00:49:04.440 --> 00:49:23.040 +I think this last year we've definitely pushed back on, if you do a search on the Django docs now and you look for REST, you wouldn't know that Django has the ability to do that into the docs. And so these were just technical challenges of like certain steering council members have written articles and stuff, blog posts saying like, you can do rest with Django. + +00:49:23.460 --> 00:50:10.820 +There's some good third party packages to do it. In the past, we've kind of been like apprehensive to want to recommend third party packages through docs. I think that's just one of those. It sounds weird when I say it out loud, at least it does to me. And I think once we have these discussions, then we start like realizing that, well, we have a website that everybody comes to. There's very specifically doc sections and there's blog, but there's no search that pulls stuff in together. And so the steering council and different members have been trying to make search work across everything. And then that way you can get hits from rest and you can get hits for third party packages that the steering council recommends just because they're good packages everybody should use. And so I think some of the lore and some of the, hey, these things have been done this certain way. It doesn't mean we want to keep doing them. It's just probably somebody had an afternoon or a night and they made search work a particular way. + +00:50:11.170 --> 00:50:12.440 +And that was really good for the docs. + +00:50:13.260 --> 00:50:15.360 +But like sometime look up REST and look up PHP. + +00:50:15.580 --> 00:50:18.840 +There's actually more hits for PHP on the Django website than there are REST. + +00:50:19.240 --> 00:50:22.120 +Unless you get into like stinks, you know, restructured text. + +00:50:22.820 --> 00:50:35.960 +But those are challenges that until we talk about and we realize, and I think like that's the advent of FastAPI kind of taking off, is I think like Django didn't have a great REST story by looking at the docs and looking at information. + +00:50:36.680 --> 00:50:41.320 +At PyCon US this year, I had dozens of people ask me, how do I use the Django ORM from FastAPI? + +00:50:41.400 --> 00:50:42.620 +And it's like, why? + +00:50:42.820 --> 00:50:43.440 +You don't have to. + +00:50:43.600 --> 00:50:46.940 +Like, Django is async enough that you don't need to do that. + +00:50:47.080 --> 00:50:47.540 +That's hard. + +00:50:47.680 --> 00:50:48.800 +And that kind of sucks to do. + +00:50:48.940 --> 00:50:50.280 +So, like, why? + +00:50:51.220 --> 00:50:53.920 +And those are things we're focusing and talking about internally, too. + +00:50:55.120 --> 00:50:55.520 +Yeah. + +00:50:56.260 --> 00:51:04.680 +You know, talking about search and search across different things, somewhat direct to you, Jeff, but all of you and maybe Simon as well. + +00:51:04.840 --> 00:51:08.500 +Like, what about just like chat.djangoeproject.com? + +00:51:09.500 --> 00:51:15.920 +And, you know, do some rag on it and let people have a conversation with Django. + +00:51:16.260 --> 00:51:16.640 +You know what? + +00:51:16.860 --> 00:51:23.660 +We don't even need it because Django is stable enough that every chatbot out there knows Django inside out already. + +00:51:23.920 --> 00:51:24.440 +It does, yeah. + +00:51:24.760 --> 00:51:34.120 +One of the great benefits of having great documentation and lots of examples for 20 years that is stable is, yeah, you can, GPT-5 will spit out a full Django project and get it all right. + +00:51:36.780 --> 00:51:56.040 +I completely disagree. I think we need a big sponsor who can come in, whether it's Anthropic or OpenAI or somebody else. We need a big sponsor, a title sponsor even, in fact, that can help give the money for this and donate the chat. Because even though you're right, that it is ingrained, you still have that delay because Django kicks out a new version or two a year. + +00:51:56.680 --> 00:51:59.000 +And so there's always that lag of nine months or a year. + +00:51:59.360 --> 00:52:00.420 +And so-- + +00:52:00.420 --> 00:52:04.560 +- Or recommends the most popular version, not the latest version or something like that, right? + +00:52:05.360 --> 00:52:06.000 +- Exactly, yeah. + +00:52:06.220 --> 00:52:10.580 +Because I see a lot of demos where they're actually building Django apps using these tools. + +00:52:10.740 --> 00:52:14.960 +And so we're selling AI by virtue of Django and Django's docs being so good, so. + +00:52:15.300 --> 00:52:21.260 +- Or occasionally one of these things we'll say that Sound Slice has a feature that doesn't have, and then Adrian has to build the feature. + +00:52:21.840 --> 00:52:22.320 +- Oh, God. + +00:52:23.300 --> 00:52:25.920 +Oh, you guys, in the chat, it's coming on here. + +00:52:26.060 --> 00:52:33.800 +We have Frank Walsh saying, but our chatbot could have a cool personality and Kenneth loves to answer a random nay every once in a while or something. + +00:52:35.300 --> 00:52:36.500 +You know, pony style. + +00:52:38.020 --> 00:52:40.600 +Adrian, what is the story behind that? + +00:52:41.200 --> 00:52:42.420 +Oh, it has nothing to do with Django. + +00:52:44.160 --> 00:52:55.160 +Yeah, just the 30-second version is ChatGPT was telling people about my site, my company, SoundSlice, having a feature that doesn't actually exist. + +00:52:55.860 --> 00:52:57.680 +And it was happening like dozens of times per day. + +00:52:59.240 --> 00:53:04.180 +And so we had this inbound set of users who had a wrong expectation. + +00:53:05.420 --> 00:53:21.400 +So we ended up just writing the feature to appease the ChatGPT gods, which I think is the first time that, at least to my knowledge of, yeah, product decisions being influenced by misinformation from LLMs. + +00:53:21.620 --> 00:53:25.960 +- I just love that it's easier to just build the feature than to get the misinformation corrected. + +00:53:27.060 --> 00:53:28.180 +- You just gotta backfill it. + +00:53:28.200 --> 00:53:29.120 +You just gotta backfill it. + +00:53:32.060 --> 00:53:32.480 +Incredible. + +00:53:33.700 --> 00:53:34.020 +All right. + +00:53:35.920 --> 00:53:36.460 +That's crazy. + +00:53:36.940 --> 00:53:39.960 +Let's do two more quick topics, somewhat lightning round. + +00:53:40.070 --> 00:53:44.480 +We have about six minutes left for this show. + +00:53:44.780 --> 00:53:56.420 +So one, there's been, you guys mentioned it a couple of times and it's also just kind of going back and forth in the audience here, that the whole REST idea. + +00:53:56.430 --> 00:54:00.200 +So we've got Tom Christie doing amazing work with Django REST framework. + +00:54:00.740 --> 00:54:12.040 +And we've also, I think, people who do talk about FastAPI and how do I use the ORM with it, like we've got Django Ninja, which is really, really close to FastAPI and so on. + +00:54:13.520 --> 00:54:17.580 +That's one side, that's a really positive side of things with Django. + +00:54:17.750 --> 00:54:33.120 +And the other side I think is, I'm not a super fan of, but it's something a lot of people do is, well, can we just make the backend a dumb crud API thing and let me just write everything in JavaScript on the front end, which I'm not as much of a fan of. + +00:54:33.240 --> 00:54:36.660 +I think a lot of people choose that because they've been told that's how it's supposed to be. + +00:54:36.670 --> 00:54:37.860 +I don't think it needs to be that way. + +00:54:39.479 --> 00:54:44.940 +So what do you all feel about that and sort of how this has an impact for Django? + +00:54:46.860 --> 00:54:53.320 +Because if people just see the backend as an API that's just a dumb thing, And a lot of the value that Django brings, I think, starts to vanish. + +00:54:54.700 --> 00:54:56.160 +You've got things like HTMX, so on. + +00:54:56.160 --> 00:54:57.380 +So, yeah, what are your thoughts on all this? + +00:54:57.800 --> 00:55:09.840 +I think one of the hardest problems in computer science right now is talking a front-end team out of building everything as a pure JavaScript app, especially if that's developers who onboarded in web development over the past five to ten years. + +00:55:11.560 --> 00:55:16.200 +It's infuriating because my impression is that it just takes longer + +00:55:16.240 --> 00:55:18.920 +and you get more bugs and it's harder to build useful software. + +00:55:19.300 --> 00:55:23.880 +The one thing I will say is that HTMX has been a huge breath of fresh air on this front. + +00:55:23.970 --> 00:55:28.280 +Like HTMX gives you your single page applications. + +00:55:28.780 --> 00:55:31.660 +It's all like stuff that's rendered server side in templates. + +00:55:31.850 --> 00:55:33.200 +It fits really well with Django. + +00:55:33.400 --> 00:55:36.480 +There's a great Django extension for looking into HTMX. + +00:55:36.730 --> 00:55:46.140 +And I feel like that the pendulum is just sort of swinging slightly back away from the everything must be rendered in JavaScript thing, partly because of the work that the HTMX people have been doing. + +00:55:46.300 --> 00:55:48.100 +So that I felt pretty good about. + +00:55:48.480 --> 00:55:57.180 +I'm going to point to Adrian now because SoundSlice is a pure JavaScript frontend app, and it should be based on what it does, right? + +00:55:58.180 --> 00:55:59.780 +Yeah, it's interactive sheet music. + +00:55:59.930 --> 00:56:06.840 +So it's sheet music that's rendered entirely in your browser, responsive, blah, blah, blah, with an editor, a lot of crazy JavaScript. + +00:56:07.500 --> 00:56:14.640 +But yeah, I mean, this is a whole other podcast episode about the horribleness of modern web development. + +00:56:15.420 --> 00:56:21.300 +I feel very lucky that I came up 20 years ago before it got so insane. + +00:56:23.339 --> 00:56:50.320 +But yeah, I have sort of a related comment, which is I would like to see Django be a stronger advocate for the web in general because it doesn't matter if you're talking to developers or people using tech products, the expectation has been shifting away from the web to the phone for many, many reasons. + +00:56:51.600 --> 00:56:53.240 +And I don't like that. + +00:56:53.540 --> 00:56:57.520 +I'm pretty sure that most, if not all, of the people on this chat don't like that either. + +00:56:57.800 --> 00:57:02.880 +And I don't really see a lot of effort being pushed in that direction. + +00:57:04.480 --> 00:57:08.940 +So just to sprinkle in a little idea here. + +00:57:09.080 --> 00:57:09.200 +Yeah. + +00:57:09.740 --> 00:57:19.140 +I personally wish that progressive web apps were more front and center, giving you an option at least, like, okay, you have to be on the phone, but could we package it up a little more? + +00:57:19.540 --> 00:57:21.440 +- There's a really exciting development there, + +00:57:21.440 --> 00:57:23.380 +just that's finally beginning to be rolled out. + +00:57:23.500 --> 00:57:36.460 +There's this thing called view transitions in CSS, where you can have a little bit of CSS that says, and when the user goes from one page to another, keep the navigation bar in the same place and do like a quick blur effect for this piece here. + +00:57:36.700 --> 00:57:40.620 +It means that, and it's been supported in Chrome for a few years, it's finally rolling out to the browsers. + +00:57:41.040 --> 00:57:45.520 +It means you can build something that feels like a slick app with page transitions and so forth. + +00:57:45.760 --> 00:57:46.240 +But it's not. + +00:57:46.380 --> 00:57:48.040 +It's just boring old HTML and CSS. + +00:57:48.520 --> 00:57:49.940 +No JavaScript required at all. + +00:57:50.320 --> 00:57:52.220 +Gives you that experience much faster to load. + +00:57:52.520 --> 00:57:55.820 +And I have been itching to see this start showing up. + +00:57:55.820 --> 00:58:01.780 +And I think it's finally this year, I feel, is the year where the browser support is good enough that we can start really depending on this. + +00:58:02.440 --> 00:58:09.140 +And, yeah, because it gives you that app-like experience on a boring old website, which is, I feel like that's the thing that we all want. + +00:58:10.500 --> 00:58:15.280 +So part of Adrian's piece and sorry, Tebow, real quick. + +00:58:15.770 --> 00:58:18.240 +My second thing I have to pitch is because of what Adrian was saying. + +00:58:18.770 --> 00:58:25.940 +I think because systematically we, you know, we are really lucky to be around for 20 years because we've had nothing but volunteer leadership for 20 years. + +00:58:26.700 --> 00:58:28.960 +One thing we want to do is find an executive director, + +00:58:29.290 --> 00:58:30.280 +somebody that can run the foundation. + +00:58:30.870 --> 00:58:33.000 +The biggest reason goes back to Adrian's point. + +00:58:33.240 --> 00:58:42.360 +I think that we sometimes don't get a seat at the table when it comes to web standards, when it comes to these conversations, is because it's the cycling of Thibault being the president now. + +00:58:42.360 --> 00:58:43.160 +It was Frank before. + +00:58:43.320 --> 00:58:44.440 +It was Anna from Africa. + +00:58:44.700 --> 00:58:47.340 +We've had many really good presidents over the years. + +00:58:47.940 --> 00:58:52.260 +But it's really hard to work with foundations and nonprofits when you don't have a set person. + +00:58:52.720 --> 00:58:55.180 +Even our fellows seem to rotate every three to four years. + +00:58:55.700 --> 00:58:59.240 +And so that's part of like Django growing up, which is something I would love to see us do. + +00:58:59.740 --> 00:59:02.520 +And then Thibault, I love the fact that this started off in the middle of like the day. + +00:59:02.740 --> 00:59:06.700 +And now it is like perfectly cycled to like dark and barely able to see you. + +00:59:06.760 --> 00:59:07.660 +But go ahead. + +00:59:08.960 --> 00:59:38.460 +Yeah, I guess it was Chaim Kirby, the last president. To me, there's a developer experience and keeping devs happy. So when we say people go to React or whichever JS Heavy, it's also because they think they need to build whatever complexity that they are after, but also because they think it has good DX, I assume. So when we say the REST framework or the Django Ninja, to me, it's this as well, that we have to be aware of what's the evolving standards for what devs want to do with their web. + +00:59:39.060 --> 00:59:48.100 +And I guess how we can make Django meet that, whether it's better types, better CLI, better packaging, more Rust, maybe, whichever. + +00:59:51.240 --> 00:59:56.520 +Will, do you want to get a quick word in on this everything's a front-end thing now + +00:59:56.800 --> 00:59:58.240 +before we move on to the last bit? + +00:59:59.260 --> 00:59:59.600 +Oh, sure. + +00:59:59.720 --> 01:00:12.700 +I'm just going to say, and this probably goes back to Adrian and Simon, like all of them, It seems very wise to keep the front end light because there was certainly 10 years ago, a lot of pressure to integrate React or Vue or Angular or something. + +01:00:12.860 --> 01:00:19.920 +But the fact that it's remained just the Django templates means HTMX can rise and we can have more innovation. + +01:00:20.540 --> 01:00:25.900 +And just shout out, Carson Gross of HTMX is giving the keynote at DjangoCon US in three weeks. + +01:00:26.840 --> 01:00:32.740 +So even though he's not a Django developer, he's in fact more of a he's a professor who does Java and obviously JavaScript. + +01:00:33.340 --> 01:00:39.400 +it still slides into Django in a way that if Django had built out the front end story, we wouldn't be able to. + +01:00:40.340 --> 01:00:41.800 +I love how that also fits. + +01:00:42.040 --> 01:00:51.080 +DjangoCon has had a long tradition of having a keynote from somebody from outside the community, which started in 2008 with Cal Henderson at DjangoCon over in Mountain View. + +01:00:52.220 --> 01:00:53.440 +Was that the why I hate Django? + +01:00:53.620 --> 01:00:57.100 +Because there was that whole history of, I feel like there should be another one of those, you know? + +01:00:57.560 --> 01:00:58.420 +Absolutely, yeah. + +01:00:59.280 --> 01:00:59.900 +But that's great. + +01:01:00.060 --> 01:01:02.360 +That's a fantastic keynote for DjangoCon, I think. + +01:01:03.620 --> 01:01:28.180 +yeah and coming up in Chicago right pretty soon so if you are in or would like to be go check it out right that's part of that community thing I talked about and people like people not listening who are not watching I just you know listening not watching they don't realize that you you did this outside and uh there's a great comment from Kojo saying it's really cool that You're using in real life dark mode. + +01:01:28.800 --> 01:01:29.100 +That's great. + +01:01:32.180 --> 01:01:32.400 +All right. + +01:01:32.700 --> 01:01:34.040 +Let's close it out with this, everyone. + +01:01:34.500 --> 01:01:35.380 +We're pretty much out of time. + +01:01:36.760 --> 01:01:37.360 +20 years. + +01:01:39.000 --> 01:01:40.220 +Amazing, amazing stuff. + +01:01:43.220 --> 01:01:46.260 +We're celebrating Django having its birthday, right? + +01:01:46.500 --> 01:01:49.320 +But it's going to live on for a long time. + +01:01:49.420 --> 01:01:50.680 +There's a ton of code. + +01:01:50.840 --> 01:01:53.320 +There's a ton of things people are doing with Django. + +01:01:55.320 --> 01:01:57.460 +Even if you guys step away at some point, it's still going to be going. + +01:01:57.900 --> 01:02:01.340 +So give me your, you know, going around and starting with Will, I guess. + +01:02:01.560 --> 01:02:05.320 +Give me your forward-looking thoughts on Django over the next five, ten years. + +01:02:08.720 --> 01:02:12.580 +Well, I think it's integrating with AI, broadly defined, however that means. + +01:02:13.040 --> 01:02:17.840 +Because in a way related to Adrian's point, like the boring old web has never been more relevant. + +01:02:18.140 --> 01:02:22.320 +Even though we're focused on chatbots right now, you still need all the other things. + +01:02:23.060 --> 01:02:24.520 +And Django can integrate into that. + +01:02:24.720 --> 01:02:31.880 +And I think it's always kind of a marketing problem for Django because we don't have one person in charge or a company behind it. + +01:02:32.020 --> 01:02:36.760 +So telling the story of we've been here, there's still double digit PRs. + +01:02:36.940 --> 01:02:38.580 +It has a sink in many areas. + +01:02:39.780 --> 01:02:45.520 +And I worry that the next generation of web developers just skip it because they don't know what it already has. + +01:02:45.880 --> 01:02:50.400 +And they have this presumption that it's too hard to learn, which it's not. + +01:02:51.660 --> 01:02:52.540 +Yeah, absolutely. + +01:02:55.780 --> 01:02:56.520 +Is it me? + +01:02:56.980 --> 01:02:57.600 +It is you. + +01:02:57.970 --> 01:03:00.100 +I hear you're boring about the web. + +01:03:00.280 --> 01:03:00.960 +What's going on now? + +01:03:01.350 --> 01:03:01.800 +Tell us about it. + +01:03:01.910 --> 01:03:03.140 +What do you see for the future? + +01:03:03.780 --> 01:03:04.340 +For the future? + +01:03:04.820 --> 01:03:14.420 +Yeah, I'm actually, perhaps ironically, the worst person to ask because I have worked on the same app for almost 13 straight years full-time now. + +01:03:16.580 --> 01:03:20.740 +I use Django every day, and it does everything I need. + +01:03:21.280 --> 01:03:24.440 +So I'm very happy with the current state. + +01:03:25.859 --> 01:03:45.900 +And I think, like I said earlier, if it were up to me, the thing I would focus on is more philosophical or trying to promote the concept of the web versus non-web stuff. + +01:03:46.780 --> 01:03:49.060 +Because if the web goes away, Django goes away. + +01:03:49.200 --> 01:03:51.120 +I mean, it's in here. + +01:03:51.160 --> 01:03:52.480 +- A lot of beautiful things go away, yeah. + +01:03:52.660 --> 01:03:53.880 +- A lot of beautiful things go away. + +01:03:54.980 --> 01:03:59.500 +So I think that would be my hope for the next five years. + +01:04:00.060 --> 01:04:06.440 +- Yeah, I've done combat with the App Store review people, both Apple and Google, and it is not pretty. + +01:04:06.600 --> 01:04:07.920 +We do not want the web to go away. + +01:04:08.800 --> 01:04:09.080 +Theobald. + +01:04:12.260 --> 01:04:15.400 +- There's definitely AI in there and definitely the web. + +01:04:15.640 --> 01:04:17.860 +I really liked that you mentioned the view transitions. + +01:04:18.680 --> 01:04:36.260 +I think if web technologies get that much better in the next five years or so, we can match what's possible with a mobile dev, but not only match it, but in terms of accessibility and how many people you can reach with a web app, I would go far, far beyond. + +01:04:36.720 --> 01:04:53.120 +So I guess to me for that, the other side is having a foundation behind Django that's probably a bit more professional, that takes it a bit more seriously, that with people looking after the framework for millions of devs at this point. + +01:04:54.060 --> 01:04:58.040 +And that, as you said, the people in this room, we come and go, it's very natural. + +01:04:58.420 --> 01:05:00.160 +So just making sure that we have the pipelines. + +01:05:00.940 --> 01:05:10.120 +So hopefully, five, 10 more years, the jungle of space you mentioned earlier, that's the people who will be the presidents and people having this podcast maybe in 10 years' time. + +01:05:11.680 --> 01:05:12.560 +- Yeah, absolutely. + +01:05:13.680 --> 01:05:13.800 +Jeff. + +01:05:15.880 --> 01:06:11.400 +really curious to see what russ's impact is i think python is just getting so much faster so much better and i know there's some really smart people working on like WebAssembly and rust and i dream of a day when a django view can be the same thing in your browser as it is on the server and i think when we look at like talking to a query set your browser you know doing data exchange serialization json something i feel like in the next five years there's this there's this opportunity that whatever python is becoming whatever django is becoming that like you know we it's less about how are we going to use react and render which i hate react by the way you got to put that plug in but i'm just really curious to see like how these things can kind of mold together as you know developer experiences have gotten a lot better because of tooling and rust uh there's just this whole new world that i see as far as like these technologies that start to kind of you know marry the two together in a way that we just haven't been able to do for a while but we're starting to see glimpses of it. + +01:06:13.299 --> 01:06:15.240 +For sure. Simon, final word for... + +01:06:15.880 --> 01:06:19.320 +I'm going to double down a little bit on the WebAssembly stuff is so exciting. + +01:06:19.490 --> 01:06:25.940 +Like Pyodide, the thing that lets you run Python in your browser, is now at least 10 years old. It's very, very good. + +01:06:26.420 --> 01:06:27.740 +It's about a two megabyte download. + +01:06:27.950 --> 01:06:30.340 +A few years ago, I'd have said two megabyte downloads is too much. + +01:06:30.430 --> 01:06:33.600 +These days, two megabyte download for a web page is hardly anything at all. + +01:06:34.320 --> 01:06:49.180 +But I feel like the thing that I love most about Django these days is It is reliable. It's so dependable. Nobody ever picks Django for a project and regrets it. Because Django, it's mature, it's stable, it's well documented, it moves at a sensible place. + +01:06:49.800 --> 01:07:35.980 +Long may that continue. I love the idea that Django becomes -- and Django for many years has been a great default technology for you to use as a project. And if we double down on that, which I think is what we're doing already, that's fantastic for me. I love that idea that you can always pick Django, it will always do what you need it to do. There will never be any nasty surprises or regrets from working with that. And if we can tap into things like free-threaded Python is coming along now, which means maybe we don't have to rewrite everything as async things just to handle large concurrency. There's all of that kind of stuff, the WebAssembly stuff. It feels like the Python ecosystem is in a really good shape and Django sits very close to the center of that ecosystem. And I'm really excited about that. I love that we can continue to provide the best + +01:07:36.260 --> 01:07:46.840 +default option for building great things. Yeah, totally agree. All right, everyone, thank you so much for being here. Congratulations. Happy birthday, Django. + +01:07:48.480 --> 01:07:51.280 +Happy birthday, Django. Happy birthday, Django. + From caa59e06d0af5ed47be652c2c5d6d39f18f1b04d Mon Sep 17 00:00:00 2001 From: Michael Kennedy Date: Tue, 19 Aug 2025 14:37:27 -0700 Subject: [PATCH 3/5] transcripts --- ...lerating-python-data-science-at-nvidia.txt | 1064 +++++++++++ ...lerating-python-data-science-at-nvidia.vtt | 1610 +++++++++++++++++ 2 files changed, 2674 insertions(+) create mode 100644 transcripts/516-accelerating-python-data-science-at-nvidia.txt create mode 100644 transcripts/516-accelerating-python-data-science-at-nvidia.vtt diff --git a/transcripts/516-accelerating-python-data-science-at-nvidia.txt b/transcripts/516-accelerating-python-data-science-at-nvidia.txt new file mode 100644 index 0000000..96f6092 --- /dev/null +++ b/transcripts/516-accelerating-python-data-science-at-nvidia.txt @@ -0,0 +1,1064 @@ +00:00:00 Python's data stack is getting a serious GPU turbo boost. + +00:00:03 In this episode, Ben Zaitlin from NVIDIA joins us to unpack Rapids, the open source toolkit that lets Pandas, scikit-learn, Spark, Polars, and even NetworkX execute on GPUs. + +00:00:15 We trace the project's origins and why NVIDIA built it in the open, then dig into the pieces that matter in practice. + +00:00:22 QDF for data frames, QML for machine learning, QGraph for graphs, QXFilter for dashboards, and friends like Qspatial and Qsignal. + +00:00:31 We talk real speedups, how the Pandas accelerator works without a rewrite, and what becomes possible when jobs that used to take hours finish in minutes. + +00:00:40 You'll hear strategies for datasets bigger than GPU memory, scaling out with Dask Array, Spark acceleration, and the growing role of vector search with QVS for AI workloads. + +00:00:51 If you know the CPU tools, this is your on-ramp to the same APIs at GPU speed. + +00:00:56 This is Talk Python To Me, episode 516, recorded July 1st, 2025. + +00:01:16 Welcome to Talk Python To Me, a weekly podcast on Python. + +00:01:20 This is your host, Michael Kennedy. + +00:01:22 Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is sponsored by Posit and Posit Workbench. + +00:01:51 Posit Workbench allows data scientists to code in Python within their preferred environment without any additional strain on IT. + +00:01:59 It gives data scientists access to all the development environments they love, including Jupyter Notebooks, JupyterLab, Positron, and VS Code, and helps ensure reproducibility and consistency. + +00:02:09 If you work on a data science team where consistency matters, check out Posit Workbench. + +00:02:13 Visit talkpython.fm/workbench for details. + +00:02:17 Ben, welcome to Talk Python To Me. + +00:02:19 So excited to be talking data science with you. + +00:02:20 Yeah, thanks so much for having me. I'm really excited to talk to you as well. + +00:02:24 Yes, data science, hardware acceleration, graphics, but not really. So it should be super fun. + +00:02:31 Yeah, let's dive into it. Before we really get into using GPUs for data science, which I think is going to be super interesting, just tell us a bit about yourself. Who's Ben? + +00:02:44 Yeah, my name is Ben Zaitlin. I am a system software manager, I think is my title, at NVIDIA. + +00:02:50 I've been working in the Python ecosystem since I left graduate school in 2006, 2005. + +00:02:58 It's actually, unlike other people, this is only my second job. + +00:03:02 I moved from graduate school to working at Continuum or Anaconda, and then I came to NVIDIA. + +00:03:09 I've always been in the space of doing some kind of science with computers. + +00:03:14 Yeah, Anaconda or Continuum at the time, as it was known, been renamed. + +00:03:20 what a launchpad for this kind of stuff, right? + +00:03:23 Yeah, it's been crazy. + +00:03:25 I mean, I feel like I'm a little bit older now than obviously I was when I first joined things, but it's nice to be able to reflect and look back at how much was built over the last decade and a half or so. + +00:03:36 It really is. + +00:03:37 People use Conda. + +00:03:39 People use Conda. + +00:03:40 A bunch of things in the hip were fixed. + +00:03:42 More things need to be fixed. + +00:03:44 More things are being built. + +00:03:45 It's really great. + +00:03:46 Yeah, and not to go too far afield, But Anaconda Inc. is doing interesting stuff to push on different boundaries of Python. + +00:03:54 In addition to the data science and ML side, you know, they're funding a lot of work on PyScript, which I think is really important for the Python in the browser. + +00:04:04 Yeah, I think one, like many conversations I've had when I was at Anaconda about like how deploying Python can be so challenging. + +00:04:11 It wouldn't be great if we had this deployment vehicle like JavaScript, if everything just ran. + +00:04:16 You didn't need anything set up. + +00:04:17 It's just all on the browser. + +00:04:19 And seeing those ideas actually mature into an existing product is really exciting. + +00:04:23 I think there's some people that are really even pushing quite hard to see how do you connect WebGL to getting other parts of your local machine, like obviously the GPU in this case, accessible through the browser. + +00:04:35 But it's really exciting to see that work. + +00:04:37 All right. + +00:04:38 Now you're just blowing my mind. + +00:04:39 I didn't connect the WebGL, the OpenGL of the browser to GPU acceleration, but of course. + +00:04:46 You can see a few people in PyScript, like if you go to the issue tracker, like, oh, I want to, how do I use PyTorch or how do I use TensorFlow inside of this, inside of my browser? + +00:04:54 Occasionally you'll see some people will talk about it later. + +00:04:56 Like, how do I use Rapids or other things like that? + +00:04:58 But once you open the doors, everybody just wants everything there. + +00:05:02 It's like, oh, yes, I understand. + +00:05:04 I see. + +00:05:04 Okay. + +00:05:05 Yeah. + +00:05:05 Yeah. + +00:05:05 And, you know, I said multiple areas, not just PyScript, but also they're doing a lot of work to support Russell Keith McGee. + +00:05:13 And it's Malcolm. + +00:05:15 I'm sorry if it's not Malcolm, the guy working with him on that, to bring Python to mobile as well. + +00:05:21 So those are really important initiatives. + +00:05:23 Yeah. + +00:05:24 Python is not just a niche language. + +00:05:28 It's found itself in every bit of computing up and down the stack from mobile to workstation, HPC, everywhere. + +00:05:36 So I want to start this conversation and jumping into Rapids with a few comments. + +00:05:42 First of all, I recently did an episode on just GPU programming. + +00:05:50 So that was really fun. + +00:05:52 And a quick, where was it? + +00:05:55 Oh, yeah. + +00:05:56 That was with Bryce Aldeis-Lolbach. + +00:06:02 Sorry, Bryce. + +00:06:02 I didn't remember the whole name. + +00:06:04 I'm like, I'm going to screw this up. + +00:06:05 I got to look it up. + +00:06:06 So we talked a little bit about GPUs and stuff, and not so much about Rapids and the side of things. + +00:06:12 that you're working on, although we definitely did touch on it a little bit. So that's another resource for people that really want to go deep into this area. But the thing that I want to go and actually talk about is I want to introduce this with a story from when I was in college in the 90s and a question that really surprised me. I was working doing this like applied math project using Silicon Graphics mainframes in complex analysis, if people know what that is. + +00:06:41 And I was programming with OpenGL to do some visualizations and stuff. + +00:06:46 And people, I was trying to get some help and someone's like, Hey, Hey, are you using the GPU for the math calculations? + +00:06:53 Because I want to hear about that. + +00:06:54 I'm like, I don't even, that doesn't even make sense to me. + +00:06:56 Like, why would you even ask me that? + +00:06:57 GPUs are for pictures and graphics. + +00:07:00 I like write loops and stuff for the math part. + +00:07:02 Right. + +00:07:03 But we've come so far and now GPUs really are a huge part of computation. + +00:07:09 Right. + +00:07:09 And Bryce said, hey, I've worked at NVIDIA for a long, long time, and I know nothing about graphics and 3D programming. + +00:07:16 Yeah, everybody's journey into this has been somewhat rejecting some popular notions about what GPUs are and are not for and to really testing those ideas. + +00:07:26 I think many of us, even still right now, think of GPUs as mostly being for drawing triangles or just linear algebra. + +00:07:34 And those are best in class of what GPUs are for, but it turns out that they're actually not terrible for doing string processing, or they're not as fast as they are for doing dense linear algebra, but they're really still quite good computing platforms for doing a lot of bulk processing around all kinds of data that are not typically what you think of for GPU processing. + +00:07:57 But like in data science, so much of what we're seeing now is actually still string based. + +00:08:02 So how do we get, we don't, we can't just, we can't always just isolate parts of the code to be just for strings or just for compute. + +00:08:09 We have to do all of it together and exploring how we can do that all on the device has been really quite, quite revelatory that it's actually pushing us to tell maybe how to inform how hardware maybe should be built or where we can actually do some software tricks to make some of these non-standard processing still quite performant. + +00:08:28 I'm sorry, I think I cut you off a little bit. + +00:08:30 Oh, no, no, it's great. + +00:08:32 If people haven't physically handled some of these GPUs lately, they might not appreciate just how intense they are, right? + +00:08:40 Like, you know, you think, oh, I've got a little laptop and it's got a GPU in there and it must do a thing. + +00:08:44 But like the desktop high-end ones, like I couldn't put a higher GeForce card into my gaming computer because the power supply was only something like 800 watts or something insane. + +00:08:57 Can I even plug that into the wall without melting it, even if I got a bigger power supply? + +00:09:02 These things are crazy. + +00:09:03 And that's not even touching on the H100, H200 type of server things, right? + +00:09:08 Which is just next level. + +00:09:10 Yeah, there's a lot of power consumption for these accelerators. + +00:09:14 I think what she touched on as well is something that I've seen change over the last 10 years that we thought about GPUs as mostly being for just graphics as these niche computing devices. And they're still not exactly, at least in my mind, not exactly commodity hardware, but they're a lot more commonplace where it's not so revolutionary to think of, oh, what else can I do with the hardware that's in my laptop or in my workstation? And GPUs are definitely part of that narrative. We're becoming very common to think of what are other things that I can do with this just sitting around not doing something. + +00:09:50 Yeah, absolutely. + +00:09:51 And I think just coincidentally, or just the way it works out, data science type of work, and really most significantly, the data science libraries, the way that they're built and the way they execute, line up perfectly with the way GPUs do their work. + +00:10:07 And what I'm thinking of is pandas, pollers, all the vector type of stuff. + +00:10:12 So instead of saying, I'm going to loop over and do one thing at a time, you just say, here's a million rows apply this operation to all million and then either update it in place or give me a new data frame or whatever and that is perfect for like let me load that into a gpu and turn it loose in parallel on all these pieces because as a programmer a data scientist i don't i don't write the imperative bits of it right i just let it go and it's easy for things like rapids to grab that and parallelize + +00:10:43 without me having to know about parallelism, right? Yeah, I think the same is also true in the CPU world as well. I don't know a lot about BLAS or LEPAC or these linear algebra libraries that have been in existence for 30 or 40 years. Those have been tuned to the gills to work on CPU, but I'm just the inheritor of all of that academic research. I don't need to know about caching algorithms or tiling algorithms. I just write my numpy or my pandas or my pullers, and generally, I'm pretty happy. Year after year after year, things generally get better for me without having to go very deep into computer science or even computer hardware design. I can still focus on boring business things or exciting business things or whatever particular vertical I'm in, whether it's genomics or selling ads or whatever it may be. + +00:11:33 There's a lot of layers to that, right? We've got the layers of like, you're talking about the libraries or like the CPU operations, but just the pip install or conda install type of layers that you can add on. They honestly don't need to know too much about what even they're doing, right? + +00:11:48 Yeah. I think that the demands of the user have definitely gone up and really pushed, I think all of the library developers to meet those demands. But when I, when I was first starting in computing, you know, you'd read a two or three pages worth of, you know, change this little bit of XML, compile this thing, do something else. Now the expectation really is like single button, if not single button data science, single button deployment of whatever it is I'm trying to + +00:12:13 work on. Yeah. Yeah. I want to write six lines of code in a notebook and have it do stuff that was + +00:12:19 previously impossible, basically. Right. Or I just want to express like math and then have it work. + +00:12:24 Or I mean, even now in some AI systems, I just want to express text and some agent takes care of everything for me. + +00:12:32 Yeah, it's absolutely. + +00:12:33 The AI stuff is absolutely crazy. + +00:12:34 And we'll come back to some of the integrations with like vector embeddings and vector search and all that kind of stuff at the end. + +00:12:43 But for now, let's maybe, I want to talk a bit about open source at NVIDIA and why you all decided to open source Rapids. + +00:12:54 And then we could maybe talk about how it came to be as well. + +00:12:57 So people probably think of NVIDIA mostly, Obviously, there's a lot of audience bias by listening to this show. + +00:13:04 But a lot of people think of NVIDIA as the gaming company or maybe just the GPU company. + +00:13:10 What's the software story and the open source story there? + +00:13:14 I think software is very important to NVIDIA. + +00:13:18 Obviously, CUDA is one of its primary vehicles to interact with the GPU. + +00:13:24 But NVIDIA has been exploring software in a more concentrated way over the last, I don't know, five, six years, at least since I've been there, probably it predates. + +00:13:34 I can't speak for all of NVIDIA in this way. + +00:13:36 But software becomes quite critical that if you want to deliver a full platform to people and have them use it, you need the software to be as good, if not better, than just the hardware. + +00:13:49 Everything, probably everything needs to work. + +00:13:51 I don't need to disparage any group or elevate any one group over another. + +00:13:56 And Rapids kicks off probably in late 2018, but it predates my time there. + +00:14:04 With a thesis of, well, we see a lot of, there's a lot of signal out in the world of whatever it is that I'm doing right now, how do I make it go 10x faster? + +00:14:13 And I think that's a very natural response to any timeframe, whether it's the 60s or 70s and having them search for whatever the hardware was doing back then, faster cores or multi, I don't know, as they approached, I think the first multi-core thing comes out in the early 80s. But we have this desire to do whatever it is that's happening right now, it can always be faster. There's actually this really great Grace Hopper quote that I like, where she's reflecting on some things from the 70s, that not only is data going to increase, but the demand for access to that data is going to increase. And I've heard a lot about data, beta size is increasing, but it was the first time I really saw somebody even back then saying like, oh, the demand for access was going to increase. So it's really, it's like innate for us to just always go faster. And then Rapid's approach to this problem was, well, these libraries have really become like canon for how you do data science in the Python world. The NumPy, Pandas, NetworkX, Matplotlib become the underlying pillars for this huge explosion of Python and PyData libraries. + +00:15:23 And we want to join that effort. + +00:15:25 How do we take a bunch of knowledge around writing very fast GPU kernels and bring it to this very large community? + +00:15:33 And there's a whole bunch of strategies that we try to employ to make it attractive, to make it possible, and to actually deliver something that ultimately will benefit what + +00:15:43 today versus what didn't happen yesterday. Yeah. You know, you're talking about the Grace Hopper quote. I know way back in the early mainframe days before they had many computers or whatever that were still huge, but every one of the, like the early history of Cray and the places that gave birth to that company, every one of those computers, those big computers had its own programming style, its own basically assembly language and the way that it worked. And if you got a new computer, you'd have to rewrite your software to run on this new computer. + +00:16:17 We've come a long ways. + +00:16:18 So we have these nice building blocks like pandas and numpy and pullers and so on. + +00:16:21 Yeah. + +00:16:22 So I love that period of time just because it seemed like so bonkers where, you know, like I think in like the later 2000s when LVM really kind of becomes popular, there's a lot of languages. + +00:16:33 I think in like the 60s and 70s, that time period was also this like Cambrian explosion of languages that very niche things, many of them with the defense department, but also business things like the rise of COBOL comes around, so does FORTRAN, but wonderful languages like the stepped Reckoner, like a harking back to Leibniz things. There's a bunch of really cool... If you have a minute, you can look up this cool image from Gene Samet, who built this Tower of Babel-like image showing all these languages stacked on top of each other and highlighting that problem that you were just describing that if you moved between the IBM 360 to the Omdahl 720 or something like you had to rewrite your whole stack even though the map didn't didn't change or what you were + +00:17:18 working like the problem that you were actually trying to solve didn't really change. Yeah what what an insane time that was but so interesting there's actually a really good YouTube video if people want to check it out called Cray it was the rise and fall of the Cray supercomputer by Asianometry. I'll put a link to that. That goes a lot into it. It's really neat. + +00:17:37 Cool. + +00:17:37 Yeah, absolutely. So let's talk about Rapids. There's a bunch of cool stuff right on the homepage that are like little announcements that I think are going to be really fun to dive into. + +00:17:47 But the H2 here is GPU accelerated data science. And if, you know, Rapids is a Python package that you can install, but it's not just, it's kind of a, I guess you call it a meta package, right? + +00:18:00 Like when you install it, you get a bunch of things that work together. + +00:18:03 So tell us what is Rapids. + +00:18:06 Yeah, so Rapids is a suite of very popular data science libraries that have been GPU accelerated. + +00:18:12 So we've been exploring the space of how you do, again, those libraries that I was describing before that make the pillars of the PyData stack, NumPy, Pandas, Polars. + +00:18:22 Like it keeps, it's grown since we first started and have GPU equivalents of them. + +00:18:28 So, but maybe I can like wax on for a little bit longer. + +00:18:33 That's okay. + +00:18:33 Because the world has changed since we first started these things. + +00:18:36 So when Rapids first kicks off, we say, we'll take, I think many people actually, not just Rapids, says, well, I want, how do I make pandas go faster? + +00:18:46 How do I make Syketlin go faster? + +00:18:48 And there's a lot of products that are built that are import foo as PD or import X as SKLARM. + +00:18:59 And that's where we start off as well. So we build QDF, which is as close to a one-to-one mapping of the Pandas API and build QML that's a similar strategy. It's as close as possible to a one-to-one mapping and same thing with QGraph and NetworkX. And what you have here on the screen is QSPATIAL and parts of SciPySignal. And QSIM is also related to scikit image. + +00:19:26 But when you, I don't know, your experience may differ, but when you go to actually import Foo as PD or import Kupai as NP, it doesn't work out as well as you might want it to. + +00:19:40 There's still enough edge cases there. + +00:19:42 There's enough sharp edges that it actually prevents you from having it just magically work as much as you might want it to. + +00:19:49 And that can be very frustrating. + +00:19:50 So you just move on. + +00:19:51 Right. + +00:19:52 So what you're saying is a lot of times people say things like import pandas as PD. + +00:19:58 A trick somebody might want to try or a technique would be like, well, if it's kind of a one-to-one mapping, could we just say import QDF as PD and see if it just keeps going? + +00:20:09 It's a really great first starting point, but there are some subtle differences. + +00:20:14 And if you go to the QDF documentation page, you can see a lot of these, a lot of that we've tried to highlight where things differ. + +00:20:21 So like on joins or value counts or group buys, pandas guarantees some ordering that Kudf by default doesn't. + +00:20:31 And we care deeply about performance. + +00:20:33 So we could probably meet those API expectations, but we're trying to balance both ergonomics and performance. + +00:20:40 I think even in the case of like Kudf and NumPy, there's going to be differences on indexing behavior. + +00:20:46 There's going to be some behavior where it won't allow you to do an implicit device-to-host calls. + +00:20:53 It will prevent you from doing things in a way that, again, it's a good starting point, but it's not enough to actually deliver on the magic of what I have of this one-to-one mapping that perfectly works. + +00:21:07 This portion of Talk Python To Me is brought to you by the folks at Posit. + +00:21:11 Posit has made a huge investment in the Python community lately, known originally for our they've been building out a suite of tools and services for Team Python. + +00:21:21 Have you ever thought of all the things that go into a Python data science project? + +00:21:25 You need your notebook or IDE, sure. + +00:21:27 Also a server or cloud environment to run it, a version of Python, packages, access to your databases, and internal APIs. + +00:21:35 That's a lot of setup. + +00:21:37 And if you change any of these things, when you return to your projects a month down the road, you might get different results. + +00:21:43 Wouldn't it be nice to have all of this set up for you in one easy-to-access place whenever you want to get work done? + +00:21:49 That's the goal of Posit Workbench. + +00:21:52 Posit Workbench allows data scientists to code in Python within their preferred environment without an additional strain on IT. + +00:21:59 It gives data scientists access to all the development environments they love, including Jupyter Notebooks, JupyterLab, Positron, and VS Code. + +00:22:07 And yet, it helps ensure reproducibility. + +00:22:10 Here's how it works. + +00:22:11 You or your IT team set up Posit Workbench on a powerful, dedicated server within your organization or on the same cloud service that is hosting your most important data sources, such as AWS, SageMaker, Azure, GCP, Kubernetes, or pretty much anywhere. + +00:22:27 There, you create dedicated pre-configured environments to run your code and notebooks, and importantly, you also configure access to proprietary databases and internal APIs. + +00:22:37 When it's time to onboard a new data scientist or start a new project, you just fire it up in Workbench, and it's fully configured and ready to go, including on the infrastructure side of things. All of this is securely administered by your organization. If you work on a data science team where consistency matters, you owe it to you and your org to check out Posit Workbench. + +00:22:56 Visit talkpython.fm/workbench today and get a three-month free trial to see if it's a good fit. + +00:23:01 That's talkpython.fm/workbench. The link is in your podcast player's show notes. Thank you deposit for supporting talk python to me so what should you yeah yeah well what's the likelihood it works if i have a simple problem if i'm say a biology student and i've i've written 20 lines of panda related code and it's kind of slow but could i just get away with it or is it like where are + +00:23:26 these rough edges yeah i think the rough edges come when you again with like some of the assumptions that we've made are usually around some of these ordering problems that i described before I think probably 20 lines of code, yeah, you're probably safe doing it. It's not that big. But as you get into enterprise code, things that are maybe using pandas as a library where you have a lot of all the different kinds of ways that pandas is delightful and sometimes complex, that makes all these guarantees hard. It makes it more and more challenging to make sure that we have met that we've delivered with what we say on the tin. It's harder to meet those things. + +00:24:08 Yeah, whatever that's. + +00:24:09 Yeah, I can. Well, the farther into the edge cases you go, the more that's true, right? + +00:24:13 Yeah. So this is where we start and that's not where we finished. So maybe I can also back up and say part of how Rapids has been interacting with this broader open source ecosystem is, Well, this is what we've done. + +00:24:28 But the ecosystem also wants to do these things. + +00:24:31 They are interested. + +00:24:32 The community, the much broader community is interested in exploring how to use these APIs that people have grown to love and depend on and have them dispatch or be used by other kinds of engines. + +00:24:43 So it's not just Rapids pushing something out into the world. + +00:24:47 It's also working with this broader community. + +00:24:49 So you see this in like the array API standard of how scikit-learn or how NumPy can dispatch to not just Kupy and NumPy, but also to Jax or to Desk or Xarray. + +00:25:01 And same thing with scikit-learn as they explore the space of how to do GP things. + +00:25:06 Yeah, super interesting. + +00:25:08 One thing that I think as I look through this list here, as you see, okay, here's all the ways in which, all the different aspects, all the different libraries that you're compatible with, right? + +00:25:20 Handis, scikit-learn, NetworkX, Scikit-Image, and so on. + +00:25:26 Those things are moving targets, right? + +00:25:29 Yeah. + +00:25:29 So how much of your job is to chase changes to those libraries to keep up what it says on the 10 that you're compatible with them? + +00:25:38 Yeah, it's a lot of work. So we try to adhere to NumPy deprecation cycles of making sure that we're within some kind of range of which version of NumPy we're supporting. But we do spend a lot of time trying to go back in time as much as possible to the older versions that we support, but also still keep up with the bleeding edge of the newest release. + +00:26:03 The way that we have tried to also solve this problem has been in a set of newer developments where we have these zero code change experiences. + +00:26:12 So while QDF and QML and QGraph provide as close to possible one-to-one mappings, we've been pushing on, I think the marketing term that we have for this is truly zero code change. + +00:26:26 So for QDF, for example, we have QDF.pandas. + +00:26:30 And this is a different product on top of Kudia, where we really try no code change, no import, no changes of imports. + +00:26:41 And it's a bunch of really amazing code that has gone into what might be kind of considered a giant try except. + +00:26:48 So you try and we'll take whatever code you have. + +00:26:52 We'll do some, you know, Python lets you kind of muck around with all sorts of fun things under the hood. + +00:26:56 We do that. + +00:26:58 And we'll try and run that on the GPU. + +00:26:59 if it doesn't work, we'll fall back to the CPU library. And that has been really fun and exciting to see that work for lots of reasons. One is because the engineering to make that happen has + +00:27:10 been really fun to get. You have to go into the depths of Python because it's not just using + +00:27:16 pandas directly, but using pandas as a library. How do you make sure that you actually have a pandas object or when a third-party library is using pandas, we don't do something crazy or we to do something wrong. + +00:27:28 Somebody says, is instance or especially a third-party library is doing something, right? + +00:27:35 And there's a lot of that. + +00:27:36 A lot of these Matplotlib and Xray and Seaborn and a whole bunch of other folks, or all these other libraries, do a bunch of instance checking. + +00:27:44 We need to make sure that that's guaranteed. + +00:27:46 So we built this for QDF Pandas. + +00:27:49 We did something similar for QML and scikit-learn. + +00:27:53 But each community is actually different. + +00:27:55 The NetworkX community instead has built a dispatching mechanism. + +00:27:58 So it's an environment variable that you can set to, instead of using NetworkX, it will dispatch to Kugrath. + +00:28:07 And I think the NetworkX community did that as part of, like they have other ideas of different accelerated NetworkX experiences, like NX parallel. + +00:28:16 So let's talk about this, just maybe talk through a little bit of code with QDF.pandas. + +00:28:22 So if I'm in a notebook, I can say percent load ext qdf.pandas. + +00:28:28 And then you just import pandas as PD. + +00:28:30 But you must be overriding the import hooks to actually change what that means. + +00:28:36 First of all, let me take a step back. + +00:28:37 And what if I'm writing a Python script? + +00:28:39 I don't have like these percent magic things. + +00:28:42 Yeah. + +00:28:42 So you can use a module. + +00:28:44 So you can say python-m load qdf.pandas as well. + +00:28:49 I think there's some instructions. + +00:28:50 Yeah. + +00:28:51 I see. + +00:28:51 And then like execute, give it an argument of your script or something like that. + +00:28:55 Okay. + +00:28:55 So, all right. + +00:28:56 That's interesting. + +00:28:56 The other part here is it's a, there's a, you know, comment, Pandas APIs now at GPU Accelerate. + +00:29:02 Great. + +00:29:02 The first thing you've got here is pd.readcsv and it says hash uses the GPU. + +00:29:09 How can you read a CSV faster using the GPU? + +00:29:13 Like, help me understand this. + +00:29:15 Yeah. + +00:29:15 So actually the QDF CSV reader was one of the first things that we, one of the earlier things that we tackled and it forced, it's a very, actually very broad problem because you immediately need to tackle, you don't have to tackle compression and decompression, but you do have to tackle string parsing on the, on, on the device and, and formatting issues and, and a whole bunch of other fun IO tasks. + +00:29:40 And it turns out that as like you can get an op reading CSV is depending, as they get much larger, typically in the like multiple gigabytes to tens of gigabytes, is a lot faster on GPU compared to the Panda CSV reader, because you're doing so much of that, so much of that parsing can be parallelized as you convert a one to an int. + +00:30:02 Yeah, because like the quote one to the 0, 0, 0, 0, 0, 1 in binary, right? That sort of thing. + +00:30:08 Yeah. Okay. Yeah, I guess I didn't really think of it, but especially with Pandas and pullers as Well, it'll potentially try to guess the data type and then do like conversion to date times or conversions to numbers. + +00:30:21 And then that could actually be the slow part, right? + +00:30:23 That's right, yeah. + +00:30:24 Okay. + +00:30:25 Well, I guess using the GPU for that makes sense. + +00:30:27 You just jam a bunch of text in there and you tell it to go wild on it and see what it can do. + +00:30:32 Yeah, that's right. + +00:30:33 Kudief, I should also say, sorry, I forgot to mention it, that Kudief Polar is also a relatively new offering. + +00:30:40 And that we've been working very closely with the Polarist community to not have... + +00:30:44 We're not... + +00:30:45 This mechanism, working closely with the Polarist community allowed us to just say instead of... + +00:30:50 On your collect call, you can define a particular engine type. + +00:30:54 So whether it's streaming or whether it's GPU, now we have a similar kind of very easy button for these worlds. + +00:31:01 So it's not like we're trying to dogmatically dictate what each experience has to be, but work with all these, the community at large or each individual library communities and what best + +00:31:13 works for them. Yeah, super neat. One of the things that I saw, where did I see it? Somewhere. + +00:31:19 One of these levels here. I saw that with, here we go, on the QDF top level bit, it says that it's built on Apache Arrow, a columnar memory format. Polars is also built on Apache Arrow. And Pandas now supports that as a possible backend instead of NumPy. + +00:31:40 But as Pandas 3 is coming out, it's going to be the default as well. + +00:31:44 So something I've heard a lot when I was reading about Rapids and stuff is zero copy interop with other parts of the ecosystem. + +00:31:53 And I asked you about the staying API compliant, but staying in-memory shape compliant. + +00:32:00 So you can take one thing and just go, here, have this. + +00:32:03 you don't have to transform it or marshal it over to a different format. You can just pass it over. + +00:32:08 Like that's pretty neat, right? Yeah. That's been, it's, yeah, we, we are definitely big. + +00:32:14 Many of us in Rapids have wear open source badges very proudly and want, and push these, push ourselves to do these kinds of things because it only works if you get interop, like throughout the much broader community. So it's not just that we built a very fast merge that doesn't work with anybody else or that we have a GPU accelerated library that you have to stop what you're doing in order to do some viz. It works everywhere. And that means relying on Arrow as an in-memory data format, or even things like array function dispatching from NumPy, or the CUDA array interface, things like DLPack. All these things have to actually work in some amount of harmony to actually help the end user do what they're trying to do. It's all about the + +00:32:58 user at the end of the day. Yeah. It's a pretty wild initiative to say, let's try to replicate the most important data science libraries into a cohesive whole that does a lot of what, you know, Pandas and scikit-learn and stuff do, but like just on GPUs, that's a big undertaking. + +00:33:14 And still also interoperate with them, yeah. + +00:33:16 Yeah, it's really big. + +00:33:18 It's a bit grand, more than a bit grand. + +00:33:24 But I think we've seen like a lot of success. + +00:33:26 I mean, there's definitely like some trials and tribulations along the way, But I think we're ultimately pushing something and exploring the space in a way that gives users something that they can try out right this minute and actually get some benefit right now. + +00:33:39 So we have a lot of actually paying customers that want to use these libraries or that are deriving a lot of value from it and helping them accelerate what they were doing yesterday, today. + +00:33:49 There is what I really like about this world, though, is that it's still a bit of research. + +00:33:54 I mean, sorry, maybe more than grand thing to say is that there's not a whole lot of people that explore this space. + +00:34:00 And not only are we exploring it, but we're also putting it into production or we're not writing a white paper. + +00:34:08 Like our success is building a wheel or building a conda package. + +00:34:11 Yeah, yeah, that's super neat. + +00:34:14 So building this stuff as an open source library, which is pretty cool. + +00:34:17 So for example, QDF is a Apache 2 license. + +00:34:21 It's great doing it on GitHub, really nice. + +00:34:24 it allows people who are using this to say, not just go, please, please add a feature, but maybe they can look and suggest how to add the feature or they can do a PR or whatever. + +00:34:33 So what's the breakdown of NVIDIA people contributing versus other contributors? + +00:34:40 You know, like what's the story there? + +00:34:41 I said there's 12,000 closed PRs for QDF. + +00:34:46 The far majority of PRs that are being committed are by people that work at NVIDIA or work closely with LibQDF. + +00:34:54 We've seen other companies that have gotten involved in some niche cases. + +00:35:00 We've also seen a number of academics from other, usually from a GPU CS oriented lab that will get involved here. + +00:35:10 But what we see actually more open source interactions, it's the best thing in the world for any library. + +00:35:15 It's when somebody says, oh, I have a problem. + +00:35:17 That's so wonderful. + +00:35:18 We see a lot of issues from our users, which is so great. + +00:35:22 these when we were doing things as like import library as PD or scikit-learn, the users would probably not say anything. But now that we've built these zero-code change experiences and thought more about profiling and how to actually inform the user whether something is or is not happening, when something doesn't meet their expectations, they now have this opportunity to tell us something didn't go quite right or not getting the acceleration that I want, please help me. And that happens on GitHub issues and that happens on the GoAI Slack channel. It's really great to see. But for day-to-day contributions, yeah, the majority of them are happening at NVIDIA. But suggestions can... It's open source, so you can please come commit if you want to learn about GPU data science. Or if you have a feature request, please, we try to stay as responsive as possible to all the community interactions that are community vectors that we're a part of. + +00:36:19 Yeah, super neat. It's really cool that it's out there like that. So let's talk about sort of the effect of choosing something like Rapids over NumPy-backed pandas. + +00:36:33 I've read a bunch of stuff about that used to take a week and now it takes minutes. That's an insane that's an insane sort of speed up and you know i was talking about the power just kind of like yeah these things are crazy like the power consumption and stuff but the other angle that you know people say like oh this uses so much energy one consideration though is it might use a ton of energy for this compute but it might do it for minutes instead of for a week on a cluster of cpus right so it's not yeah yeah it's not as insanely far out as you think but there's a lot of stuff that makes sort of like scaling up, scaling down, I think pretty interesting that I want to talk to you about. But first of all, just maybe give us some examples of what are data scientists and people doing computational stuff seeing by adopting this or what's become + +00:37:21 possible that used to be unreasonable? Yeah. So an internal goal, at least for many of the people that I work with, is usually we're trying to get to like five to 10x performance speed up of versus what already exists out there, whether that's, yeah, for typically comparing against some CPU equivalent. There are definitely cases where we're trying to push into that area. There are definitely cases where it's not as performed, where you're getting like one and a half or two. + +00:37:49 Generally, our metric for success here is like in the five to 10X range. You will definitely come across these absolutely bonkers speed upsets, a thousand X faster. And they're not fabricating those speedups. Usually that's because they're doing some single threaded Python thing. And now the GPU has just unlocked this unreal performance that they were doing before. + +00:38:13 Go and find a bunch of NVIDIA blog posts that make those claims. I think there's been some on climate science and writing some Numba GPU kernels. But we typically see this where you get these benefits. If you're comparing QDF to pandas, you're comparing this incredibly parallel powerful GPU machine to what might mostly be a single core, in some cases, a little bit of multi-core interactions on CPU. And you can get, it's very easy to get these very, very large speedups where I think the same is also true. The same can be true for scikit-learn as well, where we're model training and just running, especially like hyperparameter searching, where you're just doing the training over and over and over again with different parameters. + +00:39:01 You can get very powerful, very large speedups comparing scikit-learn to QML or just CPU to GPU. + +00:39:11 But I think what I also find exciting is that the CPU world and the Python world is not sitting on their hands. + +00:39:17 There's all these other scikit-learn developers are pushing into doing more multi-core things. + +00:39:23 And Polars has actually come out with a bunch of very, very powerful multi-core native tooling that's very exciting. + +00:39:32 So when you compare Kudiev to Pandas, you can see these very powerful speedups. + +00:39:36 You compare GPU Polars to CPU Polars, the speedups are definitely still there, but they're somewhat diminished because CPU Polars itself is quite powerful. + +00:39:47 Yeah, just to give people a sense, I'm speaking to you all right now on my Mac Mini M2 Pro, which has 10 CPUs or CP cores. + +00:39:56 And if I go and run Python, computational Python code, it's single-threaded, so it's one-tenth of my machine, right? + +00:40:05 But if there's nothing stopping people like Richie Vink from adding internal parallelism to certain important operations inside pullers, right? + +00:40:14 And just by that virtue, it's 10 times faster on my machine. + +00:40:18 So, well, ish, right? + +00:40:21 It's on the scale of 10 times more compute resources anyway. + +00:40:25 So if you said, you know, the rapid stuff was 100 times faster before, well, now maybe it's 10 times faster. + +00:40:31 And that might sound not as impressive, but that's just progress in other areas, right? + +00:40:35 Yeah, I think it's great all around. + +00:40:38 Yeah, it's fun to see. + +00:40:40 I mean, even though I'm here talking about GPU data science, I think it's just like really great to see more of the Python data science ecosystem really leveraging and understanding more about the hardware. + +00:40:51 whether that's like the multi-core nature of all the machines that we have now, or even I think like, you know, a decade ago, people were like, oh, there's these L1, L2, L3 caches that we can take advantage of. We should target that. How do I do that? How do I make that? How do I expose that in Python? Where it's not already baked in, your work isn't baked into BLOS or these vector code + +00:41:14 bases that have existed for a long time. Yeah. I hadn't even thought about like specifically trying to address the L1, L2, L3 cache sort of deals. + +00:41:22 Like those caches are hundreds of times faster than main memory. + +00:41:26 There really is a big, big difference. + +00:41:29 And if you structure, it's like, well, what if we aligned our data structures that are allocated them this way in memory, then maybe we would like be able to hit the cache a lot more, you know, the L2 cache or whatever. + +00:41:40 It's crazy. + +00:41:41 Yeah. + +00:41:42 Working at NVIDIA, we think about, or when we think about Rapids, we think about that entire pipeline as well. + +00:41:46 How do we move data? + +00:41:48 as efficiently as possible from disk to memory to GPU memory, do some compute and try and take advantage of all of the bits of hardware in between them. Yeah. What do you think about + +00:42:00 Python T as in free threaded Python? The big news is just a week or two ago, it was officially accepted, you know, like was that PEP 703? I think it was that got accepted in Python 313 as sort of experimentally accepted. And I've never heard of something being added to Python as like, well we'll give it a try but we might take it out but that's how that was added and it is it kind of got the all right you're going to the next stage you're you're more likely to not be kicked out or i'm not sure if it's 100 guarantee but that's going to have a lot of knock-on effects as well right it especially affects the data science space because if you're writing extensions through the c apis or rust right you've got to think more about thread safety because python all of a sudden can become concurrent like it didn't used to be able to? + +00:42:46 Yeah, it opens up a big door. + +00:42:50 I think in the initial PEP, one of their highlighted use cases was actually the problem that we were just talking about of how do you pipeline efficiently across multiple devices? + +00:43:01 So in the PyTorch case, you need to maybe spin up a bunch of different, at the moment, you need to spin up a bunch of Python processes in order to efficiently load data from disk into your PyTorch or your deep learning pipeline. + +00:43:16 Doing that with multiprocessing is not a wonderful world. + +00:43:20 And we could probably be a lot better. + +00:43:22 Free threading maybe opens up that door. + +00:43:25 Yeah. + +00:43:25 I also see it adding possible challenges, not just benefits. + +00:43:29 Like, for example, if I go and write a bunch of multi-threaded code that's truly multi-threaded, like C and C# and other languages have been for a while, pretty much forever, and I start interacting with it, Like, does that, you know, we talked about L2 cache and keeping it active. + +00:43:45 Like, what about the GPU? + +00:43:46 Like, does that potentially open up a case where the GPU gets loaded up with, you know, tons of data and gets dropped because a thread contact switch happened and like just it thrashes? + +00:43:57 It's possible that that could happen. + +00:43:59 Sorry, this line of questioning also opens the door for me to just briefly talk about like these larger systems that NVIDIA, + +00:44:06 but other folks have been building as well, like where you have coherent memory. + +00:44:09 Yeah, so in this new architecture, Grace Hopper or Grace Blackwell, there's a specific communication channel between device and host. + +00:44:20 It can, I think it's called chip-to-chip technology or NBLink C2C, and you can move data back and forth between device and host at around 900 gigabytes per second. + +00:44:30 That's basically free, right? + +00:44:33 Or sometimes it's fun to think about it. + +00:44:35 I'm pretty sure it's faster than my RAM on my Apple Silicon. + +00:44:38 Yeah. + +00:44:39 So thrashing is not good, but if you're for whatever reason in that scenario for a pipeline, you might not feel it in these new coherent memory systems. + +00:44:52 Yeah, that's wild. + +00:44:54 I also think you probably just, you know, it might be one of those things where it's like, doctor, my leg hurts when I bend it this way. + +00:45:00 And they said, well, don't bend it that way. + +00:45:02 You know what I mean? + +00:45:03 It hurts when I try to run like 10 concurrent jobs on the GPU on the same computer. + +00:45:08 well don't do that you know what i mean sure that that might be that might be the way like use a + +00:45:13 use a lock and don't let that stuff run then right yeah the answer for for probably these things is probably don't do it you shouldn't you shouldn't just absorb that problem dr hitters like no don't + +00:45:23 do that um but there are ways to scale up and i got all this comfort this is kind of what i was leaning towards is like there's interesting ways to scale up um across like multi gpu i know that Dask has an interesting interop story and Dask has super interesting grid computing ways to scale. So like Dask can kind of do pandas, but larger than memory on your machine. And it can take advantage of the multiple CPUs cores, and it can even scale out and across clusters. Right. And so there's some integration with Dask and other things. Do you want to talk about that side of the + +00:45:58 story? Yeah. Yeah. So maybe very briefly, Dask is trying to scale Python data science as well. + +00:46:06 So I think actually, if I can just get into a little bit of the history, + +00:46:09 there's lots of people, I think just before a bunch of people are importing library as PD, + +00:46:16 there's a lot of people, I think even historical people that have been exploring, how do I do distributed NumPy or distributed memory array applications, like both in the HPC world and also in the enterprise world. And they're rewriting a library from scratch. And Dask comes along with the idea of, well, I'll just take the... I'll build some block, I'll build some distributed version of NumPy, but still keep NumPy or still keep pandas as the central compute engine for what's happening. And I'll build orchestration mechanisms around that and build distributed algorithms around pandas or around NumPy. And there's a way for you to both scale out horizontally and also scale up because you could get pandas now as a multi-core thing and you could get numpy as this distributed scale out solution and much of the dask world actually evolved with with rapids as well in the last like five five years is last five years where because we were building a pandas like library in rapids like qdf we could get dasks to also do our scale out scale-out mechanisms. So we built some hooks for Desk to, or we tried to generalize what is a data frame. If it meets these things, I can use pandas, I can use QDF, I can use the next data frame library after that. We also built some specific hooks inside of Desk to take advantage of accelerated networking and making sure that GPU buffers got shipped around to all the different + +00:47:45 workers. That's pretty wild. So could I have, I guess I could probably have multiple GPUs on workstation, right? When I say that, I know you can have multiple GPUs and you can link them in hardware, but could I just literally plug in multiple GPUs and take advantage of them as well? + +00:48:03 Yeah. You can have, underneath my desk, I have a two GPU workstation that does have an NVLink bridge between the two, but you could also just have them work at many, many, many workstations, + +00:48:17 just have two GPUs plugged into the PCIe board. And things will work there as well. + +00:48:21 Yeah. There are some performance considerations there where if you want to move, if you're communicating data between those two devices, you pay a bit of a cost. You have to move data from device to host. You then have to serialize it across the network and then move data from host to device. This is why having that NVLink bridge is so powerful if you have it in your system. + +00:48:45 Okay. So if people are already using Dask, how easy is it to adopt the setup? Or do you even have to think about it? Is this Dask storage just underneath the covers of the API? + +00:48:57 If you're already using Dask, you can already use DaskQDF or DaskKupai. Those things work. + +00:49:06 I've done some experiments and I've not had as much success, but people are still pushing quite hard as using Dask as a third-party library. + +00:49:13 So how do I make a GPU version of X-Ray? + +00:49:17 Well, that actually takes a bit more work. + +00:49:19 And there are people that are pushing quite hard, as I was saying before. + +00:49:21 But X-Ray, at least when I attempted it like three or four years ago, has a lot of mixture of DESK, of Pandas calls, of NumPy calls. + +00:49:30 And it was hard, at least in my attempt, to perfectly articulate all the GPU mechanisms that needed to be satisfied to make it work seamlessly or get any performance benefit. + +00:49:41 But I'm not as up to date on it. + +00:49:43 Maybe there's been some recent developments there. + +00:49:45 Yeah, there's a lot of moving parts. + +00:49:48 And they're all moving. + +00:49:49 And what used to be impossible and now is no problem. + +00:49:53 But you haven't tested that combination, right? + +00:49:56 Yeah. + +00:49:57 I'm very encouraging of anybody who wants to work on that problem or explore that space. + +00:50:02 I think geospatial, geoscience things definitely need all the attention they can get in an ever-changing climate world, climate science kind of problems that we are all experiencing as humans. + +00:50:13 Yeah, absolutely. + +00:50:14 So we talked earlier about the challenge of staying in touch with all these different APIs and staying consistent with them. + +00:50:23 Do you have really interesting test cases? + +00:50:25 Do you have some mother-of-all-py test execution sort of thing where do you maybe take the pandas unit tests and try to run them on QDF and similarly with scikit-learn and so on? + +00:50:39 Yeah, that should have been my first answer now that I think about it. That's exactly what we do do. For Kudyat Pandas, we run the... So for the Kudyat Pandas product, we do run the Pandas unit test. And we see... That's the goal, to have this run perfectly across it. It's not necessarily to accelerate all the APIs, but making sure that we never fail. + +00:51:02 They fall back to CPU if they have to. + +00:51:04 They fall back to CPU if they have to, exactly. + +00:51:07 We're also recording where we aren't using the GPU. + +00:51:10 So it gives us some directional information about the kinds of things that aren't accelerated. + +00:51:15 So maybe there's some niche datetime features or some niche extension D-type things that we aren't handling or can't be handled. + +00:51:24 And the same thing is also true for scikit-learn and QML. + +00:51:28 I think there are some known, actually at the moment, there are some known failures for for QML and SecondLearn. But we do, that is like the easiest thing that we could do. And we, + +00:51:35 we do that. Yeah. And are they, they're pretty much all running. I know you talked a little bit about QML and stuff, but how much pytest.ignore is in there? There's not as much as you, + +00:51:47 I think for KUDIF pandas, we are at like 99.x% passing for, for it. For QML, I have to look it up. But I think we still have like pretty, pretty good coverage of, of the entire scikit-learn code base in terms of not a falling back correctly, not necessarily accelerating it. There's a lot of, + +00:52:07 there's a lot of classifiers there. Okay. Yeah. I'm sure there are. + +00:52:11 That might be interesting. I don't know if this is like documented somewhere or whatever, but that might be interesting as a way for people who are considering adopting it to go like, well, let's see what the failing or ignored tests are like, no, these don't seem to apply to me. + +00:52:23 we're probably okay i think actually on the kumel documentation page there's a known limitation section that outlines the kinds of estimators that are not are not accelerated on some edge cases that are not supported at the moment but they're working again the team is quite motivated to like keep on as you've mentioned before it's an ever-changing world we're not we're + +00:52:44 going to keep on working on these problems my now admittedly pretty old geforce card in my gaming computer I got in 2020. I don't know how much RAM it has, four gigs, eight gigs, something like that. + +00:52:56 But I know there are data problems that are certainly bigger than four or eight gigs of data. + +00:53:01 What happens if I try to read CSV and the CSV is 12 gigs and I've got eight gigs of available? + +00:53:07 I love that you asked this question because it's been a focus of our group for like the last year and a half. A lot of very fun engineering problems have to be solved when you want to do of core processing and there's a lot of tools that we can deploy to solve this problem. + +00:53:25 So for single GPU, there already is a solution that was available to us that we needed to improve upon, but still could largely just deploy. + +00:53:35 And that's a CUDA memory type. + +00:53:38 CUDA has a bunch of different kinds of memory that you can use. + +00:53:41 So there's not just CUDA malloc, but there's an asynchronous malloc. + +00:53:46 And there's also a larger pool that you can build and then pull memory from this larger pool to help speed things up. + +00:53:53 There's also something called UVM or Unified Virtual Memory. + +00:53:58 And in this case, the driver, the CUDA driver itself, is going to try and allocate some memory on the GPU. + +00:54:05 And if it can't, it will spill. + +00:54:07 It will page data from the GPU onto the CPU. + +00:54:11 And the driver just takes care of all of that for me. + +00:54:13 So if you have a 12 gigabyte data set and you're trying to read it into a car that only has eight gigabytes, you could probably get by with just UVM. + +00:54:26 A question that you still should ask yourself is whether you received any performance benefit. + +00:54:30 I really want to be very clear. + +00:54:32 If you're trying to do something on CPU, and it's faster and everything just works, you should stay there. + +00:54:37 You shouldn't bend over backwards. + +00:54:38 We're really working hard to make sure that you get a benefit from using these devices. + +00:54:42 The other thing that you can do is batch. + +00:54:45 So there's this exotic memory type that is like the default. + +00:54:49 Most users don't ever have to think about it or worry about it, especially in the case of QDF pandas. + +00:54:56 But the other thing that you can do is sip data out. + +00:55:00 You can batch it. + +00:55:01 So for pandas, that's a lot of work to write a lazy framework on top of it. + +00:55:06 But for polars, that already exists. + +00:55:08 I was thinking that's the default of kind of how polars works. + +00:55:12 Right. + +00:55:12 So we already have this mechanism to push to higher than memory, larger than memory limits, or do this out of core kind of processing because it's native to the Polars experience. + +00:55:22 I think we've also been seeing that more with some of our machine learning algorithms. + +00:55:29 So if you look at the 3.0 release of XGBoost has this external memory allocator where you can use host memory to batch data in before you start processing on GPU. And the same thing is also true for UMAP as well, where you can use host memory to store data temporarily as you start processing it on GPU. And it allows you to really push much higher than what this resource-constrained + +00:55:59 GPU environment is. If I want to run this stuff in the cloud, what are some good options? Do I go to DigitalOcean and just pick a GPU-enabled droplet? Or is there like super heavy-duty things I can get? Or maybe you're like me and you have a Mac Mini, but you want to play with stuff and you don't have an NVIDIA GPU on this computer. Yeah. Unfortunately for the Mac Mini, + +00:56:22 there aren't a lot of options. While all this is open source and the code can be read and can be contributed from anybody, these only work on NVIDIA hardware. So yeah, you can go get a a Droplet, spin up a Docker image. + +00:56:39 I think we not only have pip and kind of packages, but also have Docker containers. + +00:56:45 You can get AWS. + +00:56:46 You can get, I think, on this page, maybe not the page that you're on, but there's a deploy page where we have a lot of recommendations about how to deploy rapids in a variety of environments. + +00:56:59 Some things like some of the MLOps tools like SageMaker. + +00:57:03 Yeah, that's the page. + +00:57:04 I probably could do something like Google CoLab or something like that in my MacBan, right? + +00:57:10 Actually, that's the best place to get started. + +00:57:13 That's where we direct a lot of our users. + +00:57:16 Google CoLab has a free tier offering where you can just select a different backend. + +00:57:20 I think for Kaggle users, there's now even like L4s or multi-GPU L4s that they can get access to. + +00:57:26 Nice. + +00:57:27 Okay. + +00:57:28 And then one other thing before we wrap it up is, let's see, I just go to Rapids. + +00:57:34 I know at the beginning here, it says you got the new Polars GPU engine, what we talked about, and it's pre-installed on Google Colab and things like that. + +00:57:44 But you also have vector search now with QVS, which is what I'm going with unless I'm told otherwise. + +00:57:49 But if you're not using AI, you're not using LLMs, but you're literally building LLMs or you're augmenting LLMs, this vector search stuff and embeddings and whatnot is what you need. So I know you're not a super expert in vector embeddings and stuff, and neither am I, but maybe tell people quick about this + +00:58:08 library, this aspect. Yeah, this actually grows out of the QVS, or QD Accelerated Vector Search, grows out of the ML world that you have. In the ML world, for UMAP or clustering algorithms, You need neighborhood algorithms to help you do regular data science. + +00:58:30 And it turns out that taking a bunch of strings and doing math on them, which is what a vector, which is what an embedding is, I have some text, I need to do some math on it, and then I need to understand how that text relates to other things with even more math. + +00:58:44 It'd be something like cosine distance or Jaccard similarity or minhash, whatever it is. + +00:58:50 And I need to do that across a very large corpus of text. + +00:58:54 So vector search and vector rag become this go-to tool for the LLM space, which is, as I was just saying, a lot of math on text. + +00:59:03 How do I make that go faster? + +00:59:05 How do I build an index? + +00:59:06 How do I re-index faster and faster and faster as the corpus changes or it has an update? + +00:59:11 Yeah, you think geospatial is an interesting query. + +00:59:14 Like the number of dimensions of this kind of stuff is unimaginable, right? + +00:59:19 That's right. + +00:59:19 I don't know that as many day-to-day, like if you're in the LLM space, LLM building space or in the retrieval space, QVS is definitely going to be something you should take a look at. + +00:59:31 For the rest of the broader data science community, I don't think QVS is as relevant to that + +00:59:35 just as a small clarification. + +00:59:37 Right, like how many people are using vector databases that are not using LLMs? + +00:59:43 Probably not too many. + +00:59:44 Right, probably not too many. + +00:59:45 Yeah. + +00:59:46 All right, let's wrap this up with one final question. + +00:59:49 where do you go from here? What's next? + +00:59:51 What's next for Rapids? + +00:59:53 Yeah, Rapids and basically Python plus NVIDIA. + +00:59:57 I think what's next for Rapids is it's always going to be some bit of maintenance of what we currently have pushing more and more performance and then trying to always encourage people to try out what we have and actually deliver something that is ultimately providing value for them. + +01:00:18 I think that there's a lot of really cool things that are happening on the hardware side, as I mentioned before. + +01:00:23 Like Blackwell has some pretty cool things that I don't know that the users will feel, but I don't know that they're going to have to interact with. + +01:00:31 So there's this very fancy decompression inside of Blackwell. + +01:00:36 There's also, again, as I mentioned before, in this coherent memory world where there's just memory or can I treat the system as just memory, how does software look? + +01:00:45 If I want to, like, we're posing these questions to ourselves, but if there was just malloc in the world and that happened seamlessly between host and device in it, what kind of software would I be running? How would I try and architect that code? I think those pose a lot of, like, very interesting computer science and also computer engineering questions. I think for, like, NVIDIA and Python, as the guest that you had on before, Bryce Ledback was describing, I think exposing all these ideas to Python developers is really exciting. They might not maybe move into the C++ world, but I think a lot of Python developers want to understand how this device works and how they can manipulate it through their language of choice. We've seen that as we were actually just describing earlier, like I want to get access to the different cache levels, but we see Python core developers made that available to us. That's really wonderful. The whole world isn't just vector computing. I want to take advantage of the entire system. So I think a lot of it is going to be exposure, education, + +01:01:46 and more and more performance. Awesome. Any concrete releases coming up that + +01:01:51 people should know about? Rapins does releases every two months. So you can see some announcements of what we're planning in the next release. I think that we're coming up 2508 should be baking right now. And then we'll have 2510 and 2512, where we just announced a few months ago a multi-GP Polarist experience. You can scale, not just have scale up, but scale out with the Polarist front end. We're doing a lot of good work around Kuml XL, as I was mentioning before, getting more algorithms, trying to push to higher and higher data sets that UMAP works with. And we're also looking at, we're trying to spend some time looking at particular verticals, I think, especially in like the bioinformatic space as well. But, you know, really excited to hear from anybody if they have a problem that they need, that they need more power, more performance, you know, please, + +01:02:44 please raise your hand and come talk to us. Awesome. Yeah, I'll put your, some link to you somehow in the show notes that people can reach out. But you mentioned Slack earlier, is there Slack Discord? What are the ways? Yeah, there's a couple Slack channels. One of them is + +01:03:00 called GoAI. There's a CUDA Slack channel. + +01:03:05 And then there's also a community-based GPU mode Slack channel that's a little bit more deep learning oriented. But the GoAI Slack channel is something that is specific to + +01:03:15 Rapids. Yeah. Awesome. + +01:03:16 Ben, this has been really fun. + +01:03:19 People out there listening, maybe they want to get started. + +01:03:21 What do you tell them? What do they do to try out Rapids? Go to CoLab and just import CUDA Pandas. Import Kumel XL. Import NXCUDRAF. + +01:03:31 Everything is baked in. + +01:03:32 We worked really hard with Google to make sure that that environment was set up from the get-go. + +01:03:36 Awesome. + +01:03:37 All right. + +01:03:37 Well, it's been great to chat GPUs and data science with you. + +01:03:41 Thanks for being on the show. + +01:03:41 Thanks so much for having me. + +01:03:42 I really appreciate it. + +01:03:43 Yeah, you bet. + +01:03:44 Bye-bye. + +01:03:45 This has been another episode of Talk Python To Me. + +01:03:48 Thank you to our sponsors. + +01:03:50 Be sure to check out what they're offering. + +01:03:51 It really helps support the show. + +01:03:53 This episode is sponsored by Posit and Posit Workbench. + +01:03:57 Posit Workbench allows data scientists to code in Python within their preferred environment without any additional strain on IT. + +01:04:05 It gives data scientists access to all the development environments they love, including Jupyter Notebooks, JupyterLab, Positron, and VS Code, and helps ensure reproducibility and consistency. + +01:04:14 If you work on a data science team where consistency matters, check out Posit Workbench. + +01:04:19 Visit talkpython.fm/workbench for details. + +01:04:22 Want to level up your Python? + +01:04:23 We have one of the largest catalogs of Python video courses over at Talk Python. + +01:04:28 Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:04:33 And best of all, there's not a subscription in sight. + +01:04:35 Check it out for yourself at training.talkpython.fm. + +01:04:39 Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:04:43 We should be right at the top. + +01:04:45 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +01:04:54 We're live streaming most of our recordings these days. + +01:04:57 If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:05:05 This is your host, Michael Kennedy. + +01:05:07 Thanks so much for listening. + +01:05:08 I really appreciate it. + +01:05:09 Now get out there and write some Python code. + +01:05:28 *music* + diff --git a/transcripts/516-accelerating-python-data-science-at-nvidia.vtt b/transcripts/516-accelerating-python-data-science-at-nvidia.vtt new file mode 100644 index 0000000..a67bdbb --- /dev/null +++ b/transcripts/516-accelerating-python-data-science-at-nvidia.vtt @@ -0,0 +1,1610 @@ +WEBVTT + +00:00:00.020 --> 00:00:03.280 +Python's data stack is getting a serious GPU turbo boost. + +00:00:03.510 --> 00:00:15.060 +In this episode, Ben Zaitlin from NVIDIA joins us to unpack Rapids, the open source toolkit that lets Pandas, scikit-learn, Spark, Polars, and even NetworkX execute on GPUs. + +00:00:15.540 --> 00:00:21.840 +We trace the project's origins and why NVIDIA built it in the open, then dig into the pieces that matter in practice. + +00:00:22.420 --> 00:00:30.660 +QDF for data frames, QML for machine learning, QGraph for graphs, QXFilter for dashboards, and friends like Qspatial and Qsignal. + +00:00:31.200 --> 00:00:39.760 +We talk real speedups, how the Pandas accelerator works without a rewrite, and what becomes possible when jobs that used to take hours finish in minutes. + +00:00:40.360 --> 00:00:50.660 +You'll hear strategies for datasets bigger than GPU memory, scaling out with Dask Array, Spark acceleration, and the growing role of vector search with QVS for AI workloads. + +00:00:51.180 --> 00:00:55.860 +If you know the CPU tools, this is your on-ramp to the same APIs at GPU speed. + +00:00:56.420 --> 00:01:01.060 +This is Talk Python To Me, episode 516, recorded July 1st, 2025. + +00:01:16.820 --> 00:01:19.760 +Welcome to Talk Python To Me, a weekly podcast on Python. + +00:01:20.340 --> 00:01:22.000 +This is your host, Michael Kennedy. + +00:01:22.160 --> 00:01:50.420 +Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is sponsored by Posit and Posit Workbench. + +00:01:51.220 --> 00:01:58.400 +Posit Workbench allows data scientists to code in Python within their preferred environment without any additional strain on IT. + +00:01:59.160 --> 00:02:08.679 +It gives data scientists access to all the development environments they love, including Jupyter Notebooks, JupyterLab, Positron, and VS Code, and helps ensure reproducibility and consistency. + +00:02:09.080 --> 00:02:13.260 +If you work on a data science team where consistency matters, check out Posit Workbench. + +00:02:13.900 --> 00:02:16.340 +Visit talkpython.fm/workbench for details. + +00:02:17.440 --> 00:02:18.840 +Ben, welcome to Talk Python To Me. + +00:02:19.060 --> 00:02:20.560 +So excited to be talking data science with you. + +00:02:20.940 --> 00:02:23.720 +Yeah, thanks so much for having me. I'm really excited to talk to you as well. + +00:02:24.080 --> 00:02:31.340 +Yes, data science, hardware acceleration, graphics, but not really. So it should be super fun. + +00:02:31.700 --> 00:02:43.480 +Yeah, let's dive into it. Before we really get into using GPUs for data science, which I think is going to be super interesting, just tell us a bit about yourself. Who's Ben? + +00:02:44.380 --> 00:02:50.080 +Yeah, my name is Ben Zaitlin. I am a system software manager, I think is my title, at NVIDIA. + +00:02:50.780 --> 00:02:57.420 +I've been working in the Python ecosystem since I left graduate school in 2006, 2005. + +00:02:58.820 --> 00:03:02.360 +It's actually, unlike other people, this is only my second job. + +00:03:02.680 --> 00:03:08.920 +I moved from graduate school to working at Continuum or Anaconda, and then I came to NVIDIA. + +00:03:09.040 --> 00:03:14.540 +I've always been in the space of doing some kind of science with computers. + +00:03:14.940 --> 00:03:19.820 +Yeah, Anaconda or Continuum at the time, as it was known, been renamed. + +00:03:20.800 --> 00:03:23.280 +what a launchpad for this kind of stuff, right? + +00:03:23.610 --> 00:03:25.080 +Yeah, it's been crazy. + +00:03:25.840 --> 00:03:35.900 +I mean, I feel like I'm a little bit older now than obviously I was when I first joined things, but it's nice to be able to reflect and look back at how much was built over the last decade and a half or so. + +00:03:36.700 --> 00:03:37.240 +It really is. + +00:03:37.580 --> 00:03:38.260 +People use Conda. + +00:03:39.100 --> 00:03:39.780 +People use Conda. + +00:03:40.050 --> 00:03:41.520 +A bunch of things in the hip were fixed. + +00:03:42.720 --> 00:03:44.040 +More things need to be fixed. + +00:03:44.240 --> 00:03:45.100 +More things are being built. + +00:03:45.600 --> 00:03:46.060 +It's really great. + +00:03:46.340 --> 00:03:53.900 +Yeah, and not to go too far afield, But Anaconda Inc. is doing interesting stuff to push on different boundaries of Python. + +00:03:54.460 --> 00:04:03.320 +In addition to the data science and ML side, you know, they're funding a lot of work on PyScript, which I think is really important for the Python in the browser. + +00:04:04.060 --> 00:04:11.540 +Yeah, I think one, like many conversations I've had when I was at Anaconda about like how deploying Python can be so challenging. + +00:04:11.620 --> 00:04:16.260 +It wouldn't be great if we had this deployment vehicle like JavaScript, if everything just ran. + +00:04:16.579 --> 00:04:17.560 +You didn't need anything set up. + +00:04:17.720 --> 00:04:18.640 +It's just all on the browser. + +00:04:19.109 --> 00:04:23.080 +And seeing those ideas actually mature into an existing product is really exciting. + +00:04:23.600 --> 00:04:34.840 +I think there's some people that are really even pushing quite hard to see how do you connect WebGL to getting other parts of your local machine, like obviously the GPU in this case, accessible through the browser. + +00:04:35.100 --> 00:04:37.520 +But it's really exciting to see that work. + +00:04:37.520 --> 00:04:37.720 +All right. + +00:04:38.400 --> 00:04:39.380 +Now you're just blowing my mind. + +00:04:39.560 --> 00:04:46.220 +I didn't connect the WebGL, the OpenGL of the browser to GPU acceleration, but of course. + +00:04:46.480 --> 00:04:53.920 +You can see a few people in PyScript, like if you go to the issue tracker, like, oh, I want to, how do I use PyTorch or how do I use TensorFlow inside of this, inside of my browser? + +00:04:54.160 --> 00:04:56.620 +Occasionally you'll see some people will talk about it later. + +00:04:56.820 --> 00:04:58.660 +Like, how do I use Rapids or other things like that? + +00:04:58.940 --> 00:05:01.780 +But once you open the doors, everybody just wants everything there. + +00:05:02.240 --> 00:05:04.000 +It's like, oh, yes, I understand. + +00:05:04.230 --> 00:05:04.560 +I see. + +00:05:04.910 --> 00:05:05.000 +Okay. + +00:05:05.180 --> 00:05:05.300 +Yeah. + +00:05:05.580 --> 00:05:05.660 +Yeah. + +00:05:05.810 --> 00:05:13.000 +And, you know, I said multiple areas, not just PyScript, but also they're doing a lot of work to support Russell Keith McGee. + +00:05:13.940 --> 00:05:15.280 +And it's Malcolm. + +00:05:15.380 --> 00:05:21.140 +I'm sorry if it's not Malcolm, the guy working with him on that, to bring Python to mobile as well. + +00:05:21.360 --> 00:05:22.940 +So those are really important initiatives. + +00:05:23.420 --> 00:05:23.600 +Yeah. + +00:05:24.180 --> 00:05:28.020 +Python is not just a niche language. + +00:05:28.080 --> 00:05:35.300 +It's found itself in every bit of computing up and down the stack from mobile to workstation, HPC, everywhere. + +00:05:36.240 --> 00:05:42.540 +So I want to start this conversation and jumping into Rapids with a few comments. + +00:05:42.700 --> 00:05:49.740 +First of all, I recently did an episode on just GPU programming. + +00:05:50.340 --> 00:05:51.800 +So that was really fun. + +00:05:52.220 --> 00:05:54.980 +And a quick, where was it? + +00:05:55.260 --> 00:05:55.480 +Oh, yeah. + +00:05:56.080 --> 00:06:01.640 +That was with Bryce Aldeis-Lolbach. + +00:06:02.240 --> 00:06:02.680 +Sorry, Bryce. + +00:06:02.960 --> 00:06:04.040 +I didn't remember the whole name. + +00:06:04.160 --> 00:06:05.080 +I'm like, I'm going to screw this up. + +00:06:05.080 --> 00:06:05.720 +I got to look it up. + +00:06:06.320 --> 00:06:12.440 +So we talked a little bit about GPUs and stuff, and not so much about Rapids and the side of things. + +00:06:12.620 --> 00:06:40.700 +that you're working on, although we definitely did touch on it a little bit. So that's another resource for people that really want to go deep into this area. But the thing that I want to go and actually talk about is I want to introduce this with a story from when I was in college in the 90s and a question that really surprised me. I was working doing this like applied math project using Silicon Graphics mainframes in complex analysis, if people know what that is. + +00:06:41.000 --> 00:06:45.920 +And I was programming with OpenGL to do some visualizations and stuff. + +00:06:46.020 --> 00:06:52.320 +And people, I was trying to get some help and someone's like, Hey, Hey, are you using the GPU for the math calculations? + +00:06:53.020 --> 00:06:54.240 +Because I want to hear about that. + +00:06:54.300 --> 00:06:56.420 +I'm like, I don't even, that doesn't even make sense to me. + +00:06:56.480 --> 00:06:57.520 +Like, why would you even ask me that? + +00:06:57.740 --> 00:06:59.400 +GPUs are for pictures and graphics. + +00:07:00.200 --> 00:07:02.420 +I like write loops and stuff for the math part. + +00:07:02.580 --> 00:07:02.680 +Right. + +00:07:03.020 --> 00:07:08.900 +But we've come so far and now GPUs really are a huge part of computation. + +00:07:09.260 --> 00:07:09.320 +Right. + +00:07:09.420 --> 00:07:14.980 +And Bryce said, hey, I've worked at NVIDIA for a long, long time, and I know nothing about graphics and 3D programming. + +00:07:16.900 --> 00:07:26.420 +Yeah, everybody's journey into this has been somewhat rejecting some popular notions about what GPUs are and are not for and to really testing those ideas. + +00:07:26.450 --> 00:07:33.420 +I think many of us, even still right now, think of GPUs as mostly being for drawing triangles or just linear algebra. + +00:07:34.260 --> 00:07:57.880 +And those are best in class of what GPUs are for, but it turns out that they're actually not terrible for doing string processing, or they're not as fast as they are for doing dense linear algebra, but they're really still quite good computing platforms for doing a lot of bulk processing around all kinds of data that are not typically what you think of for GPU processing. + +00:07:57.980 --> 00:08:02.360 +But like in data science, so much of what we're seeing now is actually still string based. + +00:08:02.430 --> 00:08:08.920 +So how do we get, we don't, we can't just, we can't always just isolate parts of the code to be just for strings or just for compute. + +00:08:09.010 --> 00:08:28.600 +We have to do all of it together and exploring how we can do that all on the device has been really quite, quite revelatory that it's actually pushing us to tell maybe how to inform how hardware maybe should be built or where we can actually do some software tricks to make some of these non-standard processing still quite performant. + +00:08:28.880 --> 00:08:30.500 +I'm sorry, I think I cut you off a little bit. + +00:08:30.660 --> 00:08:32.200 +Oh, no, no, it's great. + +00:08:32.539 --> 00:08:39.860 +If people haven't physically handled some of these GPUs lately, they might not appreciate just how intense they are, right? + +00:08:40.099 --> 00:08:44.219 +Like, you know, you think, oh, I've got a little laptop and it's got a GPU in there and it must do a thing. + +00:08:44.420 --> 00:08:56.740 +But like the desktop high-end ones, like I couldn't put a higher GeForce card into my gaming computer because the power supply was only something like 800 watts or something insane. + +00:08:57.590 --> 00:09:01.420 +Can I even plug that into the wall without melting it, even if I got a bigger power supply? + +00:09:02.460 --> 00:09:03.220 +These things are crazy. + +00:09:03.480 --> 00:09:08.660 +And that's not even touching on the H100, H200 type of server things, right? + +00:09:08.820 --> 00:09:10.320 +Which is just next level. + +00:09:10.720 --> 00:09:14.460 +Yeah, there's a lot of power consumption for these accelerators. + +00:09:14.660 --> 00:09:49.980 +I think what she touched on as well is something that I've seen change over the last 10 years that we thought about GPUs as mostly being for just graphics as these niche computing devices. And they're still not exactly, at least in my mind, not exactly commodity hardware, but they're a lot more commonplace where it's not so revolutionary to think of, oh, what else can I do with the hardware that's in my laptop or in my workstation? And GPUs are definitely part of that narrative. We're becoming very common to think of what are other things that I can do with this just sitting around not doing something. + +00:09:50.100 --> 00:09:50.800 +Yeah, absolutely. + +00:09:51.300 --> 00:10:07.420 +And I think just coincidentally, or just the way it works out, data science type of work, and really most significantly, the data science libraries, the way that they're built and the way they execute, line up perfectly with the way GPUs do their work. + +00:10:07.560 --> 00:10:12.740 +And what I'm thinking of is pandas, pollers, all the vector type of stuff. + +00:10:12.920 --> 00:10:43.640 +So instead of saying, I'm going to loop over and do one thing at a time, you just say, here's a million rows apply this operation to all million and then either update it in place or give me a new data frame or whatever and that is perfect for like let me load that into a gpu and turn it loose in parallel on all these pieces because as a programmer a data scientist i don't i don't write the imperative bits of it right i just let it go and it's easy for things like rapids to grab that and parallelize + +00:10:43.660 --> 00:11:33.520 +without me having to know about parallelism, right? Yeah, I think the same is also true in the CPU world as well. I don't know a lot about BLAS or LEPAC or these linear algebra libraries that have been in existence for 30 or 40 years. Those have been tuned to the gills to work on CPU, but I'm just the inheritor of all of that academic research. I don't need to know about caching algorithms or tiling algorithms. I just write my numpy or my pandas or my pullers, and generally, I'm pretty happy. Year after year after year, things generally get better for me without having to go very deep into computer science or even computer hardware design. I can still focus on boring business things or exciting business things or whatever particular vertical I'm in, whether it's genomics or selling ads or whatever it may be. + +00:11:33.740 --> 00:11:48.620 +There's a lot of layers to that, right? We've got the layers of like, you're talking about the libraries or like the CPU operations, but just the pip install or conda install type of layers that you can add on. They honestly don't need to know too much about what even they're doing, right? + +00:11:48.980 --> 00:12:13.840 +Yeah. I think that the demands of the user have definitely gone up and really pushed, I think all of the library developers to meet those demands. But when I, when I was first starting in computing, you know, you'd read a two or three pages worth of, you know, change this little bit of XML, compile this thing, do something else. Now the expectation really is like single button, if not single button data science, single button deployment of whatever it is I'm trying to + +00:12:13.980 --> 00:12:18.920 +work on. Yeah. Yeah. I want to write six lines of code in a notebook and have it do stuff that was + +00:12:19.180 --> 00:12:24.460 +previously impossible, basically. Right. Or I just want to express like math and then have it work. + +00:12:24.960 --> 00:12:32.180 +Or I mean, even now in some AI systems, I just want to express text and some agent takes care of everything for me. + +00:12:32.400 --> 00:12:33.380 +Yeah, it's absolutely. + +00:12:33.430 --> 00:12:34.800 +The AI stuff is absolutely crazy. + +00:12:34.910 --> 00:12:43.320 +And we'll come back to some of the integrations with like vector embeddings and vector search and all that kind of stuff at the end. + +00:12:43.940 --> 00:12:54.320 +But for now, let's maybe, I want to talk a bit about open source at NVIDIA and why you all decided to open source Rapids. + +00:12:54.370 --> 00:12:57.300 +And then we could maybe talk about how it came to be as well. + +00:12:57.620 --> 00:13:04.460 +So people probably think of NVIDIA mostly, Obviously, there's a lot of audience bias by listening to this show. + +00:13:04.800 --> 00:13:09.700 +But a lot of people think of NVIDIA as the gaming company or maybe just the GPU company. + +00:13:10.640 --> 00:13:13.320 +What's the software story and the open source story there? + +00:13:14.600 --> 00:13:18.060 +I think software is very important to NVIDIA. + +00:13:18.220 --> 00:13:22.640 +Obviously, CUDA is one of its primary vehicles to interact with the GPU. + +00:13:24.000 --> 00:13:34.000 +But NVIDIA has been exploring software in a more concentrated way over the last, I don't know, five, six years, at least since I've been there, probably it predates. + +00:13:34.240 --> 00:13:36.060 +I can't speak for all of NVIDIA in this way. + +00:13:36.720 --> 00:13:49.020 +But software becomes quite critical that if you want to deliver a full platform to people and have them use it, you need the software to be as good, if not better, than just the hardware. + +00:13:49.260 --> 00:13:50.980 +Everything, probably everything needs to work. + +00:13:51.040 --> 00:13:55.180 +I don't need to disparage any group or elevate any one group over another. + +00:13:56.940 --> 00:14:03.200 +And Rapids kicks off probably in late 2018, but it predates my time there. + +00:14:04.360 --> 00:14:12.980 +With a thesis of, well, we see a lot of, there's a lot of signal out in the world of whatever it is that I'm doing right now, how do I make it go 10x faster? + +00:14:13.380 --> 00:15:22.140 +And I think that's a very natural response to any timeframe, whether it's the 60s or 70s and having them search for whatever the hardware was doing back then, faster cores or multi, I don't know, as they approached, I think the first multi-core thing comes out in the early 80s. But we have this desire to do whatever it is that's happening right now, it can always be faster. There's actually this really great Grace Hopper quote that I like, where she's reflecting on some things from the 70s, that not only is data going to increase, but the demand for access to that data is going to increase. And I've heard a lot about data, beta size is increasing, but it was the first time I really saw somebody even back then saying like, oh, the demand for access was going to increase. So it's really, it's like innate for us to just always go faster. And then Rapid's approach to this problem was, well, these libraries have really become like canon for how you do data science in the Python world. The NumPy, Pandas, NetworkX, Matplotlib become the underlying pillars for this huge explosion of Python and PyData libraries. + +00:15:23.560 --> 00:15:25.400 +And we want to join that effort. + +00:15:25.840 --> 00:15:33.760 +How do we take a bunch of knowledge around writing very fast GPU kernels and bring it to this very large community? + +00:15:33.760 --> 00:15:43.980 +And there's a whole bunch of strategies that we try to employ to make it attractive, to make it possible, and to actually deliver something that ultimately will benefit what + +00:15:43.980 --> 00:16:16.500 +today versus what didn't happen yesterday. Yeah. You know, you're talking about the Grace Hopper quote. I know way back in the early mainframe days before they had many computers or whatever that were still huge, but every one of the, like the early history of Cray and the places that gave birth to that company, every one of those computers, those big computers had its own programming style, its own basically assembly language and the way that it worked. And if you got a new computer, you'd have to rewrite your software to run on this new computer. + +00:16:17.080 --> 00:16:18.060 +We've come a long ways. + +00:16:18.280 --> 00:16:21.680 +So we have these nice building blocks like pandas and numpy and pullers and so on. + +00:16:21.960 --> 00:16:22.300 +Yeah. + +00:16:22.470 --> 00:16:32.780 +So I love that period of time just because it seemed like so bonkers where, you know, like I think in like the later 2000s when LVM really kind of becomes popular, there's a lot of languages. + +00:16:33.270 --> 00:17:18.020 +I think in like the 60s and 70s, that time period was also this like Cambrian explosion of languages that very niche things, many of them with the defense department, but also business things like the rise of COBOL comes around, so does FORTRAN, but wonderful languages like the stepped Reckoner, like a harking back to Leibniz things. There's a bunch of really cool... If you have a minute, you can look up this cool image from Gene Samet, who built this Tower of Babel-like image showing all these languages stacked on top of each other and highlighting that problem that you were just describing that if you moved between the IBM 360 to the Omdahl 720 or something like you had to rewrite your whole stack even though the map didn't didn't change or what you were + +00:17:18.420 --> 00:17:37.160 +working like the problem that you were actually trying to solve didn't really change. Yeah what what an insane time that was but so interesting there's actually a really good YouTube video if people want to check it out called Cray it was the rise and fall of the Cray supercomputer by Asianometry. I'll put a link to that. That goes a lot into it. It's really neat. + +00:17:37.360 --> 00:17:37.460 +Cool. + +00:17:37.900 --> 00:17:46.560 +Yeah, absolutely. So let's talk about Rapids. There's a bunch of cool stuff right on the homepage that are like little announcements that I think are going to be really fun to dive into. + +00:17:47.260 --> 00:18:00.320 +But the H2 here is GPU accelerated data science. And if, you know, Rapids is a Python package that you can install, but it's not just, it's kind of a, I guess you call it a meta package, right? + +00:18:00.480 --> 00:18:03.640 +Like when you install it, you get a bunch of things that work together. + +00:18:03.810 --> 00:18:05.880 +So tell us what is Rapids. + +00:18:06.460 --> 00:18:12.440 +Yeah, so Rapids is a suite of very popular data science libraries that have been GPU accelerated. + +00:18:12.610 --> 00:18:22.900 +So we've been exploring the space of how you do, again, those libraries that I was describing before that make the pillars of the PyData stack, NumPy, Pandas, Polars. + +00:18:22.950 --> 00:18:27.880 +Like it keeps, it's grown since we first started and have GPU equivalents of them. + +00:18:28.900 --> 00:18:32.820 +So, but maybe I can like wax on for a little bit longer. + +00:18:33.020 --> 00:18:33.300 +That's okay. + +00:18:33.960 --> 00:18:36.680 +Because the world has changed since we first started these things. + +00:18:36.820 --> 00:18:46.680 +So when Rapids first kicks off, we say, we'll take, I think many people actually, not just Rapids, says, well, I want, how do I make pandas go faster? + +00:18:46.770 --> 00:18:48.300 +How do I make Syketlin go faster? + +00:18:48.920 --> 00:18:57.760 +And there's a lot of products that are built that are import foo as PD or import X as SKLARM. + +00:18:59.100 --> 00:19:26.320 +And that's where we start off as well. So we build QDF, which is as close to a one-to-one mapping of the Pandas API and build QML that's a similar strategy. It's as close as possible to a one-to-one mapping and same thing with QGraph and NetworkX. And what you have here on the screen is QSPATIAL and parts of SciPySignal. And QSIM is also related to scikit image. + +00:19:26.380 --> 00:19:40.100 +But when you, I don't know, your experience may differ, but when you go to actually import Foo as PD or import Kupai as NP, it doesn't work out as well as you might want it to. + +00:19:40.700 --> 00:19:42.840 +There's still enough edge cases there. + +00:19:42.900 --> 00:19:49.140 +There's enough sharp edges that it actually prevents you from having it just magically work as much as you might want it to. + +00:19:49.600 --> 00:19:50.580 +And that can be very frustrating. + +00:19:50.900 --> 00:19:51.620 +So you just move on. + +00:19:51.740 --> 00:19:51.960 +Right. + +00:19:52.000 --> 00:19:57.840 +So what you're saying is a lot of times people say things like import pandas as PD. + +00:19:58.330 --> 00:20:08.620 +A trick somebody might want to try or a technique would be like, well, if it's kind of a one-to-one mapping, could we just say import QDF as PD and see if it just keeps going? + +00:20:09.480 --> 00:20:14.500 +It's a really great first starting point, but there are some subtle differences. + +00:20:14.630 --> 00:20:21.740 +And if you go to the QDF documentation page, you can see a lot of these, a lot of that we've tried to highlight where things differ. + +00:20:21.960 --> 00:20:30.700 +So like on joins or value counts or group buys, pandas guarantees some ordering that Kudf by default doesn't. + +00:20:31.380 --> 00:20:33.600 +And we care deeply about performance. + +00:20:33.660 --> 00:20:40.040 +So we could probably meet those API expectations, but we're trying to balance both ergonomics and performance. + +00:20:40.820 --> 00:20:45.920 +I think even in the case of like Kudf and NumPy, there's going to be differences on indexing behavior. + +00:20:46.520 --> 00:20:53.240 +There's going to be some behavior where it won't allow you to do an implicit device-to-host calls. + +00:20:53.750 --> 00:21:06.000 +It will prevent you from doing things in a way that, again, it's a good starting point, but it's not enough to actually deliver on the magic of what I have of this one-to-one mapping that perfectly works. + +00:21:07.800 --> 00:21:11.240 +This portion of Talk Python To Me is brought to you by the folks at Posit. + +00:21:11.980 --> 00:21:20.020 +Posit has made a huge investment in the Python community lately, known originally for our they've been building out a suite of tools and services for Team Python. + +00:21:21.320 --> 00:21:24.740 +Have you ever thought of all the things that go into a Python data science project? + +00:21:25.500 --> 00:21:27.580 +You need your notebook or IDE, sure. + +00:21:27.980 --> 00:21:35.120 +Also a server or cloud environment to run it, a version of Python, packages, access to your databases, and internal APIs. + +00:21:35.680 --> 00:21:36.380 +That's a lot of setup. + +00:21:37.080 --> 00:21:42.540 +And if you change any of these things, when you return to your projects a month down the road, you might get different results. + +00:21:43.320 --> 00:21:49.040 +Wouldn't it be nice to have all of this set up for you in one easy-to-access place whenever you want to get work done? + +00:21:49.560 --> 00:21:51.320 +That's the goal of Posit Workbench. + +00:21:52.020 --> 00:21:58.420 +Posit Workbench allows data scientists to code in Python within their preferred environment without an additional strain on IT. + +00:21:59.340 --> 00:22:06.860 +It gives data scientists access to all the development environments they love, including Jupyter Notebooks, JupyterLab, Positron, and VS Code. + +00:22:07.200 --> 00:22:09.640 +And yet, it helps ensure reproducibility. + +00:22:10.060 --> 00:22:10.740 +Here's how it works. + +00:22:11.060 --> 00:22:26.480 +You or your IT team set up Posit Workbench on a powerful, dedicated server within your organization or on the same cloud service that is hosting your most important data sources, such as AWS, SageMaker, Azure, GCP, Kubernetes, or pretty much anywhere. + +00:22:27.100 --> 00:22:36.640 +There, you create dedicated pre-configured environments to run your code and notebooks, and importantly, you also configure access to proprietary databases and internal APIs. + +00:22:37.380 --> 00:22:56.060 +When it's time to onboard a new data scientist or start a new project, you just fire it up in Workbench, and it's fully configured and ready to go, including on the infrastructure side of things. All of this is securely administered by your organization. If you work on a data science team where consistency matters, you owe it to you and your org to check out Posit Workbench. + +00:22:56.520 --> 00:23:01.600 +Visit talkpython.fm/workbench today and get a three-month free trial to see if it's a good fit. + +00:23:01.940 --> 00:23:26.620 +That's talkpython.fm/workbench. The link is in your podcast player's show notes. Thank you deposit for supporting talk python to me so what should you yeah yeah well what's the likelihood it works if i have a simple problem if i'm say a biology student and i've i've written 20 lines of panda related code and it's kind of slow but could i just get away with it or is it like where are + +00:23:26.660 --> 00:24:07.940 +these rough edges yeah i think the rough edges come when you again with like some of the assumptions that we've made are usually around some of these ordering problems that i described before I think probably 20 lines of code, yeah, you're probably safe doing it. It's not that big. But as you get into enterprise code, things that are maybe using pandas as a library where you have a lot of all the different kinds of ways that pandas is delightful and sometimes complex, that makes all these guarantees hard. It makes it more and more challenging to make sure that we have met that we've delivered with what we say on the tin. It's harder to meet those things. + +00:24:08.400 --> 00:24:08.980 +Yeah, whatever that's. + +00:24:09.020 --> 00:24:12.880 +Yeah, I can. Well, the farther into the edge cases you go, the more that's true, right? + +00:24:13.220 --> 00:24:28.020 +Yeah. So this is where we start and that's not where we finished. So maybe I can also back up and say part of how Rapids has been interacting with this broader open source ecosystem is, Well, this is what we've done. + +00:24:28.640 --> 00:24:30.880 +But the ecosystem also wants to do these things. + +00:24:31.040 --> 00:24:31.940 +They are interested. + +00:24:32.400 --> 00:24:43.820 +The community, the much broader community is interested in exploring how to use these APIs that people have grown to love and depend on and have them dispatch or be used by other kinds of engines. + +00:24:43.940 --> 00:24:47.340 +So it's not just Rapids pushing something out into the world. + +00:24:47.420 --> 00:24:48.920 +It's also working with this broader community. + +00:24:49.140 --> 00:25:00.880 +So you see this in like the array API standard of how scikit-learn or how NumPy can dispatch to not just Kupy and NumPy, but also to Jax or to Desk or Xarray. + +00:25:01.230 --> 00:25:05.820 +And same thing with scikit-learn as they explore the space of how to do GP things. + +00:25:06.640 --> 00:25:08.140 +Yeah, super interesting. + +00:25:08.360 --> 00:25:19.940 +One thing that I think as I look through this list here, as you see, okay, here's all the ways in which, all the different aspects, all the different libraries that you're compatible with, right? + +00:25:20.320 --> 00:25:26.200 +Handis, scikit-learn, NetworkX, Scikit-Image, and so on. + +00:25:26.560 --> 00:25:28.480 +Those things are moving targets, right? + +00:25:29.120 --> 00:25:29.280 +Yeah. + +00:25:29.380 --> 00:25:38.280 +So how much of your job is to chase changes to those libraries to keep up what it says on the 10 that you're compatible with them? + +00:25:38.520 --> 00:26:02.200 +Yeah, it's a lot of work. So we try to adhere to NumPy deprecation cycles of making sure that we're within some kind of range of which version of NumPy we're supporting. But we do spend a lot of time trying to go back in time as much as possible to the older versions that we support, but also still keep up with the bleeding edge of the newest release. + +00:26:03.220 --> 00:26:12.300 +The way that we have tried to also solve this problem has been in a set of newer developments where we have these zero code change experiences. + +00:26:12.890 --> 00:26:25.780 +So while QDF and QML and QGraph provide as close to possible one-to-one mappings, we've been pushing on, I think the marketing term that we have for this is truly zero code change. + +00:26:26.220 --> 00:26:30.040 +So for QDF, for example, we have QDF.pandas. + +00:26:30.680 --> 00:26:40.260 +And this is a different product on top of Kudia, where we really try no code change, no import, no changes of imports. + +00:26:41.440 --> 00:26:48.020 +And it's a bunch of really amazing code that has gone into what might be kind of considered a giant try except. + +00:26:48.500 --> 00:26:51.720 +So you try and we'll take whatever code you have. + +00:26:52.080 --> 00:26:56.580 +We'll do some, you know, Python lets you kind of muck around with all sorts of fun things under the hood. + +00:26:56.860 --> 00:26:57.460 +We do that. + +00:26:58.040 --> 00:26:59.340 +And we'll try and run that on the GPU. + +00:26:59.540 --> 00:27:10.520 +if it doesn't work, we'll fall back to the CPU library. And that has been really fun and exciting to see that work for lots of reasons. One is because the engineering to make that happen has + +00:27:10.660 --> 00:27:15.880 +been really fun to get. You have to go into the depths of Python because it's not just using + +00:27:16.240 --> 00:27:27.820 +pandas directly, but using pandas as a library. How do you make sure that you actually have a pandas object or when a third-party library is using pandas, we don't do something crazy or we to do something wrong. + +00:27:28.180 --> 00:27:35.380 +Somebody says, is instance or especially a third-party library is doing something, right? + +00:27:35.800 --> 00:27:36.580 +And there's a lot of that. + +00:27:36.720 --> 00:27:44.000 +A lot of these Matplotlib and Xray and Seaborn and a whole bunch of other folks, or all these other libraries, do a bunch of instance checking. + +00:27:44.460 --> 00:27:46.020 +We need to make sure that that's guaranteed. + +00:27:46.600 --> 00:27:49.100 +So we built this for QDF Pandas. + +00:27:49.390 --> 00:27:52.340 +We did something similar for QML and scikit-learn. + +00:27:53.100 --> 00:27:54.940 +But each community is actually different. + +00:27:55.240 --> 00:27:58.400 +The NetworkX community instead has built a dispatching mechanism. + +00:27:58.800 --> 00:28:07.060 +So it's an environment variable that you can set to, instead of using NetworkX, it will dispatch to Kugrath. + +00:28:07.100 --> 00:28:15.540 +And I think the NetworkX community did that as part of, like they have other ideas of different accelerated NetworkX experiences, like NX parallel. + +00:28:16.919 --> 00:28:22.300 +So let's talk about this, just maybe talk through a little bit of code with QDF.pandas. + +00:28:22.620 --> 00:28:28.260 +So if I'm in a notebook, I can say percent load ext qdf.pandas. + +00:28:28.640 --> 00:28:30.380 +And then you just import pandas as PD. + +00:28:30.550 --> 00:28:35.380 +But you must be overriding the import hooks to actually change what that means. + +00:28:36.720 --> 00:28:37.820 +First of all, let me take a step back. + +00:28:37.820 --> 00:28:39.380 +And what if I'm writing a Python script? + +00:28:39.550 --> 00:28:41.680 +I don't have like these percent magic things. + +00:28:42.220 --> 00:28:42.500 +Yeah. + +00:28:42.550 --> 00:28:44.580 +So you can use a module. + +00:28:44.770 --> 00:28:49.080 +So you can say python-m load qdf.pandas as well. + +00:28:49.130 --> 00:28:50.260 +I think there's some instructions. + +00:28:50.860 --> 00:28:50.920 +Yeah. + +00:28:51.170 --> 00:28:51.440 +I see. + +00:28:51.560 --> 00:28:54.380 +And then like execute, give it an argument of your script or something like that. + +00:28:55.140 --> 00:28:55.260 +Okay. + +00:28:55.590 --> 00:28:56.080 +So, all right. + +00:28:56.200 --> 00:28:56.880 +That's interesting. + +00:28:56.950 --> 00:29:01.940 +The other part here is it's a, there's a, you know, comment, Pandas APIs now at GPU Accelerate. + +00:29:02.240 --> 00:29:02.400 +Great. + +00:29:02.700 --> 00:29:09.140 +The first thing you've got here is pd.readcsv and it says hash uses the GPU. + +00:29:09.940 --> 00:29:13.480 +How can you read a CSV faster using the GPU? + +00:29:13.660 --> 00:29:14.460 +Like, help me understand this. + +00:29:15.060 --> 00:29:15.220 +Yeah. + +00:29:15.260 --> 00:29:39.680 +So actually the QDF CSV reader was one of the first things that we, one of the earlier things that we tackled and it forced, it's a very, actually very broad problem because you immediately need to tackle, you don't have to tackle compression and decompression, but you do have to tackle string parsing on the, on, on the device and, and formatting issues and, and a whole bunch of other fun IO tasks. + +00:29:40.960 --> 00:30:01.760 +And it turns out that as like you can get an op reading CSV is depending, as they get much larger, typically in the like multiple gigabytes to tens of gigabytes, is a lot faster on GPU compared to the Panda CSV reader, because you're doing so much of that, so much of that parsing can be parallelized as you convert a one to an int. + +00:30:02.200 --> 00:30:08.740 +Yeah, because like the quote one to the 0, 0, 0, 0, 0, 1 in binary, right? That sort of thing. + +00:30:08.970 --> 00:30:20.820 +Yeah. Okay. Yeah, I guess I didn't really think of it, but especially with Pandas and pullers as Well, it'll potentially try to guess the data type and then do like conversion to date times or conversions to numbers. + +00:30:21.080 --> 00:30:23.600 +And then that could actually be the slow part, right? + +00:30:23.980 --> 00:30:24.460 +That's right, yeah. + +00:30:24.740 --> 00:30:24.920 +Okay. + +00:30:25.100 --> 00:30:27.020 +Well, I guess using the GPU for that makes sense. + +00:30:27.060 --> 00:30:31.820 +You just jam a bunch of text in there and you tell it to go wild on it and see what it can do. + +00:30:32.160 --> 00:30:32.700 +Yeah, that's right. + +00:30:33.540 --> 00:30:39.520 +Kudief, I should also say, sorry, I forgot to mention it, that Kudief Polar is also a relatively new offering. + +00:30:40.000 --> 00:30:44.600 +And that we've been working very closely with the Polarist community to not have... + +00:30:44.600 --> 00:30:45.220 +We're not... + +00:30:45.220 --> 00:30:50.100 +This mechanism, working closely with the Polarist community allowed us to just say instead of... + +00:30:50.680 --> 00:30:54.060 +On your collect call, you can define a particular engine type. + +00:30:54.680 --> 00:31:01.680 +So whether it's streaming or whether it's GPU, now we have a similar kind of very easy button for these worlds. + +00:31:01.710 --> 00:31:13.180 +So it's not like we're trying to dogmatically dictate what each experience has to be, but work with all these, the community at large or each individual library communities and what best + +00:31:13.480 --> 00:31:18.600 +works for them. Yeah, super neat. One of the things that I saw, where did I see it? Somewhere. + +00:31:19.160 --> 00:31:40.000 +One of these levels here. I saw that with, here we go, on the QDF top level bit, it says that it's built on Apache Arrow, a columnar memory format. Polars is also built on Apache Arrow. And Pandas now supports that as a possible backend instead of NumPy. + +00:31:40.200 --> 00:31:44.140 +But as Pandas 3 is coming out, it's going to be the default as well. + +00:31:44.560 --> 00:31:53.480 +So something I've heard a lot when I was reading about Rapids and stuff is zero copy interop with other parts of the ecosystem. + +00:31:53.980 --> 00:32:00.820 +And I asked you about the staying API compliant, but staying in-memory shape compliant. + +00:32:00.930 --> 00:32:03.240 +So you can take one thing and just go, here, have this. + +00:32:03.840 --> 00:32:08.460 +you don't have to transform it or marshal it over to a different format. You can just pass it over. + +00:32:08.840 --> 00:32:14.520 +Like that's pretty neat, right? Yeah. That's been, it's, yeah, we, we are definitely big. + +00:32:14.880 --> 00:32:57.940 +Many of us in Rapids have wear open source badges very proudly and want, and push these, push ourselves to do these kinds of things because it only works if you get interop, like throughout the much broader community. So it's not just that we built a very fast merge that doesn't work with anybody else or that we have a GPU accelerated library that you have to stop what you're doing in order to do some viz. It works everywhere. And that means relying on Arrow as an in-memory data format, or even things like array function dispatching from NumPy, or the CUDA array interface, things like DLPack. All these things have to actually work in some amount of harmony to actually help the end user do what they're trying to do. It's all about the + +00:32:58.140 --> 00:33:13.800 +user at the end of the day. Yeah. It's a pretty wild initiative to say, let's try to replicate the most important data science libraries into a cohesive whole that does a lot of what, you know, Pandas and scikit-learn and stuff do, but like just on GPUs, that's a big undertaking. + +00:33:14.380 --> 00:33:16.200 +And still also interoperate with them, yeah. + +00:33:16.680 --> 00:33:17.860 +Yeah, it's really big. + +00:33:18.440 --> 00:33:22.460 +It's a bit grand, more than a bit grand. + +00:33:24.180 --> 00:33:26.480 +But I think we've seen like a lot of success. + +00:33:26.920 --> 00:33:39.880 +I mean, there's definitely like some trials and tribulations along the way, But I think we're ultimately pushing something and exploring the space in a way that gives users something that they can try out right this minute and actually get some benefit right now. + +00:33:39.890 --> 00:33:48.820 +So we have a lot of actually paying customers that want to use these libraries or that are deriving a lot of value from it and helping them accelerate what they were doing yesterday, today. + +00:33:49.920 --> 00:33:54.440 +There is what I really like about this world, though, is that it's still a bit of research. + +00:33:54.880 --> 00:33:59.980 +I mean, sorry, maybe more than grand thing to say is that there's not a whole lot of people that explore this space. + +00:34:00.580 --> 00:34:07.680 +And not only are we exploring it, but we're also putting it into production or we're not writing a white paper. + +00:34:08.149 --> 00:34:09.899 +Like our success is building a wheel + +00:34:10.050 --> 00:34:11.080 +or building a conda package. + +00:34:11.580 --> 00:34:13.020 +Yeah, yeah, that's super neat. + +00:34:14.090 --> 00:34:17.560 +So building this stuff as an open source library, which is pretty cool. + +00:34:17.740 --> 00:34:20.600 +So for example, QDF is a Apache 2 license. + +00:34:21.100 --> 00:34:24.000 +It's great doing it on GitHub, really nice. + +00:34:24.360 --> 00:34:33.220 +it allows people who are using this to say, not just go, please, please add a feature, but maybe they can look and suggest how to add the feature or they can do a PR or whatever. + +00:34:33.840 --> 00:34:39.960 +So what's the breakdown of NVIDIA people contributing versus other contributors? + +00:34:40.370 --> 00:34:41.860 +You know, like what's the story there? + +00:34:41.919 --> 00:34:44.940 +I said there's 12,000 closed PRs for QDF. + +00:34:46.320 --> 00:34:53.919 +The far majority of PRs that are being committed are by people that work at NVIDIA or work closely with LibQDF. + +00:34:54.159 --> 00:35:00.260 +We've seen other companies that have gotten involved in some niche cases. + +00:35:00.520 --> 00:35:09.020 +We've also seen a number of academics from other, usually from a GPU CS oriented lab that will get involved here. + +00:35:10.220 --> 00:35:15.420 +But what we see actually more open source interactions, it's the best thing in the world for any library. + +00:35:15.580 --> 00:35:17.020 +It's when somebody says, oh, I have a problem. + +00:35:17.340 --> 00:35:18.100 +That's so wonderful. + +00:35:18.280 --> 00:35:22.320 +We see a lot of issues from our users, which is so great. + +00:35:22.600 --> 00:36:18.440 +these when we were doing things as like import library as PD or scikit-learn, the users would probably not say anything. But now that we've built these zero-code change experiences and thought more about profiling and how to actually inform the user whether something is or is not happening, when something doesn't meet their expectations, they now have this opportunity to tell us something didn't go quite right or not getting the acceleration that I want, please help me. And that happens on GitHub issues and that happens on the GoAI Slack channel. It's really great to see. But for day-to-day contributions, yeah, the majority of them are happening at NVIDIA. But suggestions can... It's open source, so you can please come commit if you want to learn about GPU data science. Or if you have a feature request, please, we try to stay as responsive as possible to all the community interactions that are community vectors that we're a part of. + +00:36:19.100 --> 00:36:33.320 +Yeah, super neat. It's really cool that it's out there like that. So let's talk about sort of the effect of choosing something like Rapids over NumPy-backed pandas. + +00:36:33.680 --> 00:37:21.860 +I've read a bunch of stuff about that used to take a week and now it takes minutes. That's an insane that's an insane sort of speed up and you know i was talking about the power just kind of like yeah these things are crazy like the power consumption and stuff but the other angle that you know people say like oh this uses so much energy one consideration though is it might use a ton of energy for this compute but it might do it for minutes instead of for a week on a cluster of cpus right so it's not yeah yeah it's not as insanely far out as you think but there's a lot of stuff that makes sort of like scaling up, scaling down, I think pretty interesting that I want to talk to you about. But first of all, just maybe give us some examples of what are data scientists and people doing computational stuff seeing by adopting this or what's become + +00:37:21.980 --> 00:37:49.320 +possible that used to be unreasonable? Yeah. So an internal goal, at least for many of the people that I work with, is usually we're trying to get to like five to 10x performance speed up of versus what already exists out there, whether that's, yeah, for typically comparing against some CPU equivalent. There are definitely cases where we're trying to push into that area. There are definitely cases where it's not as performed, where you're getting like one and a half or two. + +00:37:49.800 --> 00:38:13.440 +Generally, our metric for success here is like in the five to 10X range. You will definitely come across these absolutely bonkers speed upsets, a thousand X faster. And they're not fabricating those speedups. Usually that's because they're doing some single threaded Python thing. And now the GPU has just unlocked this unreal performance that they were doing before. + +00:38:13.750 --> 00:39:01.840 +Go and find a bunch of NVIDIA blog posts that make those claims. I think there's been some on climate science and writing some Numba GPU kernels. But we typically see this where you get these benefits. If you're comparing QDF to pandas, you're comparing this incredibly parallel powerful GPU machine to what might mostly be a single core, in some cases, a little bit of multi-core interactions on CPU. And you can get, it's very easy to get these very, very large speedups where I think the same is also true. The same can be true for scikit-learn as well, where we're model training and just running, especially like hyperparameter searching, where you're just doing the training over and over and over again with different parameters. + +00:39:01.910 --> 00:39:10.020 +You can get very powerful, very large speedups comparing scikit-learn to QML or just CPU to GPU. + +00:39:11.270 --> 00:39:17.660 +But I think what I also find exciting is that the CPU world and the Python world is not sitting on their hands. + +00:39:17.860 --> 00:39:23.660 +There's all these other scikit-learn developers are pushing into doing more multi-core things. + +00:39:23.920 --> 00:39:32.280 +And Polars has actually come out with a bunch of very, very powerful multi-core native tooling that's very exciting. + +00:39:32.480 --> 00:39:36.020 +So when you compare Kudiev to Pandas, you can see these very powerful speedups. + +00:39:36.460 --> 00:39:46.720 +You compare GPU Polars to CPU Polars, the speedups are definitely still there, but they're somewhat diminished because CPU Polars itself is quite powerful. + +00:39:47.060 --> 00:39:56.240 +Yeah, just to give people a sense, I'm speaking to you all right now on my Mac Mini M2 Pro, which has 10 CPUs or CP cores. + +00:39:56.800 --> 00:40:05.160 +And if I go and run Python, computational Python code, it's single-threaded, so it's one-tenth of my machine, right? + +00:40:05.340 --> 00:40:14.060 +But if there's nothing stopping people like Richie Vink from adding internal parallelism to certain important operations inside pullers, right? + +00:40:14.380 --> 00:40:18.320 +And just by that virtue, it's 10 times faster on my machine. + +00:40:18.840 --> 00:40:20.080 +So, well, ish, right? + +00:40:21.440 --> 00:40:25.040 +It's on the scale of 10 times more compute resources anyway. + +00:40:25.800 --> 00:40:31.640 +So if you said, you know, the rapid stuff was 100 times faster before, well, now maybe it's 10 times faster. + +00:40:31.800 --> 00:40:35.420 +And that might sound not as impressive, but that's just progress in other areas, right? + +00:40:35.640 --> 00:40:37.660 +Yeah, I think it's great all around. + +00:40:38.720 --> 00:40:39.720 +Yeah, it's fun to see. + +00:40:40.080 --> 00:40:50.980 +I mean, even though I'm here talking about GPU data science, I think it's just like really great to see more of the Python data science ecosystem really leveraging and understanding more about the hardware. + +00:40:51.580 --> 00:41:14.420 +whether that's like the multi-core nature of all the machines that we have now, or even I think like, you know, a decade ago, people were like, oh, there's these L1, L2, L3 caches that we can take advantage of. We should target that. How do I do that? How do I make that? How do I expose that in Python? Where it's not already baked in, your work isn't baked into BLOS or these vector code + +00:41:14.520 --> 00:41:21.820 +bases that have existed for a long time. Yeah. I hadn't even thought about like specifically trying to address the L1, L2, L3 cache sort of deals. + +00:41:22.030 --> 00:41:24.600 +Like those caches are hundreds of times faster + +00:41:24.820 --> 00:41:25.920 +than main memory. + +00:41:26.380 --> 00:41:29.080 +There really is a big, big difference. + +00:41:29.270 --> 00:41:40.080 +And if you structure, it's like, well, what if we aligned our data structures that are allocated them this way in memory, then maybe we would like be able to hit the cache a lot more, you know, the L2 cache or whatever. + +00:41:40.340 --> 00:41:40.860 +It's crazy. + +00:41:41.200 --> 00:41:41.600 +Yeah. + +00:41:42.250 --> 00:41:46.840 +Working at NVIDIA, we think about, or when we think about Rapids, we think about that entire pipeline as well. + +00:41:46.940 --> 00:41:48.000 +How do we move data? + +00:41:48.480 --> 00:41:59.560 +as efficiently as possible from disk to memory to GPU memory, do some compute and try and take advantage of all of the bits of hardware in between them. Yeah. What do you think about + +00:42:00.160 --> 00:42:46.220 +Python T as in free threaded Python? The big news is just a week or two ago, it was officially accepted, you know, like was that PEP 703? I think it was that got accepted in Python 313 as sort of experimentally accepted. And I've never heard of something being added to Python as like, well we'll give it a try but we might take it out but that's how that was added and it is it kind of got the all right you're going to the next stage you're you're more likely to not be kicked out or i'm not sure if it's 100 guarantee but that's going to have a lot of knock-on effects as well right it especially affects the data science space because if you're writing extensions through the c apis or rust right you've got to think more about thread safety because python all of a sudden can become concurrent like it didn't used to be able to? + +00:42:46.660 --> 00:42:50.500 +Yeah, it opens up a big door. + +00:42:50.540 --> 00:43:01.040 +I think in the initial PEP, one of their highlighted use cases was actually the problem that we were just talking about of how do you pipeline efficiently across multiple devices? + +00:43:01.700 --> 00:43:15.740 +So in the PyTorch case, you need to maybe spin up a bunch of different, at the moment, you need to spin up a bunch of Python processes in order to efficiently load data from disk into your PyTorch or your deep learning pipeline. + +00:43:16.100 --> 00:43:20.340 +Doing that with multiprocessing is not a wonderful world. + +00:43:20.940 --> 00:43:22.620 +And we could probably be a lot better. + +00:43:22.880 --> 00:43:25.200 +Free threading maybe opens up that door. + +00:43:25.420 --> 00:43:25.640 +Yeah. + +00:43:25.960 --> 00:43:29.000 +I also see it adding possible challenges, not just benefits. + +00:43:29.600 --> 00:43:45.400 +Like, for example, if I go and write a bunch of multi-threaded code that's truly multi-threaded, like C and C# and other languages have been for a while, pretty much forever, and I start interacting with it, Like, does that, you know, we talked about L2 cache and keeping it active. + +00:43:45.510 --> 00:43:46.680 +Like, what about the GPU? + +00:43:46.890 --> 00:43:55.520 +Like, does that potentially open up a case where the GPU gets loaded up with, you know, tons of data and gets dropped because a thread contact switch happened and like just it thrashes? + +00:43:57.120 --> 00:43:58.340 +It's possible that that could happen. + +00:43:59.180 --> 00:44:05.740 +Sorry, this line of questioning also opens the door for me to just briefly talk about like these larger systems that NVIDIA, + +00:44:06.010 --> 00:44:08.760 +but other folks have been building as well, like where you have coherent memory. + +00:44:09.460 --> 00:44:19.740 +Yeah, so in this new architecture, Grace Hopper or Grace Blackwell, there's a specific communication channel between device and host. + +00:44:20.360 --> 00:44:30.140 +It can, I think it's called chip-to-chip technology or NBLink C2C, and you can move data back and forth between device and host at around 900 gigabytes per second. + +00:44:30.740 --> 00:44:32.760 +That's basically free, right? + +00:44:33.260 --> 00:44:35.160 +Or sometimes it's fun to think about it. + +00:44:35.420 --> 00:44:38.160 +I'm pretty sure it's faster than my RAM on my Apple Silicon. + +00:44:38.640 --> 00:44:38.760 +Yeah. + +00:44:39.060 --> 00:44:51.920 +So thrashing is not good, but if you're for whatever reason in that scenario for a pipeline, you might not feel it in these new coherent memory systems. + +00:44:52.840 --> 00:44:53.440 +Yeah, that's wild. + +00:44:54.030 --> 00:45:00.460 +I also think you probably just, you know, it might be one of those things where it's like, doctor, my leg hurts when I bend it this way. + +00:45:00.490 --> 00:45:02.060 +And they said, well, don't bend it that way. + +00:45:02.240 --> 00:45:02.680 +You know what I mean? + +00:45:03.640 --> 00:45:08.320 +It hurts when I try to run like 10 concurrent jobs on the GPU on the same computer. + +00:45:08.520 --> 00:45:13.420 +well don't do that you know what i mean sure that that might be that might be the way like use a + +00:45:13.780 --> 00:45:23.200 +use a lock and don't let that stuff run then right yeah the answer for for probably these things is probably don't do it you shouldn't you shouldn't just absorb that problem dr hitters like no don't + +00:45:23.200 --> 00:45:58.540 +do that um but there are ways to scale up and i got all this comfort this is kind of what i was leaning towards is like there's interesting ways to scale up um across like multi gpu i know that Dask has an interesting interop story and Dask has super interesting grid computing ways to scale. So like Dask can kind of do pandas, but larger than memory on your machine. And it can take advantage of the multiple CPUs cores, and it can even scale out and across clusters. Right. And so there's some integration with Dask and other things. Do you want to talk about that side of the + +00:45:58.440 --> 00:46:05.240 +story? Yeah. Yeah. So maybe very briefly, Dask is trying to scale Python data science as well. + +00:46:06.440 --> 00:46:09.040 +So I think actually, if I can just get into a little bit of the history, + +00:46:09.740 --> 00:46:16.000 +there's lots of people, I think just before a bunch of people are importing library as PD, + +00:46:16.480 --> 00:47:45.040 +there's a lot of people, I think even historical people that have been exploring, how do I do distributed NumPy or distributed memory array applications, like both in the HPC world and also in the enterprise world. And they're rewriting a library from scratch. And Dask comes along with the idea of, well, I'll just take the... I'll build some block, I'll build some distributed version of NumPy, but still keep NumPy or still keep pandas as the central compute engine for what's happening. And I'll build orchestration mechanisms around that and build distributed algorithms around pandas or around NumPy. And there's a way for you to both scale out horizontally and also scale up because you could get pandas now as a multi-core thing and you could get numpy as this distributed scale out solution and much of the dask world actually evolved with with rapids as well in the last like five five years is last five years where because we were building a pandas like library in rapids like qdf we could get dasks to also do our scale out scale-out mechanisms. So we built some hooks for Desk to, or we tried to generalize what is a data frame. If it meets these things, I can use pandas, I can use QDF, I can use the next data frame library after that. We also built some specific hooks inside of Desk to take advantage of accelerated networking and making sure that GPU buffers got shipped around to all the different + +00:47:45.260 --> 00:48:03.280 +workers. That's pretty wild. So could I have, I guess I could probably have multiple GPUs on workstation, right? When I say that, I know you can have multiple GPUs and you can link them in hardware, but could I just literally plug in multiple GPUs and take advantage of them as well? + +00:48:03.680 --> 00:48:16.820 +Yeah. You can have, underneath my desk, I have a two GPU workstation that does have an NVLink bridge between the two, but you could also just have them work at many, many, many workstations, + +00:48:17.000 --> 00:48:21.240 +just have two GPUs plugged into the PCIe board. And things will work there as well. + +00:48:21.620 --> 00:48:45.560 +Yeah. There are some performance considerations there where if you want to move, if you're communicating data between those two devices, you pay a bit of a cost. You have to move data from device to host. You then have to serialize it across the network and then move data from host to device. This is why having that NVLink bridge is so powerful if you have it in your system. + +00:48:45.660 --> 00:48:57.360 +Okay. So if people are already using Dask, how easy is it to adopt the setup? Or do you even have to think about it? Is this Dask storage just underneath the covers of the API? + +00:48:57.720 --> 00:49:04.140 +If you're already using Dask, you can already use DaskQDF or DaskKupai. Those things work. + +00:49:06.020 --> 00:49:13.000 +I've done some experiments and I've not had as much success, but people are still pushing quite hard as using Dask as a third-party library. + +00:49:13.460 --> 00:49:17.020 +So how do I make a GPU version of X-Ray? + +00:49:17.520 --> 00:49:19.100 +Well, that actually takes a bit more work. + +00:49:19.160 --> 00:49:21.280 +And there are people that are pushing quite hard, as I was saying before. + +00:49:21.920 --> 00:49:30.580 +But X-Ray, at least when I attempted it like three or four years ago, has a lot of mixture of DESK, of Pandas calls, of NumPy calls. + +00:49:30.630 --> 00:49:41.000 +And it was hard, at least in my attempt, to perfectly articulate all the GPU mechanisms that needed to be satisfied to make it work seamlessly or get any performance benefit. + +00:49:41.140 --> 00:49:43.280 +But I'm not as up to date on it. + +00:49:43.360 --> 00:49:45.520 +Maybe there's been some recent developments there. + +00:49:45.800 --> 00:49:47.680 +Yeah, there's a lot of moving parts. + +00:49:48.480 --> 00:49:49.160 +And they're all moving. + +00:49:49.240 --> 00:49:53.660 +And what used to be impossible and now is no problem. + +00:49:53.980 --> 00:49:56.260 +But you haven't tested that combination, right? + +00:49:56.540 --> 00:49:56.720 +Yeah. + +00:49:57.040 --> 00:50:02.120 +I'm very encouraging of anybody who wants to work on that problem or explore that space. + +00:50:02.120 --> 00:50:13.180 +I think geospatial, geoscience things definitely need all the attention they can get in an ever-changing climate world, climate science kind of problems that we are all experiencing as humans. + +00:50:13.600 --> 00:50:14.340 +Yeah, absolutely. + +00:50:14.900 --> 00:50:21.800 +So we talked earlier about the challenge of staying in touch with all these different APIs and staying consistent with them. + +00:50:23.120 --> 00:50:25.280 +Do you have really interesting test cases? + +00:50:25.820 --> 00:50:39.860 +Do you have some mother-of-all-py test execution sort of thing where do you maybe take the pandas unit tests and try to run them on QDF and similarly with scikit-learn and so on? + +00:50:39.900 --> 00:51:01.740 +Yeah, that should have been my first answer now that I think about it. That's exactly what we do do. For Kudyat Pandas, we run the... So for the Kudyat Pandas product, we do run the Pandas unit test. And we see... That's the goal, to have this run perfectly across it. It's not necessarily to accelerate all the APIs, but making sure that we never fail. + +00:51:02.420 --> 00:51:04.480 +They fall back to CPU if they have to. + +00:51:04.520 --> 00:51:06.840 +They fall back to CPU if they have to, exactly. + +00:51:07.300 --> 00:51:10.380 +We're also recording where we aren't using the GPU. + +00:51:10.530 --> 00:51:15.820 +So it gives us some directional information about the kinds of things that aren't accelerated. + +00:51:15.910 --> 00:51:23.640 +So maybe there's some niche datetime features or some niche extension D-type things that we aren't handling or can't be handled. + +00:51:24.400 --> 00:51:27.680 +And the same thing is also true for scikit-learn and QML. + +00:51:28.210 --> 00:51:35.660 +I think there are some known, actually at the moment, there are some known failures for for QML and SecondLearn. But we do, that is like the easiest thing that we could do. And we, + +00:51:35.800 --> 00:51:47.580 +we do that. Yeah. And are they, they're pretty much all running. I know you talked a little bit about QML and stuff, but how much pytest.ignore is in there? There's not as much as you, + +00:51:47.860 --> 00:52:06.740 +I think for KUDIF pandas, we are at like 99.x% passing for, for it. For QML, I have to look it up. But I think we still have like pretty, pretty good coverage of, of the entire scikit-learn code base in terms of not a falling back correctly, not necessarily accelerating it. There's a lot of, + +00:52:07.000 --> 00:52:10.420 +there's a lot of classifiers there. Okay. Yeah. I'm sure there are. + +00:52:11.040 --> 00:52:23.640 +That might be interesting. I don't know if this is like documented somewhere or whatever, but that might be interesting as a way for people who are considering adopting it to go like, well, let's see what the failing or ignored tests are like, no, these don't seem to apply to me. + +00:52:23.760 --> 00:52:44.220 +we're probably okay i think actually on the kumel documentation page there's a known limitation section that outlines the kinds of estimators that are not are not accelerated on some edge cases that are not supported at the moment but they're working again the team is quite motivated to like keep on as you've mentioned before it's an ever-changing world we're not we're + +00:52:44.220 --> 00:52:56.580 +going to keep on working on these problems my now admittedly pretty old geforce card in my gaming computer I got in 2020. I don't know how much RAM it has, four gigs, eight gigs, something like that. + +00:52:56.920 --> 00:53:00.600 +But I know there are data problems that are certainly bigger than four or eight gigs of data. + +00:53:01.120 --> 00:53:07.640 +What happens if I try to read CSV and the CSV is 12 gigs and I've got eight gigs of available? + +00:53:07.880 --> 00:53:24.540 +I love that you asked this question because it's been a focus of our group for like the last year and a half. A lot of very fun engineering problems have to be solved when you want to do of core processing and there's a lot of tools that we can deploy to solve this problem. + +00:53:25.850 --> 00:53:35.020 +So for single GPU, there already is a solution that was available to us that we needed to improve upon, but still could largely just deploy. + +00:53:35.210 --> 00:53:37.800 +And that's a CUDA memory type. + +00:53:38.100 --> 00:53:41.060 +CUDA has a bunch of different kinds of memory that you can use. + +00:53:41.510 --> 00:53:46.200 +So there's not just CUDA malloc, but there's an asynchronous malloc. + +00:53:46.380 --> 00:53:53.440 +And there's also a larger pool that you can build and then pull memory from this larger pool to help speed things up. + +00:53:53.940 --> 00:53:57.860 +There's also something called UVM or Unified Virtual Memory. + +00:53:58.420 --> 00:54:05.080 +And in this case, the driver, the CUDA driver itself, is going to try and allocate some memory on the GPU. + +00:54:05.740 --> 00:54:07.400 +And if it can't, it will spill. + +00:54:07.470 --> 00:54:10.780 +It will page data from the GPU onto the CPU. + +00:54:11.380 --> 00:54:13.320 +And the driver just takes care of all of that for me. + +00:54:13.780 --> 00:54:25.940 +So if you have a 12 gigabyte data set and you're trying to read it into a car that only has eight gigabytes, you could probably get by with just UVM. + +00:54:26.940 --> 00:54:30.580 +A question that you still should ask yourself is whether you received any performance benefit. + +00:54:30.750 --> 00:54:32.360 +I really want to be very clear. + +00:54:32.600 --> 00:54:37.100 +If you're trying to do something on CPU, and it's faster and everything just works, you should stay there. + +00:54:37.210 --> 00:54:38.340 +You shouldn't bend over backwards. + +00:54:38.560 --> 00:54:42.160 +We're really working hard to make sure that you get a benefit from using these devices. + +00:54:42.960 --> 00:54:45.100 +The other thing that you can do is batch. + +00:54:45.700 --> 00:54:49.560 +So there's this exotic memory type that is like the default. + +00:54:49.980 --> 00:54:55.140 +Most users don't ever have to think about it or worry about it, especially in the case of QDF pandas. + +00:54:56.380 --> 00:54:59.980 +But the other thing that you can do is sip data out. + +00:55:00.120 --> 00:55:01.200 +You can batch it. + +00:55:01.400 --> 00:55:05.840 +So for pandas, that's a lot of work to write a lazy framework on top of it. + +00:55:06.100 --> 00:55:08.260 +But for polars, that already exists. + +00:55:08.980 --> 00:55:11.560 +I was thinking that's the default of kind of how polars works. + +00:55:12.160 --> 00:55:12.240 +Right. + +00:55:12.640 --> 00:55:22.320 +So we already have this mechanism to push to higher than memory, larger than memory limits, or do this out of core kind of processing because it's native to the Polars experience. + +00:55:22.900 --> 00:55:28.580 +I think we've also been seeing that more with some of our machine learning algorithms. + +00:55:29.220 --> 00:55:58.940 +So if you look at the 3.0 release of XGBoost has this external memory allocator where you can use host memory to batch data in before you start processing on GPU. And the same thing is also true for UMAP as well, where you can use host memory to store data temporarily as you start processing it on GPU. And it allows you to really push much higher than what this resource-constrained + +00:55:59.120 --> 00:56:21.580 +GPU environment is. If I want to run this stuff in the cloud, what are some good options? Do I go to DigitalOcean and just pick a GPU-enabled droplet? Or is there like super heavy-duty things I can get? Or maybe you're like me and you have a Mac Mini, but you want to play with stuff and you don't have an NVIDIA GPU on this computer. Yeah. Unfortunately for the Mac Mini, + +00:56:22.020 --> 00:56:39.760 +there aren't a lot of options. While all this is open source and the code can be read and can be contributed from anybody, these only work on NVIDIA hardware. So yeah, you can go get a a Droplet, spin up a Docker image. + +00:56:39.860 --> 00:56:44.760 +I think we not only have pip and kind of packages, but also have Docker containers. + +00:56:45.740 --> 00:56:46.320 +You can get AWS. + +00:56:46.870 --> 00:56:58.820 +You can get, I think, on this page, maybe not the page that you're on, but there's a deploy page where we have a lot of recommendations about how to deploy rapids in a variety of environments. + +00:56:59.780 --> 00:57:02.680 +Some things like some of the MLOps tools like SageMaker. + +00:57:03.080 --> 00:57:03.780 +Yeah, that's the page. + +00:57:04.880 --> 00:57:09.020 +I probably could do something like Google CoLab or something like that in my MacBan, right? + +00:57:10.920 --> 00:57:13.640 +Actually, that's the best place to get started. + +00:57:13.860 --> 00:57:15.180 +That's where we direct a lot of our users. + +00:57:16.520 --> 00:57:20.180 +Google CoLab has a free tier offering where you can just select a different backend. + +00:57:20.680 --> 00:57:26.140 +I think for Kaggle users, there's now even like L4s or multi-GPU L4s that they can get access to. + +00:57:26.420 --> 00:57:26.680 +Nice. + +00:57:27.740 --> 00:57:27.900 +Okay. + +00:57:28.320 --> 00:57:34.820 +And then one other thing before we wrap it up is, let's see, I just go to Rapids. + +00:57:34.860 --> 00:57:44.120 +I know at the beginning here, it says you got the new Polars GPU engine, what we talked about, and it's pre-installed on Google Colab and things like that. + +00:57:44.120 --> 00:57:49.340 +But you also have vector search now with QVS, which is what I'm going with unless I'm told otherwise. + +00:57:49.680 --> 00:58:08.200 +But if you're not using AI, you're not using LLMs, but you're literally building LLMs or you're augmenting LLMs, this vector search stuff and embeddings and whatnot is what you need. So I know you're not a super expert in vector embeddings and stuff, and neither am I, but maybe tell people quick about this + +00:58:08.640 --> 00:58:29.780 +library, this aspect. Yeah, this actually grows out of the QVS, or QD Accelerated Vector Search, grows out of the ML world that you have. In the ML world, for UMAP or clustering algorithms, You need neighborhood algorithms to help you do regular data science. + +00:58:30.060 --> 00:58:44.280 +And it turns out that taking a bunch of strings and doing math on them, which is what a vector, which is what an embedding is, I have some text, I need to do some math on it, and then I need to understand how that text relates to other things with even more math. + +00:58:44.430 --> 00:58:49.780 +It'd be something like cosine distance or Jaccard similarity or minhash, whatever it is. + +00:58:50.400 --> 00:58:53.740 +And I need to do that across a very large corpus of text. + +00:58:54.420 --> 00:59:02.560 +So vector search and vector rag become this go-to tool for the LLM space, which is, as I was just saying, a lot of math on text. + +00:59:03.180 --> 00:59:05.300 +How do I make that go faster? + +00:59:05.350 --> 00:59:06.380 +How do I build an index? + +00:59:06.600 --> 00:59:11.340 +How do I re-index faster and faster and faster as the corpus changes or it has an update? + +00:59:11.700 --> 00:59:14.000 +Yeah, you think geospatial is an interesting query. + +00:59:14.700 --> 00:59:18.840 +Like the number of dimensions of this kind of stuff is unimaginable, right? + +00:59:19.200 --> 00:59:19.500 +That's right. + +00:59:19.740 --> 00:59:22.620 +I don't know that as many day-to-day, + +00:59:22.740 --> 00:59:31.000 +like if you're in the LLM space, LLM building space or in the retrieval space, QVS is definitely going to be something you should take a look at. + +00:59:31.260 --> 00:59:35.400 +For the rest of the broader data science community, I don't think QVS is as relevant to that + +00:59:35.680 --> 00:59:37.240 +just as a small clarification. + +00:59:37.660 --> 00:59:42.920 +Right, like how many people are using vector databases that are not using LLMs? + +00:59:43.040 --> 00:59:43.640 +Probably not too many. + +00:59:44.080 --> 00:59:45.100 +Right, probably not too many. + +00:59:45.480 --> 00:59:45.580 +Yeah. + +00:59:46.320 --> 00:59:48.920 +All right, let's wrap this up with one final question. + +00:59:49.060 --> 00:59:51.220 +where do you go from here? What's next? + +00:59:51.500 --> 00:59:52.900 +What's next for Rapids? + +00:59:53.100 --> 00:59:57.340 +Yeah, Rapids and basically Python plus NVIDIA. + +00:59:57.740 --> 01:00:17.340 +I think what's next for Rapids is it's always going to be some bit of maintenance of what we currently have pushing more and more performance and then trying to always encourage people to try out what we have and actually deliver something that is ultimately providing value for them. + +01:00:18.460 --> 01:00:23.020 +I think that there's a lot of really cool things that are happening on the hardware side, as I mentioned before. + +01:00:23.130 --> 01:00:31.520 +Like Blackwell has some pretty cool things that I don't know that the users will feel, but I don't know that they're going to have to interact with. + +01:00:31.680 --> 01:00:35.760 +So there's this very fancy decompression inside of Blackwell. + +01:00:36.180 --> 01:00:45.480 +There's also, again, as I mentioned before, in this coherent memory world where there's just memory or can I treat the system as just memory, how does software look? + +01:00:45.920 --> 01:01:46.020 +If I want to, like, we're posing these questions to ourselves, but if there was just malloc in the world and that happened seamlessly between host and device in it, what kind of software would I be running? How would I try and architect that code? I think those pose a lot of, like, very interesting computer science and also computer engineering questions. I think for, like, NVIDIA and Python, as the guest that you had on before, Bryce Ledback was describing, I think exposing all these ideas to Python developers is really exciting. They might not maybe move into the C++ world, but I think a lot of Python developers want to understand how this device works and how they can manipulate it through their language of choice. We've seen that as we were actually just describing earlier, like I want to get access to the different cache levels, but we see Python core developers made that available to us. That's really wonderful. The whole world isn't just vector computing. I want to take advantage of the entire system. So I think a lot of it is going to be exposure, education, + +01:01:46.520 --> 01:01:51.160 +and more and more performance. Awesome. Any concrete releases coming up that + +01:01:51.500 --> 01:02:43.840 +people should know about? Rapins does releases every two months. So you can see some announcements of what we're planning in the next release. I think that we're coming up 2508 should be baking right now. And then we'll have 2510 and 2512, where we just announced a few months ago a multi-GP Polarist experience. You can scale, not just have scale up, but scale out with the Polarist front end. We're doing a lot of good work around Kuml XL, as I was mentioning before, getting more algorithms, trying to push to higher and higher data sets that UMAP works with. And we're also looking at, we're trying to spend some time looking at particular verticals, I think, especially in like the bioinformatic space as well. But, you know, really excited to hear from anybody if they have a problem that they need, that they need more power, more performance, you know, please, + +01:02:44.360 --> 01:03:00.960 +please raise your hand and come talk to us. Awesome. Yeah, I'll put your, some link to you somehow in the show notes that people can reach out. But you mentioned Slack earlier, is there Slack Discord? What are the ways? Yeah, there's a couple Slack channels. One of them is + +01:03:00.980 --> 01:03:04.780 +called GoAI. There's a CUDA Slack channel. + +01:03:05.320 --> 01:03:14.800 +And then there's also a community-based GPU mode Slack channel that's a little bit more deep learning oriented. But the GoAI Slack channel is something that is specific to + +01:03:15.100 --> 01:03:16.340 +Rapids. Yeah. Awesome. + +01:03:16.820 --> 01:03:17.840 +Ben, this has been really fun. + +01:03:19.360 --> 01:03:21.000 +People out there listening, maybe they want to get started. + +01:03:21.040 --> 01:03:22.880 +What do you tell them? What do they do to try out + +01:03:22.980 --> 01:03:30.780 +Rapids? Go to CoLab and just import CUDA Pandas. Import Kumel XL. Import NXCUDRAF. + +01:03:31.020 --> 01:03:32.220 +Everything is baked in. + +01:03:32.230 --> 01:03:36.220 +We worked really hard with Google to make sure that that environment was set up from the get-go. + +01:03:36.380 --> 01:03:36.620 +Awesome. + +01:03:37.040 --> 01:03:37.220 +All right. + +01:03:37.540 --> 01:03:40.940 +Well, it's been great to chat GPUs and data science with you. + +01:03:41.050 --> 01:03:41.620 +Thanks for being on the show. + +01:03:41.880 --> 01:03:42.760 +Thanks so much for having me. + +01:03:42.950 --> 01:03:43.480 +I really appreciate it. + +01:03:43.520 --> 01:03:44.080 +Yeah, you bet. + +01:03:44.300 --> 01:03:44.420 +Bye-bye. + +01:03:45.360 --> 01:03:47.920 +This has been another episode of Talk Python To Me. + +01:03:48.740 --> 01:03:49.760 +Thank you to our sponsors. + +01:03:50.170 --> 01:03:51.380 +Be sure to check out what they're offering. + +01:03:51.600 --> 01:03:52.820 +It really helps support the show. + +01:03:53.340 --> 01:03:56.280 +This episode is sponsored by Posit and Posit Workbench. + +01:03:57.080 --> 01:04:04.260 +Posit Workbench allows data scientists to code in Python within their preferred environment without any additional strain on IT. + +01:04:05.020 --> 01:04:14.540 +It gives data scientists access to all the development environments they love, including Jupyter Notebooks, JupyterLab, Positron, and VS Code, and helps ensure reproducibility and consistency. + +01:04:14.940 --> 01:04:19.120 +If you work on a data science team where consistency matters, check out Posit Workbench. + +01:04:19.760 --> 01:04:22.200 +Visit talkpython.fm/workbench for details. + +01:04:22.740 --> 01:04:23.600 +Want to level up your Python? + +01:04:23.900 --> 01:04:27.720 +We have one of the largest catalogs of Python video courses over at Talk Python. + +01:04:28.180 --> 01:04:32.880 +Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:04:33.260 --> 01:04:35.400 +And best of all, there's not a subscription in sight. + +01:04:35.900 --> 01:04:38.420 +Check it out for yourself at training.talkpython.fm. + +01:04:39.100 --> 01:04:43.280 +Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:04:43.720 --> 01:04:44.620 +We should be right at the top. + +01:04:45.120 --> 01:04:53.980 +You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +01:04:54.640 --> 01:04:56.880 +We're live streaming most of our recordings these days. + +01:04:57.210 --> 01:05:04.720 +If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:05:05.740 --> 01:05:06.860 +This is your host, Michael Kennedy. + +01:05:07.280 --> 01:05:08.120 +Thanks so much for listening. + +01:05:08.310 --> 01:05:09.260 +I really appreciate it. + +01:05:09.530 --> 01:05:11.220 +Now get out there and write some Python code. + +01:05:28.620 --> 01:05:31.420 +*music* + From edfc74ff7b3bbeeff9500dcf485bca1969b96236 Mon Sep 17 00:00:00 2001 From: Michael Kennedy Date: Tue, 19 Aug 2025 14:58:18 -0700 Subject: [PATCH 4/5] transcripts --- transcripts/517-agentic-ai.txt | 1520 +++++++++++++++++++++ transcripts/517-agentic-ai.vtt | 2312 ++++++++++++++++++++++++++++++++ 2 files changed, 3832 insertions(+) create mode 100644 transcripts/517-agentic-ai.txt create mode 100644 transcripts/517-agentic-ai.vtt diff --git a/transcripts/517-agentic-ai.txt b/transcripts/517-agentic-ai.txt new file mode 100644 index 0000000..5d6d222 --- /dev/null +++ b/transcripts/517-agentic-ai.txt @@ -0,0 +1,1520 @@ +00:00:00 Agentic AI programming is what happens when coding assistants stop acting like autocomplete and start collaborating on real work. In this episode, we cut through the hype and incentives to define agentic, then get hands-on with how tools like Cursor, CloudCode, and Langchain actually behave inside an established code base. Our guest, Matt McKay, now the Vice President of Developer Relations at DigitalOcean, the creator of FullStackPython, and PlushCap, shares hard-won tactics. + +00:00:29 We unpack what breaks from brittle, generate a bunch of test requests to agents amplifying technical debt and uneven design patterns. + +00:00:38 Plus, we also discuss a sane Git workflow for AI-sized diffs. + +00:00:43 You'll hear practical clawed tips, why developers write more bugs when they get less time programming, and where open source agents are headed. + +00:00:52 Hint, the destination is humans as editors of systems, not just typists of code. + +00:00:58 This is Talk Python To Me, episode 517, recorded July 29th, 2025. + +00:01:20 Welcome to Talk Python To Me, a weekly podcast on Python. + +00:01:23 This is your host, Michael Kennedy. + +00:01:25 Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is sponsored by Posit Connect from the makers of Shiny. Publish, share, and deploy all of your data projects that you're creating using Python. + +00:01:58 Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards, and APIs. + +00:02:05 Posit Connect supports all of them. Try Posit Connect for free by going to talkpython.fm/posit, P-O-S-I-T. Matt, great to have you back on the show. Welcome back to Talk Python. + +00:02:17 Thanks, Michael. Been a little while. Good to be back. + +00:02:19 It has been a little while. + +00:02:21 As I was entering your show details into the backend system for Talk Python to say what episodes are coming up in what order and so on, I have to enter the guest ID into that thing. + +00:02:33 Yes, it could be a drop-down list, but then there's so much scrolling. + +00:02:35 I just can't do it. + +00:02:37 And it's multi-select. + +00:02:38 So your guest ID is 29. + +00:02:41 We first did an episode together about Fullstack Python when I was sitting on the couch in Stuttgart, Germany, back when I lived there. + +00:02:49 How about that? + +00:02:50 Wow. + +00:02:50 All right. + +00:02:51 Yeah, it's been a while. + +00:02:52 I feel like that's like that credibility. + +00:02:54 You have that like only a double digit guest ID. + +00:02:59 Exactly. + +00:02:59 That's a low double digits. + +00:03:01 Yes. + +00:03:02 This is like a pro number in racing or something. + +00:03:05 Indeed. + +00:03:06 Well, that was the past. + +00:03:08 We talked about some awesome things then. + +00:03:11 The idea of full stack Python, I believe, was a lot of the focus, but it's honestly been 10 years. + +00:03:16 I don't remember exactly what we covered in detail. + +00:03:19 However, I can tell you, this is definitely not a backward looking episode. + +00:03:24 This is cutting edge stuff. + +00:03:25 Yeah. + +00:03:26 I mean, so much of this has changed over even three months. + +00:03:30 I mean, we'll talk about it all, but there's like the philosophy behind all this, the holy religious wars around what developers should be doing with it. + +00:03:40 And then let's dig into the details because that's the fun stuff. + +00:03:44 Absolutely. + +00:03:45 Absolutely. + +00:03:46 I would say it's certainly one of the most controversial topics in dev space these days. + +00:03:52 Well, it's almost like, if you remember when Django and Ruby on Rails came out and they were the new frameworks, even before all the JavaScript frameworks and everything like that, it was backend server, server-side code, and you could do so much more. + +00:04:05 I remember this was back when I was an early professional developer. + +00:04:08 I was working in Java with servlets in a framework called Tapestry and everything was so hard to do. + +00:04:15 And they tried to add so many layers of abstraction. + +00:04:17 It was like peak enterprise, Java development. + +00:04:20 How many layers of dependency injection were at play? + +00:04:22 Oh, yeah. + +00:04:23 And so then I would go home at night and I'm like learning Django. + +00:04:26 And I was like, I can't believe how much more I can get done in an hour than I did eight, nine, 10 hours a day working in the Java code. + +00:04:34 Not because there's literally nothing wrong with Java, but it was just that the frameworks were so well built for what we were trying to do. + +00:04:43 So that's the only parallel I personally have to like what is happening right now, which, and this is like 10X, 100X of what that is. + +00:04:50 It's like, if you use the tools properly for the right purposes at the right time period, because like a year ago versus today is very different, you can be wildly more productive. + +00:05:03 There's also the downsides, but there is substantial improvement to be had in certain areas of software development. + +00:05:09 And I feel like that is actually really the big takeaway. + +00:05:15 Among all the hype, break everything out. + +00:05:16 It's like, there's tactical things you can use this, agentic tools for LLMs that will 10X, 100X certain things that are super annoying about software development right now + +00:05:26 in a way that was previously impossible. + +00:05:28 Right. Summarize my architecture. + +00:05:30 Have you tried that compared to like, I have a hundred thousand lines of code and I've got to like study the crap out of it just to figure out what piece connects to what. + +00:05:38 I love the roast my code. + +00:05:40 Like, don't just tell me about it. + +00:05:43 Like, give it to me straight here. + +00:05:45 Like, how could this function be improved? + +00:05:48 Roast it? + +00:05:49 Okay, now tell me how I can improve it. + +00:05:51 Okay. + +00:05:53 And at the end, just give me a little bit of like ego boost because I need it after all the roasting. + +00:05:58 Yes, exactly. + +00:05:59 Well, it's like, okay, you roast me. + +00:06:01 Anytime you talk me down, Just include a little spirit lifting comment as well. + +00:06:06 Well, the best part is like, okay, roast me. + +00:06:09 Okay, if you're so smart, go fix it for me. + +00:06:11 Yeah, you're like, oh, that's right. + +00:06:13 It did work. + +00:06:14 Sometimes it does actually. + +00:06:15 Sometimes. + +00:06:16 Which is insane. + +00:06:17 Yeah, I think probably the best way I could say we should frame, people should frame this conversation to at least to get started with in their mind is a balanced take, right? + +00:06:28 It's not necessarily vibe coding. + +00:06:31 while that's hilarious and has spawned many amazing YouTube videos, right? + +00:06:37 But it's also, it doesn't necessarily make sense to imagine that these tools don't exist. + +00:06:43 And just out of principle say, well, I'm never going to touch them because I'm not going to replace coders with this. + +00:06:49 It's like saying, I was just playing a bunch with rough, you know, the formatting tool, right? + +00:06:54 And it's ripping through 500 files at a time on these projects I'm working on. + +00:06:59 Ruff format, rough check fix, bam, 37 files repaired. + +00:07:04 I could have done that by hand and flexed my dev skills, but should I? + +00:07:09 No, of course I shouldn't. + +00:07:10 It takes microseconds. + +00:07:10 Would that be the best use of your time? + +00:07:12 It certainly does not. + +00:07:14 And so if these tools exist, well, then I think it comes down to like, well, what is the right way to use them? + +00:07:19 How do they go well? + +00:07:20 How do they go poorly? + +00:07:21 Yeah. + +00:07:21 Are they too expensive? + +00:07:23 That's spot on. + +00:07:23 I mean, what I find fascinating about this shift is like most professional developers have spent their entire adult lives just like learning new technologies. + +00:07:35 I actually don't know why this is any different. + +00:07:37 Like the whole point of being a developer is like keeping an open mind, an open philosophy of like, what can I learn? + +00:07:45 How should I be learning? + +00:07:47 Just that is what appeals to me so much about being a software developer is like this mindset of I go into this year not even knowing what tools I may be using by the end of the year because they may not exist yet. + +00:08:00 And I want to go learn them. And some of them I am going to throw immediately in the wastebasket + +00:08:05 and other ones are going to stick with me for potentially decades. And in fact, when it comes to LLMs, I had put out a post on X and I was like, I feel like Olama is one of those tools that I've added to my tool belt, like Django or Vim or Teamux that will be with me potentially for a couple of decades because it is just so straightforward and it makes it so easy to use these open-weighted models. And so that's incredible. 18 months ago, I hadn't even heard of this tool. And actually, it didn't exist before, I don't know, 18, 24 months ago. + +00:08:40 And here's something that I'm like, wow, this is going to stick with me. So I think that's what's maybe getting lost in a little bit of the hype cycle is we're developers. All we do is learn new stuff. Why is this any different from learning an open source project that you just found on + +00:08:54 GitHub? A hundred percent. A hundred percent. I'm definitely here with you on that. I'd see in other areas like education, I see it, you know, like write this essay for me. That's very problematic if you're in 10th grade and you're supposed to learn how to write an essay, but your job is to create software that works, add features, make it awesome. Like there's not a test. Yeah. + +00:09:15 Other than shipping. + +00:09:45 you if you're not a senior enough developer you're not experienced enough or you're not willing to dig deep enough you suddenly are stopped there's nothing you can do from there you're like no fix + +00:09:54 it fix it it doesn't matter how many times you tell the lom to fix it or the tool to fix it please + +00:09:58 or you go to jail yeah exactly yes there's people on the train tracks you must fix this right now + +00:10:05 my grandma's life depends upon it right you gotta make this database query work yeah well + +00:10:11 I think one thing that maybe is preventing people from embracing some of this stuff is, I actually don't even think that a lot of the terminology is clear. So if you'll allow me to be pedantic for a minute, I actually think that this is often the most insightful thing that I work with people on or tell people about, which is just like, what is the difference between an AI model, an LLM, an AI agent, and some of these other things? Because actually, people use these interchangeably. + +00:10:38 People will say Gen AI, generative AI, when they mean an agent or vice versa or an LLM. + +00:10:44 These are not drop and replace terminology for each other. + +00:10:48 They have very specific meanings. + +00:10:50 And I think particularly when I've seen the traditional software companies try to all of a sudden AI all the things, this is part of why developers get so annoyed with the AI ecosystem. + +00:11:02 Because it's like saying Django is not a... + +00:11:07 You wouldn't say web framework. + +00:11:08 you would say like, oh, it's an AI agent coding tool. + +00:11:11 It's like, no, it's not. + +00:11:12 What are you talking about? + +00:11:12 Immediately you lose your credibility, right? + +00:11:14 It's an AI generative web framework builder. + +00:11:20 Right, right. + +00:11:20 Thank goodness it's not. + +00:11:21 Thank goodness Django, the creators and the maintainers know what they're good at. + +00:11:26 And they're not trying to be a part of the hype cycle. + +00:11:30 Ironically, I think Simon Wilson, one of the original Django folks, is one of the leading people in AI. + +00:11:38 I've learned so much from his writing. + +00:11:39 Yeah, absolutely. + +00:11:40 It's unreal. + +00:11:40 It's amazing. + +00:11:41 I didn't think that, go ahead. + +00:11:42 Yeah. + +00:11:43 I don't know how, I don't know how, I don't understand how he can be so prolific. + +00:11:46 He is incredible. + +00:11:47 Like he's, he's just like a true gift, to software development, having been on the leading edge of so many things. + +00:11:54 It's amazing. + +00:11:55 Yeah, absolutely. + +00:11:56 So if I could just lay out real quick, maybe for, for folks like a, an AI model was an AI model is typically when people are talking about that. + +00:12:04 Now, again, like there can be a little bit of nuance and gray areas with some of these definitions, but typically an AI model is something that is trained on some sort of training set. + +00:12:13 And it's been typically trained on the transformer architecture, which is a fairly recent, last eight years kind of development. And so this is really what's allowed the breakthrough when we talk about AI now versus AI a couple of decades ago when it was like complete AI winter and no one wanted to talk about it, is we've had this breakthrough in architecture and it's not just like one breakthrough. It is a tremendous number of breakthroughs around attention and the way that things are weighted and things like that. But essentially, to boil that down, you have these AI models. Now, an AI model is not equivalent to a large language model, an LLM. An LLM is one type of AI model. So when I was at Assembly AI, they were training or they are training state-of-the-art speech-to-text models is not an LLM. So a lot of times people will say like AI model LLM, but those are not equivalent. An AI model is a superset of the subset, which is like an LLM, which is one type of typically working on text modality. Although there are things called multimodal models, which could have different image inputs and image outputs and text inputs, text outputs, and stuff like that. But I think that's one thing where a lot of companies and maybe even developers who are learning about the space get confused. It's like, is an AI model and LLM are the exact same thing? No, there's a relationship there, but they're not the same. So then you have like generative AI, which a lot of companies just kind of like sprinkle that into everything that they talk about. It's like generative AI is really using AI models, typically LLMs, but also image generation and some other forms to create some sort of output. So it's the generative component of it. So you have some sort of input and then there's a non-deterministic set of outputs to come out the other side. + +00:13:52 So it'll be, for example, like draw me a red dragon breathing fire for the image generation. + +00:13:57 And that generative AI is basically using an AI model to produce that image out the other side. + +00:14:04 So those are some of the common terms. + +00:14:06 And then you have AI agents, which is a lot of what we talk about or we're going to talk about, which is it is using typically an LLM, but it's typically using some sort of AI model. + +00:14:16 And that is kind of almost think about it as like the core. + +00:14:19 There's inputs into the system and non-deterministic outputs that come out the other side. + +00:14:22 So you'll say something like, write me a bunch of unit tests and in Cloud Code or in Cursor or in Windsurf, and then it will interpret those inputs and then produce code or some sort of output out the other side. + +00:14:35 So I think for developers who are trying to get into, how do I even understand kind of the AI space? + +00:14:40 It's actually really important to get that terminology correct, because otherwise you won't even know when you're reading, particularly for companies or people that aren't as familiar with the space, like what they're even talking about. + +00:14:53 So I always like to kind of like- A lot of the companies, yeah, their job, it's to their benefit to obscure + +00:15:00 and make it just seem like it's everything. + +00:15:01 Yeah, they want to seem like- + +00:15:03 Yeah, totally. + +00:15:05 There's financial incentives by some companies to like obscure what they're doing and make it seem much more complicated so that they can sell this solution. + +00:15:13 Oh, it takes this really complicated problem and streamlines it down to like some sort of simple solution. + +00:15:17 And that's like often not the case. + +00:15:18 Like when you peel it back as a developer, like, you're not really doing that. + +00:15:22 Right. + +00:15:23 So I think that's often where I think a lot of developers would get frustrated with the current state of the industry because you're like, no, it's not what you're doing. + +00:15:32 You know, like you're not, you're not doing generative AI. + +00:15:34 You're not doing actual agents because agents are, you know, and then there's like autonomous agents, which are like operating independently. + +00:15:40 So that's one thing I think developers can like take away from the conversation is just like, is the company accurately describing what they are doing? + +00:15:49 Yeah, I a hundred percent agree. And let me do a little sidebar rant and I'd love to get your + +00:15:55 thoughts on this. Okay. Yeah. So when I hear people say, I've tried AI, it's a bunch of AI + +00:16:02 slop. It just makes up a bunch of mistakes. I noticed a couple of things that are often the case when I hear those and have people avoid that, right? So one, a lot of times I feel like people are not getting the best experience. They say like, I tried this. It really wasn't for me. + +00:16:20 It just messes up more than it provides. They're using the cheapest free models that they can find, right? If you use, you know, Cloud's Opus model versus some free model, you know, like a 3 billion parameter local model they're not even in the same category the type of accuracy and like insight and like the context do they they understand everything rather than well they only understand the last file they read like that that is like one half of the problem and then i think the other for people who are not being successful yet with this kind of stuff it has to do with not providing enough information and context and stuff in the prompt so i'll see like oh refactor this function it's like well, hold on. What, where, where do you even want it to go? Right. It's just going to like, start randomly doing stuff. Here's a function. It is similar to these. And I need to move this to this kind of design pattern, keeping in mind that I'm using this ORM and like, give it like the same amount of description you would to a junior developer who's not super familiar with your project. So you want to give that thing like a possibly a multi-page write-up of what it needs to do and then ask it to plan it out and then start working through the plan, not just refactor + +00:17:36 to be, you know, to do this or whatever. + +00:17:39 And I think those two things are both problems. + +00:17:42 They often go together because the cheaper models can't accept that kind of information and keep it working. + +00:17:48 Right. + +00:17:48 Yes. + +00:17:49 What do you think about that? + +00:17:50 I mean, I totally agree. + +00:17:51 I think also to like these models, you cannot just like mix and match and replace things. + +00:17:56 So you may have an experience with one model and also recognize that these models, even though Claude for Opus is the public name of that model, they are tweaking and tuning that model on the backend to figure out how they can more, maybe not profitably, but serve this model up at scale in a window of resources. + +00:18:20 So even within a given model, unless it is an open weighted model that you've downloaded, like you have under complete control, these models are changing over time. + +00:18:29 So this is like, I think that's actually been one of the most concerning parts for me as a developer. + +00:18:35 I was like, what if I rely on a tool and it changes? + +00:18:39 And I wake up one day. + +00:18:40 The tests passed last week. + +00:18:41 They don't pass this week. + +00:18:42 Right. + +00:18:42 I have no control or visibility. + +00:18:43 Right. + +00:18:44 I was productive last week or I was productive yesterday and I can no longer be productive today. + +00:18:49 And that's setting aside any downtime or API stuff, right? + +00:18:52 So I think the thing is, that's why I very much appreciate the vibe check, the whole concept of a vibe check, which is you get a new model or get access to a new API. + +00:19:03 And what are the individual components that you want to test? + +00:19:06 So even to your example of refactor, here's the ORM, that sort of thing. + +00:19:11 I started very, when I have a model, and I use a lot of cloud code. + +00:19:16 I use a lot of Opus and so on now, but I'm sure this will evolve. + +00:19:19 I would love to try it with some open weighted models soon. + +00:19:22 And I will say something like in this URLs.py, because I work a lot with Django on the backend, I'll say update just the, I'm trying to add a new page. + +00:19:34 It's going to be at this route. + +00:19:35 Here is the, please update the URLs.py file for me. + +00:19:40 And like, here's roughly what I expect, right? + +00:19:42 Super specific. + +00:19:42 And if it can't even get that part right, the chances of you saying, okay, now build a new page, now build a new view, now do X, Y, and Z are very small. So you kind of have like the smallest unit that you would work with and then start building up. And it is a two-way street. Like you have to build confidence in that model over time. Like what are its true capabilities? And I will say like it's a lot of work, but it's a lot of work being a software developer and learning a new open source project. + +00:20:09 So it's actually not that different from just like, okay, pip install, new library, reading the + +00:20:14 documentation, all those things. But it's a different format of going back and forth with + +00:20:19 the computer. I think it encourages people to be less serious because it feels like a little chat, a little back and forth. Just, hey, I just asked a little question and it came with a cool answer. + +00:20:28 It doesn't speak, you need to study this and you need to really, this is a skill. It's like I'm having just a chat with someone you know or whatever. Well, it does open up, even if you set + +00:20:41 aside code generation. It opens up new capabilities that I would argue are useful for every single developer, regardless of how soon you are. So just being able to run Claude and in plan mode, ask it questions about like, where might I have security vulnerabilities in my code? Or where could I refactor database queries in order to compress the number of database queries within this view or where could I add caching that would be most impactful and don't even have it touch your code other than to read the code. That was actually like my big kind of breakthrough with LLMs was I was like, I'm just going to use them in read only mode. I don't need them to modify my code for me. + +00:21:22 I'm comfortable doing that myself as a developer. But once I got confident in the models being able to read the code, I was kind of like, eh, just like dip my toe in the water, like maybe modifying some things. And especially in Python, I've read a lot of scripts. I'm like updating data and I just, I don't know. It's not my favorite part of coding. So having models that can write for me, + +00:21:41 even if it's not perfect, and then I can modify it. I need to export this from the database in that format. And if it works at all, it's perfect. If it's not going to work at all, right? It's a little risk. Yeah. And an export from a database is, + +00:21:53 as long as you're not accidentally dropping a table, it is just a read-only, kind of like, tell me about my code. There's all these things out there that in software development are like, you have to do the analysis yourself, but if you can shortcut it with an LLM, that actually seems like a big win to me. + +00:22:08 And I don't actually see any downside to doing that. + +00:22:11 Like if, again, if it's, if it's a read only and you're not touching the code, that to me feels like only a win. + +00:22:17 It was, it's time that I've saved that otherwise I would have to invest myself. + +00:22:23 This portion of Talk Python To Me is brought to you by the folks at Posit. + +00:22:27 Posit has made a huge investment in the Python community lately. + +00:22:31 Known originally for RStudio, they've been building out a suite of tools and services for Team Python. + +00:22:36 Today, I want to tell you about a new way to share your data science assets, Posit Connect Cloud. + +00:22:42 Posit Connect Cloud is an online platform that simplifies the deployment of data applications and documents. + +00:22:48 It might be the simplest way to share your Python content. + +00:22:51 Here's how it works in three easy steps. + +00:22:53 One, push your Python code to a public or private GitHub repo. + +00:22:58 Two, tell Posit Connect Cloud which repo contains your source code. + +00:23:02 Three, click Deploy. + +00:23:04 That's it. + +00:23:04 Posit Connect Cloud will clone your code, build your asset, and host it online at a URL for you to share. + +00:23:10 Best of all, Posit Connect Cloud will update your app as you push code changes to GitHub. + +00:23:16 If you've dreamed of Git-based continuous deployment for your projects, Posit Connect Cloud is here to deliver. + +00:23:23 Any GitHub user can create a free Posit Connect Cloud account. + +00:23:26 You don't even need a special trial to see if it's a good fit. + +00:23:29 So if you need a fast, lightweight way to share your data science content, try Posit Connect Cloud. + +00:23:34 And as we've talked about before, if you need these features, but on-prem, check out Posit Connect. + +00:23:41 Visit talkpython.fm/connect-cloud. + +00:23:45 See if it's a good fit. + +00:23:46 That's talkpython.fm/connect-cloud. + +00:23:49 The link is in your podcast player's show notes. + +00:23:52 Thank you to Posit for supporting Talk Python To Me. + +00:23:55 Let me give people an example of something I recently did. + +00:23:58 And I want to kind of tie this back to like really emphasize the agentic side of things. + +00:24:03 So I have, I just checked, I have 530 GitHub repositories on my hard drive. + +00:24:10 I have three computers. + +00:24:11 I have like this streaming course live recording computer that we're talking on now. + +00:24:16 M2 Mac mini. + +00:24:17 I have an M4 Mac mini, which I usually work on, but I also have my laptop. + +00:24:21 And so every now and then, usually I'm on top of things, but every now and then I'll up a project and i'll start making changes i'm like oh i forgot to do git pull oh no oh this might be this might be bad you know especially if it's like a binary file like a powerpoint that i'm going to use for a course it's like there's no fixing it right you just yeah put them side by side and rename one and copy it over but it's it's not great and so i was sitting at my my um kitchen table on my laptop about to take my daughter to school and i opened up cursor with cloud sonnet in agentic mode and i described what i want i said what i would like is for you to create a script that i can run as a single command that will recursively find all the git repositories in a certain whatever folder i'm in downward find that do a git pull on every single one of them and report out which ones have changed. I drove my daughter to school, which is five minutes, one way, five minutes back. I sit down. The thing was there. It had created little test directories. + +00:25:25 It ran it. It said, it looks like it's totally good. And I literally just now have that as a + +00:25:29 utility. I can just, when I sit down, I'm like, oh, it's been like a week since I worked on + +00:25:33 security. Just get refreshed and just give it a moment. And then boom, you can see a report of all the stuff that's changed across all the parts. Even on something amazing is like chat CPT. You agentic. So maybe like, let's, that's a long winded way of like saying, tell us about the magic of like, what is this agent and tool using aspect for it? Cause I think when I said my rant, it's not working for people. I think a lot of times people are not doing agentic AI. They're asking LLM to write functions or stuff, which is great, but it's not the same. Yeah. Well, I think, I would say + +00:26:05 like kind of the biggest thing, and I mean, there's like, there's multiple attributes to it. Like, again, going back to some of the definitions, it's like you have an LLM and that LLM, if it's running chatGPT on your web browser, it's not going to have access to all the stuff in your codebase. + +00:26:19 Unless you have a public GitHub repository or something. But generally, when you're working in your local development environment, it's not going to have access to that stuff. + +00:26:28 To be fair, there are some tools that will take enough context and upload it to chatGPT. But again, what you're starting to do is you're starting to get really far away from a natural workflow and into one in which you have to bend to how the LLM is set up. + +00:26:46 So to me, I think the simplest way that I would look at it as agentic is like, it's something that is running side by side with you. + +00:26:53 It's almost like, I don't want to say copilot because copilot has overloaded terms with like kind of GitHub copilot, but it is essentially like, think about it almost as like pair programming with like more junior developers. + +00:27:04 Other people have used that analogy. + +00:27:06 It's not perfect, but it's kind of the one that I have that kind of sticks with me. + +00:27:10 It's as close as we got, I think, honestly. + +00:27:11 It's as close as we got, right? + +00:27:13 You can tell it to go do stuff, and it's not just going to only read. + +00:27:16 It'll look around. + +00:27:18 It'll experiment. + +00:27:19 It'll try something. + +00:27:20 It'll see it's wrong. + +00:27:21 It'll try something. + +00:27:22 It's kind of independent. + +00:27:24 And I think the easiest one for a lot of folks to have tried, because it was the one that I frankly got the most comfortable with at first, was in Cursor, you have which is essentially VS Code. + +00:27:36 So, you know, cursor is like embedded within that. + +00:27:38 And then you have different ways of interacting with like an agent mode where you're just asking to do stuff, right? + +00:27:45 Like asking stuff about the code base or please to write this function, whatever it is. + +00:27:50 So I do think that like that works for some folks. + +00:27:53 For me, it was not kind of the light bulb moment. + +00:27:55 It was kind of like where I started initially using it was I would, if I was like, oh, I need to like use this new API or I need to like kind of like develop a script or whatever it is. + +00:28:05 It was kind of like my, because I don't normally work in VS Code, I'm like a Vim, Tmux, like for a long time, like that's kind of my natural environment. + +00:28:12 And I was never going to like kind of adapt my own workflow. + +00:28:15 And I think a lot of people are like that, right? + +00:28:16 Like you're in JetBrains or you're in Xcode or whatever it is. + +00:28:19 You don't, you, you, what there's a breakthrough for you is like to build it in a way to your work, your development workflow that just is, is natural. + +00:28:28 And I think that's kind of, not the canonical definition, but to me is kind of most representative + +00:28:34 of kind of like agentic programming is like it it's just completely a part of your workflow and you don't have to adapt so um so again like that that cursor mode is kind of like okay i'd use it for like one-off stuff for me the breakthrough was using was using cloud code and i'll actually instead of talking about cloud code i'll say here was the bet that i made my feeling was there was enough value from cloud code uh based off of what i've been the videos that I've been watching, what I've been reading, and a lot of folks have been using it, that I was like, I'm a little worried about getting too reliant upon this tool, which is an API. + +00:29:10 And I will give Anthropic a lot of credit. + +00:29:12 They have done a lot of good work on these models, but I've also used Anthropic stuff in the past, which either had bad uptime or they were changing the model too frequently on the backside or things like that, that I was worried. + +00:29:24 And I think that's a natural thing as a software developer, should I really use this API? + +00:29:30 So I was like, my bet was, okay, even if I use this and it's amazing, if it goes away, I still need to be comfortable just like moving back to my workflow. + +00:29:39 Or there could be open weighted models in the future. + +00:29:42 Like I can just run Olama and I could run either Cloud Code or Open Code or some sort of CLI that would allow me to just do the same things, right? + +00:29:52 I mean, may not exactly, but roughly the same things. + +00:29:54 So that was kind of the bet that I made. + +00:29:55 That was like the philosophical mindset shift for myself was I'm already very comfortable as a software developer. + +00:30:01 Let me add this in a way that doesn't break my coding workflow. + +00:30:04 I'm not adapting to these tools in a way that is unnatural. + +00:30:09 And then I will use them in ways that I feel like are actually going to be productive as opposed to forcing them to be used in ways that almost like all the hype cycle and the news media is talking about, you will never write another line of code again. + +00:30:22 I don't actually really believe that is true. + +00:30:25 I don't know. + +00:30:26 I feel like anybody who's saying that is not actually using these tools. And I actually don't think it's going that direction. So I don't know that that kind of sets the stage rather than like how I use the tools, like what was going through my mind as a software, as an experienced software developer of like, should I even make this shift? I don't know if that if you had to undergo that + +00:30:45 as well. I did. And I guess I haven't been as introspective about it as you. But for me, The real shift for me was I'm already a good coder. + +00:30:57 I feel very competent flying around. + +00:30:59 Our tool chain, I know yours and mine is like kind of quite different, right? + +00:31:03 You're very Tmux based. + +00:31:04 I'm very much often in PyCharm jumping around there, but we all are very fluid and capable, right? + +00:31:10 So you're feeling like this is like, I'm really productive and competent. + +00:31:13 And the last thing I want to do is just like spend my days in a chat engine. + +00:31:17 You know what I mean? + +00:31:18 Right. + +00:31:19 Like that is, that is not it. + +00:31:20 And I guess one of the main things, I sat down with a friend of mine, we were out having beer and he had his laptop, a friend named Mark, and he had his laptop. + +00:31:28 I said, well, let's just, man, I've been doing some really cool stuff with Agenda. + +00:31:31 Let me just, let's just do an example. + +00:31:33 And over like a beer, we built the most amazing app that would literally take weeks or a week at least, you know, knowing what you're doing. + +00:31:42 I got not, not, I'm going to learn these frameworks and then do it. + +00:31:46 But even with, like that actually changes my perspective. + +00:31:50 Did it use open source libraries and frameworks in order to, like, did the tool pull in a bunch of stuff and it actually stood up and built this application off of all the open source code, right? + +00:32:01 Yes. + +00:32:01 And it was ridiculous. + +00:32:02 It was a FastAPI app using SQLAlchemy to talk to SQLite. + +00:32:09 The web design was done in Bulma CSS. + +00:32:12 It used simple MDE for live markdown editing. + +00:32:18 And it just, it kept going. + +00:32:21 And one of the things I think is really interesting, and I know you have some thoughts on this as well, is it's the more you work with like new fancy HIP frameworks, the less well off you are, right? + +00:32:33 Like I could have said Tailwind, but Tailwind's got all these build steps and all these little nuances. + +00:32:37 Bulma is just include a CSS. + +00:32:39 And so it didn't have, that was like a thing it didn't have to worry about and so on, you know what I mean? + +00:32:43 Yeah. + +00:32:44 So I find it's like, I find myself trending towards more simple code than more towards frameworks, which is the opposite. + +00:32:52 I think. + +00:32:52 Yeah, well, there's I'll even take it a step further, which is I very much appreciate. + +00:32:56 I very I agree with you on the like less build steps. + +00:33:00 And in fact, like the simpler is often better. + +00:33:02 But I will say that I very much appreciate the opinionated tools and frameworks. + +00:33:07 And that's why I've actually had a better experience using Cloud Code with Django. + +00:33:15 And a big piece is also, I've written thousands, if not tens of thousands of lines of code already in Plush Cap, which is typically what I'm building, if not some side scripts and stuff like that. + +00:33:28 And I will have- + +00:33:29 Let's do a really quick sidebar and let you introduce those projects. + +00:33:33 Just because I know it's really nice to be able to reference back and say, when I was adding this feature or this is the, so give people something concrete, you know, + +00:33:40 like tell us about full stack Python and Plush Cap. + +00:33:43 - So this is full stack Python. + +00:33:44 I wrote full stack Python. + +00:33:46 It's like over 150,000 words, all about the Python ecosystem, ranging from like the Python language itself to web frameworks, to deployment options, and content delivery networks, APIs, all sorts. + +00:33:59 So it was kind of like all the things around Python and it was broken down by conceptual ideas. + +00:34:05 So like data visualization and implementations, which I feel like is something that is not particularly obvious to people that are learning to program is like you have conceptual ideas and those have specific implementations. + +00:34:16 So a web framework is a conceptual idea across many different programming languages, but the specific implementations like a Ruby on Rails, like Rails, the framework, or Django, the framework, are the specific implementations within those programming ecosystems. + +00:34:30 And so that's essentially how I built out this site over 10 years. + +00:34:34 I will say that I only really work on one side project at a time. + +00:34:39 And Fullstack Python, I felt like, kind of run its course over 10 years. + +00:34:43 So I really haven't updated it in a few years. + +00:34:45 I still think it's actually relevant for most of it, but some of the links are a little bit dated and things like that. + +00:34:52 And it's not been brought into the conversation that we're having around coding agents and things like that. + +00:35:00 It is still relevant, but I would say not as relevant as it was when I was updating it literally every day. + +00:35:07 Right. + +00:35:07 Back in 2016 or something like that. + +00:35:10 Yeah. + +00:35:10 Yeah. + +00:35:11 And I also, I struggle a little bit because I would love to go back to working on full stack Python, but what I struggle with is you can ask LLMs about all of this stuff and it will give you very good answers. + +00:35:22 And so a lot of what I was doing was just bringing like as an experienced software developer, like writing almost like essays on like, you can see here, like, why are web frameworks useful? + +00:35:32 You can ask an LLM like why web frameworks are useful. And actually it'll probably give you a better answer than what I've written here, because it's going to incorporate tons of different sources. So that's where I've struggled a little bit with like, as you know, the chat models based on LLMs, especially as they've added search have gotten better. I'm not really sure how to add enough value on top of what an LLM can tell you to justify a substantial investment in writing. + +00:36:00 And then also, the one challenge with Full Stack Python is it's a statically generated site. It is primarily Markdown built with Pelican, which is a great static site generator in Python, but it was a lot of writing and it wasn't as much coding as I wanted. The site is kind of the site And then you add a ton of content to it. To me, I really wanted to get back to, especially now as a VP for the last several years of my career in executive level, I don't get a lot of time day in and day out to code. And so I really wanted something that I was coding every single day on nights and weekends and just doing in my spare time. + +00:36:39 So that's kind of where I shifted to this project, PlushCap, which is at plushcap.com. + +00:36:43 So this is like a landscape of developer-focused companies with self-service motions. + +00:36:49 So the hypothesis behind this was like when I was at Twilio, we were tracking developer content and events and other competitors. + +00:36:58 And it wasn't back then, but now YouTube is a really important source for teaching developers. + +00:37:05 And so I want to just like to aggregate a lot of this data. + +00:37:10 So that's what I've done. + +00:37:11 It's essentially 500 developer-focused companies across all different stages from like pre-seed all the way to publicly traded companies, where they stand kind of in their competitive position. + +00:37:21 And then like leaderboards that will show you like how many videos and views and subscribers do different companies have. + +00:37:28 And so like there's just a lot of data analysis, a lot of data visualization, a lot that essentially just goes into, like, if you go to, if you scroll down on Airbyte and you go down to, on this page, if you scroll down to the website structure and click on the blog, like, sorry, if you click on, go back and just click on the blog or just go up a little bit, the blog posts. + +00:37:52 Gotcha. I see. + +00:37:53 Under content category, this lays out the number of blog posts that was published per month by the company. And so just being able to visualize like their content patterns and trends is like, been helpful for me as I talk to companies about their go-to-market motions with developers and developer marketing, things like that. So anyway, this is a Django project running on DigitalOcean, Cloudflare as a CDN, a ton of Python, a ton of database stuff on the back end. So for me, I just love digging into this and I use coding agents and they've really greatly accelerated what I can actually do with this. + +00:38:34 Because what I'll do is I'm in meetings all day. + +00:38:37 So in the start of the day, I'll just tee up like a bunch of stuff that I want to get done. + +00:38:41 And then throughout the day, as I'm like getting out of meetings, I can just hit like, okay, okay, like good. + +00:38:45 And I just run it all on my own computer. + +00:38:48 And then I've got my work computer, I got my own computer where I'm like running all this stuff. + +00:38:52 And like periodically when I'm taking a break, I'm gonna be like, okay, yep, that looks good. + +00:38:55 Or no, I don't do that thing, right? + +00:38:57 So to me, it is like having a development team, even just a small development team, because I'm not doing like huge stuff to actually implement things. So that's really where I use all these tools. And I have an established code base. I wrote the code base myself over three years by hand. So now I have all the design patterns. + +00:39:15 I have an opinionated framework with Django. I've already chosen the vast majority of the open source libraries that I need. And it is mostly about the coding tools, pattern matching against what I've already done to create a new visualization or to create a new scraper because I'll scrape all the content from all these sites. + +00:39:34 So if I add a new company in the database, I have to write some custom web scrapers in order to get all their data. And that stuff is annoying. I've already written hundreds of those. So I'd rather just have- You're done right now. + +00:39:48 And then out the other side comes something that I can actually use. + +00:39:52 And I don't have to do all the grunt work because I've done a lot of that myself. + +00:39:55 Yeah. There's no more lessons to learn by making you- figure out their nav structure. + +00:40:01 Yeah. + +00:40:01 Or like, you know, I'm pulling like, okay, which CSS class corresponds to like the author name of the blog post so that I can get some structured data out of what is otherwise like unstructured data because it's every single company has a different blog format, right? + +00:40:16 So. + +00:40:17 Yeah, I think this, you touched on two really interesting things here. + +00:40:20 First, well, three, I guess. + +00:40:22 One, PlushCap is super cool. + +00:40:23 Thanks. + +00:40:24 Yeah. + +00:40:24 Two, having a low stakes project that you can just kind of go mad on with agentic AI + +00:40:32 and it's not going to end things or end badly. + +00:40:36 And three, I think this is really interesting. + +00:40:39 And I've noticed the same trends for me as well. + +00:40:42 But having already created the project, having really carefully thought about design structures, like, okay, it's going to be in this folder structure and it's going to follow this design pattern over and over using these three libraries I like. + +00:40:55 Not the most popular ones, necessarily, but the ones that I think I want to work with. + +00:40:59 Putting that in place really limits the, oh, it just, why is it using NPM? + +00:41:05 This is a Python project. + +00:41:06 What is it up to? + +00:41:06 You know, like it's way, way more focused and more likely to be aligned with what you were hoping it would do in the first place. + +00:41:13 And you need to choose the tools that you can debug the best. + +00:41:18 And so for me, like this is a Bootstrap front end. + +00:41:21 There's a framework called Tabler, which is built kind of on top of Bootstrap, which makes it even easier to build kind of these admin style interfaces and pulls in. + +00:41:29 I think it's I think it's chart.js. + +00:41:32 Apex might be Apex. + +00:41:33 I was using a few different charting frameworks. + +00:41:35 So it might be like Apex or something now. + +00:41:37 But the whole the whole point of like choosing your tools and being opinionated about your tools, I think actually helps a lot because then you don't have to get up to speed on the framework or the library before you debug the problem that the LLM created, because the lms are typically gonna they're gonna get you often 95 of the way there and you can keep prompting them but it's often it's often better to just like go fix it yourself there might be like a one-line tweak or something like that um that you need to do as opposed to like trying to get it to like sometimes it feels like getting the last like five to ten percent of polish is like finding a needle in the haystack because it's changing too much often whereas if you can just go in and and modify it yourself, you're not going to, you're not going to introduce additional noise into the development process. + +00:42:25 Yeah. + +00:42:26 I have a, but seems like an out of left field question, but it's not. + +00:42:30 What about Git and source control? + +00:42:32 Oh, yeah. + +00:42:33 So, I mean, that's the thing that we haven't talked about, which is like wildly important, which is like after when I'm letting something run, it is off of either it's either in a separate Git branch from running a bunch of things, or I actually like a grant, I have some simplest simplifying assumptions with my own code base. + +00:42:50 I've split the code base into multiple parts. + +00:42:54 So I have scripts where the scripts are like web scrapers. + +00:42:59 And if I'm updating things through an API, like I use the YouTube API. + +00:43:03 I use the PyPy API. + +00:43:07 I use a bunch of different APIs to get data into the database. + +00:43:10 So I can have an agent running on that thing. + +00:43:14 And that's separate. + +00:43:15 I actually only have those things interface with an API that is running with Django framework with the main production application database. + +00:43:23 So I've created a layer of abstraction where I can have an agent running on this thing over here. + +00:43:28 Then I can also have it running on the main application code base. + +00:43:32 And as long as we're not changing anything about the API, then I know that they can make their changes separately and that they're not going to interfere with each other. + +00:43:41 And all of this, going back to your original question about Git, I'm just constantly just doing Git add, Git commit. + +00:43:47 The one thing I've definitely noticed about some of these tools they will go off and create a bunch of different files. + +00:43:53 And so you do have to get pretty careful with like, I used to just do, you know, git commit, all the things, + +00:44:01 git commit dot, or no, sorry, git add dot, git commit with a little message. + +00:44:06 And I've learned like, oh, geez, like there's some debugging files that it actually didn't remove. + +00:44:10 I got to like manually remove those. + +00:44:12 Now, again, like I think a lot of these tools, you can configure them to like get rid of those problems. + +00:44:18 I think that that's the thing that's really improving quickly now. + +00:44:21 So if you ask me, like, what's the thing now that's really annoying that will likely be much easier a year from now, it will be like, you're going to have a standard like claw.md file that gives instructions that's just going to be like standardized. + +00:44:35 It's like how everybody has a git ignore file. + +00:44:39 Like you just kind of copy it from somewhere, right? + +00:44:42 Yeah. + +00:44:42 And then you modify it a little bit. + +00:44:44 But like the vast majority of it's the same. + +00:44:45 Like that's where we're going to go, right? + +00:44:47 Right. + +00:44:48 Right. + +00:44:48 You're like, oh, this we've all tended towards this. + +00:44:50 Yeah. + +00:44:51 And so that goes back to my, you're not prompting enough sort of things, right? + +00:44:55 Like if, if you're opinionated about how your project is built, put that in there. + +00:45:01 And a lot of times these agentic tools have like a rules file, like cursor has a dot cursor rules and stuff that you might put in there is my preferred web framework is FastAPI. + +00:45:11 When you install things or create a virtual environment, use uv, not Python's native, right? + +00:45:19 And you'll see it when it's working. + +00:45:20 It'll say things like, I know that I'm supposed to use uv, so it'll run uv pip list when it's checking if something got installed right. + +00:45:28 And it'll even like sort of remind itself from what you've said. + +00:45:30 And it's those kinds of things that keep sit on track, right? + +00:45:34 So I would say one of the things people should actually put more effort into than maybe they initially think is like setting up these rules about just your general preferences yes and uh i think + +00:45:43 it was armin roniker had a really great uh tip the other day where essentially like you can just use i believe it's a hook in cloud code and it's like basically if the um the lm tries to use a pip install something it will actually throw an error and it will know based off of that error that it should be using uv uh and so you could add enough of these hooks and so like what i anticipate is like over time, number one, like people will develop some level of standardization for these, like right now, the configuration files are very fragmented across the different agentic tools. So you can't quickly swap from like one to the other, like, and like there's a Gemini CLI and there's a cloud, you know, cloud that has a configuration, both for project specific and for all your development workspaces. My guess is like a lot of this stuff will start to get standardized, just like MCP was standardized and an open standard. And then it will make it much easier for people to just transpose their development preferences from one tool to the other. + +00:46:44 So my guess is that's probably what's coming over the next 12 to 18 months. And if the big companies don't want that to happen, they'll be left behind by the open source tools that will make that much easier. I think the other thing is I actually don't... I don't know. Maybe I'm one of the balls with them. I don't, I have a VMRC, like for my configuration, I don't have like an insane number of extensions or, you know, or I guess plugins, or like a huge VMRC or anything like that. Like I actually try to keep things relatively standard. And then just focus on the patterns that are most important or the configurations that are most important. And I I'm still kind of like that with, with using cloud code, I do have, you know, cloud and defy on stuff. But I also found that it's not 100% accurate. And so I think there's going to be a lot of development around making sure that the tools adhere to the practices that you really want. Because right now, I feel like if you want reliability in code formatting, you need to run the code formatter as a separate step as a part of your build workflow. The agentic tools are just not... Maybe I'm not doing it right. + +00:47:55 I'm reliable now. + +00:47:56 I've gotten, cloud sonnet to, to know that it's supposed to run ruff format and rough check --fix whenever it finishes anything. + +00:48:04 And so it'll, at the end it'll say, and now I'm supposed to do, you know, this to make sure it's tidy and like the style you like according to your rough Tom, all right. + +00:48:12 Yeah. + +00:48:12 And that, that makes sense. + +00:48:13 I mean, it's, it's not, sometimes it doesn't. + +00:48:16 Right. + +00:48:17 But you usually do what do you, what's wrong with you? + +00:48:19 I know. + +00:48:19 Yeah. + +00:48:20 And also I feel like everything that you add to the agent, like the more and more steps than you add, actually, again, this will change over time, but I feel like it actually gets less reliable. And also the number of steps, I would rather just have it run a script that will handle all those things before I'm ready to commit, rather than having it run all the time. + +00:48:39 Because I've also found that sometimes I want to add a tool for code formatting or something, and then it slows down the whole process too much for me. And then the main thing with these tools is I want them to be extremely fast. And that is probably the biggest piece of advice I can get to any of these companies is like why Claude Code over others? + +00:48:57 Like it is insanely fast, even though the LLMs require, you know, processing time and stuff like it's responsive. + +00:49:03 And so like my guess is like that's also where a lot of the open source tools will go as well. + +00:49:07 They'll just be really fast and responsive, just like Vim and T-Monks are. + +00:49:10 Yeah. + +00:49:10 Leave it to a Vim user to say it's not fast enough. + +00:49:13 Just kidding. + +00:49:15 Yeah, exactly. + +00:49:16 So I think maybe just like some quick shout outs to some of the tools, right? + +00:49:21 So Cloud Code obviously is a really interesting one. + +00:49:24 You've been talking about that a lot. + +00:49:25 I've mentioned Cursor a few times. + +00:49:26 I think as much as I'm a much more fan of the PyTarm style, I find it to be very effective. + +00:49:33 I've done some stuff with Juni, which is kind of a new project from JetBrains. + +00:49:37 Like JetBrains has had an AI agent that was kind of like a super duper autocompleter. + +00:49:42 And maybe we could talk about that difference as well. + +00:49:44 But it's like a super duper autocompleter. + +00:49:46 but then they have a separate completely different install thing that is an agentic ai called juni and i think it's making good steps it like really integrates with pycharm if you're you're into that + +00:49:57 um yeah well and that's been the evolution right like we had yeah github copilot which was essentially like a very fancy auto complete it kind of it would complete your code for you but it was not an it wasn't an agent in any meaningful capacity like that's what kind of kicked off some of this stuff of like wow it's really great autocomplete and i think a lot of the tools follow that pattern. Even when you would use cursor say a year ago, it was very much like auto-complete, at least the standard pattern, right? Before they had the agent mode. + +00:50:26 For sure. And that never really connected with me. I'm like, I don't need that. I can just, I can auto-complete it word by word. I don't really, because half the time it'll auto-complete like three or four lines. I'm like, yes, but not that line. And how do I get it to auto-complete + +00:50:39 the next? I guess I got to accept it then go edit that part. You know, like it's just, + +00:50:43 I never really vibed with that, but the agentic mode, that is next level stuff. + +00:50:48 Also, if you're a PyCharm or VS Code user, the cloud code integrates in both of them, right? + +00:50:55 Yes. + +00:50:57 So maybe talk, I know you're using cloud code, so maybe tell people what it's like and why you like that one. + +00:51:03 Okay, so I started using cloud code like six months ago, but I was using it off of the API credits, which was before they came out with the plans. + +00:51:11 Started evaluating it. + +00:51:12 I was like, this is going to be way too expensive. + +00:51:15 Like this is going to be- Yes, it is very expensive very soon. + +00:51:18 I was like, based off of my usage patterns, like just like monthly, it's probably going to cost me over $1,000 a month. + +00:51:25 I was like, this is just not, it's not worth that, right? + +00:51:28 So then they came out with the max plan, which was I think originally $200 a month. + +00:51:32 Now they have $100 and $200 a month, which I thought was interesting, right? + +00:51:35 But I was like, I talked to enough people that did use it. + +00:51:37 And I was like, that's interesting, but I'm not that comfortable with this workflow yet. + +00:51:41 They came out with the pro plan, which was $20 a month. + +00:51:43 They said, okay, I'll try it for a month. + +00:51:45 It was almost like a, to me, it was almost like a free trial. + +00:51:48 Like I'm willing to invest $20 into a course or whatever it is like 200 is like a pretty high bar, but 20 bucks. + +00:51:54 I was like, yeah, okay, fine. + +00:51:56 Even if it, even if this sucks, I'll just, I'll try it. + +00:51:59 So I maxed out the pro plan. + +00:52:01 I was like, I'm gonna get every drop out of the pro plan that I possibly can for my 20 bucks, and then I'll make a decision as to whether I want to upgrade or not. + +00:52:08 I was like, this is good enough. + +00:52:11 And that was only with the Sonnet model. + +00:52:13 It wasn't the higher Opus model. + +00:52:14 So there's a lot of stuff that was a little bit more limiting about it. + +00:52:17 We have to give a lot more instructions. + +00:52:19 It wasn't as good as architecting things. + +00:52:21 But I got enough out of it. + +00:52:22 I was like, okay, I'd be willing to upgrade. + +00:52:25 And so I'm actually currently now on the $200 a month plan. + +00:52:28 I think it's worth it. + +00:52:29 I will say that it also, for me personally, it forces me to be a little bit more addicted to using it. + +00:52:37 I want to get my money's worth. + +00:52:38 and so there's a, there's a, a tool called CC usage, which, was originally just for the API, but now, is it'll just give you your usage patterns. + +00:52:49 Even if you're on the, the, you know, subscription plan, I will say like, my usage yesterday was, if it was by API would have been $105. + +00:52:58 So, you know, over a course of a month, if I was actually consistent, I'd be spending over $3,000 a month off of API credits, which is just not, not sustainable. + +00:53:06 Right. + +00:53:06 Yeah, there might be people out there listening who think, you know what, Matt is actually crazy because $200 a month to some random AI thing when I'm a developer is insane. + +00:53:16 But if you had somebody who worked for you, who you could give detailed instruction to and have some reasonable expectation that they can get it done independently, and you said, well, that person you hired is $200 a month, that would be insane, right? + +00:53:29 And this, you can really turn this thing loose and get a lot done. + +00:53:34 But I think one of the mind shifts, at least I had to make was if I'm going to pay for one of these AIs at a level like that, you better make use of it. + +00:53:42 And you better have a plan that like, this thing is, this is how I'm going to take advantage + +00:53:46 of it. + +00:53:46 But when you do all of a sudden, you're like, wow, this is actually a bargain. + +00:53:49 Yeah. + +00:53:50 Well, and let's, let's compare it to like another like AI tool that I use, which is Descript. + +00:53:54 And I use Descript quite a bit. + +00:53:55 Descript is for creating, you just record video. + +00:53:58 And then you can edit off of the transcript and it'll like edit the video. + +00:54:02 It's not perfect, but it's very, very good. + +00:54:05 And I'm on like, I don't know, it's like 150 bucks a year or something like that. + +00:54:09 I use it a lot for everyone. + +00:54:11 Like my teams are remote distributed and I like to record videos just so they can hear from me like in five to 10 minute increments of like, here's what I'm thinking about or shout outs or whatever it is. + +00:54:20 Right. + +00:54:20 So I use it like quite a bit internally, but like if I don't use it for a week because I'm traveling, I don't feel bad about it. + +00:54:27 And so like, but that's 150 bucks a year. + +00:54:30 Like I get enough usage out of it at 150 bucks a year that if I don't use it for a week, it's not that big of a deal. + +00:54:35 200 bucks a month is like a really high bar. + +00:54:37 And so my bet there was just like with the $20 a month plan was, if I don't use this enough, I'm going to cancel it. + +00:54:44 And the other bet is I think eventually that the open weighted models are going to get really interesting here. + +00:54:51 And so I just want to be on the edge of seeing this ahead of time. + +00:54:56 And so I look at it as a little bit of an investment in my own learning. + +00:55:02 And so to me, there's just no replacing hands-on time with the tools, right? + +00:55:08 And so that to me is really what this is about. + +00:55:11 And to be fair, I've used other tools, other APIs, developer tools, where I paid for them. + +00:55:18 It's actually a browser-based, which is like a web scraping tool, is awesome. + +00:55:21 I have paid 40 bucks a month here and there to use it. + +00:55:24 And then when I don't use it enough, I will just downgrade my plan. + +00:55:27 And you kind of have your short list of things that you're using as a developer, as a time, as opposed to thinking that it is like indefinitely, I'm going to be paying for this. + +00:55:36 Yeah. + +00:55:36 Yeah. + +00:55:36 I definitely think it's important for people to maybe not try the $200 plan, but like the $20 plan or something, if they're interested in these things, because it's downloading a free model that runs on your local machine, that 3 billion parameters and saying, well, I tried AI and it doesn't work is not the same as, as trying this, this type of thing. + +00:55:56 Right. + +00:55:56 Right. + +00:55:56 It's, it's, it's really, really different. + +00:55:58 Yeah. + +00:55:58 Well, and also think it's, if you have a, I think that's why my situation, okay, let's, + +00:56:03 let's maybe like set up a little framing of who I think this is most valuable for. + +00:56:07 If you were like an elite developer and you're working on things like embedded systems or things that absolutely cannot break. + +00:56:13 I actually think there is less value for you here. + +00:56:15 I think the read only mode is valuable in that. + +00:56:18 like analyze my C code, where could there be like, you know, like memory, you know, buffer, buffer or bullet attacks or whatever it is. Right. But I think if you were building like side projects, like I'm not trying to like monetize like plush cap really, but like, you know, maybe someday I will. I just really love building plush cap. Like I think if you have a side project and for you, like you're building courses, you have your website, like there's a lot there that if you just had more bandwidth, you could build so much more cool stuff. + +00:56:49 Like, yeah, like when you have a backlog of ideas and you don't have time to build them, that's actually where I think these tools are most valuable because you can build a lot of those ideas and some of those ideas might be stupid and then you can just throw them away. + +00:57:03 But the ones that you, if you can just clear your backlog and come up with new ideas, because this thing is just, you're able to ship so much faster. + +00:57:10 Even if you have to refactor by hand, that's amazing. + +00:57:13 Yeah. + +00:57:13 I have three projects that are like sort of side project-ish, but are like related to Talk Python in various ways. + +00:57:21 And yeah, it's just turn the stuff loose on it and see if it's a good idea. + +00:57:24 Who knows if it'll see the light of day, but it goes from, can I dedicate three months to that to, well, let me try an afternoon. + +00:57:32 Yeah. + +00:57:33 See how it comes out, you know? + +00:57:34 Let me show you like something that I built purely off of like Claude Code. + +00:57:39 if you go to if you go to plush cap i have a um a leaderboard uh of youtube channels and by company so you've got i actually have you know all almost 500 companies in there but it cuts off at a thousand so yeah if you go to leaderboards youtube um you know there's there's 231 companies listed here and all 231 are um uh have over a thousand subscribers now the thing that's interesting out like i was just showing this to someone and they're like oh you know it'd be really interesting is like you have like subscribers, you've got videos, you've got views. And if you click into like views, like if you click into the number of views, this is total views over time. It's a visualization of like, you know, what, how many views, open AI has on their YouTube channel over time. They were like, what would this look like? You know, cause they're, they're producing more and more videos. So are, is the average number of views going up over time or down? + +00:58:34 Like, are they becoming more efficient or less efficient? + +00:58:37 So I was like, I wonder, like I already have subscribers, videos and views. + +00:58:42 I have the visualization. + +00:58:43 I literally have the pattern for what I would like this to be. + +00:58:45 And yeah, there now you can see actually they're becoming less efficient over time as they add more videos and there's different ways you can slice this data. + +00:58:54 So I've actually got like a bunch more ideas just from creating this view. + +00:58:58 But this view was based off of the pattern that I had already created. + +00:59:02 And so it was just basically like using the Django ORM to pull the data, you know, create a new web page. + +00:59:08 And I could tell whether it was correct or not and then like tweak it. + +00:59:11 So it had the pattern and it just needed to mimic that in a way that was a new view and visualization. + +00:59:19 And I feel like that's what this is. + +00:59:20 Pull new data, make a new page, but it's pretty much the same. + +00:59:23 Exactly. + +00:59:24 Yes. + +00:59:25 So this isn't like, you know, rocket science or anything like that. + +00:59:28 But for me, like I wanted to build this view. + +00:59:30 I actually, the interesting part about the story is like, I talked to somebody in the morning and I shipped this like when I had a quick break and I was like, I sent it over to them and I was like, Hey, I built this thing. and actually no, it was my, my boss at work at DigitalOcean. + +00:59:45 He was like, cause we're looking at our YouTube channel and he's like, I wonder what the average views are over time. And I like, I was like a few hours later, I was like, I had a break, I shipped it. And he's like, this is crazy. Cause we use, you know, we use this data as a part of + +00:59:55 running my, my org, like Debra and stuff. So yeah, absolutely. Because whenever we have an idea, + +01:00:00 we're like, what does the data look like? Or what, especially when you can compare it across + +01:00:03 companies, that's really helpful. Yeah. It's the perfect type of thing. I absolutely love it. + +01:00:08 So let's close this out. We got maybe take a few more minutes and let's, let's give people some concrete tips. Like for example, when we talked about Git, you know, check in off to let it finish a job, save it and check that in. And don't let it just go wild for a while. Cause if it goes of bonkers and you're like oh i hate that you're like but that's also hours of work that's gone you know what i mean like use use source control as a um um sort of a save along the way + +01:00:35 um you've got a bunch of tips and stuff very give us tips yes i know okay number one thing i actually feel like i can clear uh i have better clarity around technical debt and how important it is to get rid of technical debt now because if an l if you're using the like an agent and it it like is looking to pattern match to what you have in your code base, and it finds the area that you know is basically technical debt, and it copies that, and you're not really on top of looking through the code, it will often copy design patterns that you really do not want in your code base. + +01:01:08 And so I actually feel like it allows you to make the argument that cleaning up technical debt is more important when you use LLMs and agents than when you were doing things by hand where you knew So as a developer, I'm just going to ignore that section of the code and I'm not going to use it. + +01:01:25 So that's like a very specific tip, which is like, if you are going to pattern match against something you're doing in your code base, make sure it's not copying the area that you know is a problem. + +01:01:35 And then just like shipping that. + +01:01:37 I think that's, that's one really. + +01:01:38 You don't want to supercharge your bad habits. + +01:01:41 Yeah. + +01:01:42 Yeah. + +01:01:42 I would say on top of this, you absolutely do want to clean up technical debt. + +01:01:47 you can, when it's done, say, great work, that's valid, but now please apply some dry principles and some design patterns that you know that we're already using and just polish this up one more time and you'll get pretty good results. On the other side is if you've got a project that's got a ton of technical debt, you can kind of just have this conversation with the agentic AI and say, help me understand, where are the long functions? Where is there too much coupling? Let's start breaking that down. And when it's in that mindset, it's, it's pretty good at like addressing those + +01:02:20 problems. Right. And that's, that's the flip side of like the technical debt is like, it should be easier, arguably to clean up technical debt, because if you fix something or you're, you're asking the L like the agent, like the introspective, like how could this be improved? What are the design patterns? And particularly like, at least in clock code, there's like a plan mode. I use plan mode. It's like a shift tab. I use it all the time because then it's not, it's going to just like tell you what it's going to do, not go and do it, not going to make any changes. And so I will ask it, like, how can I optimize these database queries? How can I simplify this code without oversimplifying? That type of thing. And that's actually really helpful for just identifying what changes actually should be made. Then I can make the decision, are you going to go ahead and implement those? Or should I just go ahead and implement them because they're easy enough for me to make the changes myself. Awesome. Next tip. Yeah. So I feel like you've already talked a little bit about this. Like the context is really important and the prompting is important. I will say that like where I've failed in the past and I like was not happy with the LLMs was like, I'd say something like, can you just write a bunch of unit tests for this thing? And like, of course, it's not going to have enough. Like I give it the code that I want it to be testing. But now what I found is like I, when I, after I have it write some code, I will say like, okay, now write me like a few happy paths and I'll like evaluate the happy paths and I'll be like, write the unhappy paths. And I'll like, look at those and how it's failing. And then I'm like, what are the edge cases? And so you just have to, even if you're like, a lot of people are like, oh, here's how you write the perfect prompt. Like I actually really hate doing that because what if I wrote the perfect prompt and it still didn't give me what I wanted at the other side? I just tend to + +01:04:00 have to tell it it's going to go to jail please you know no i just tend to be really incremental + +01:04:05 i'm like really specific like just like again going back to like that junior developer mentality it's like just write me like three happy path tests to test this function yeah like give me some inputs and the expected outputs okay now i want you to write the things that like basically would break it and make sure that it doesn't break so what that does is like rather than i i have not found success or it's been a lot harder with like write me a bunch of tests both happy path the non-happy path and then it's kind of you know depending which model you're using or what day you get like it may or may not be correct and so i brought it's also think about like for yourself and your own sanity when you're doing reviewing the changes that it's making uh you really want it to just like be consumable like okay those are the happy path and mentally you're like okay happy path tests or mentally not that um you should if you instruct it in the way that you want to consume the information that it's created out the other side, that can actually be really helpful as well. + +01:05:00 Yeah. I find it's terrible if you just, this, this project needs tests, write tests. It ends up testing like just the smallest arbitrary aspects of the code. And you're like, what is this? + +01:05:09 Like now I got to maintain this junk that like one plus one is two. Like, no, this is not what I want from you. Right. You got to be really specific and do bite size. And it's a little bit of the plan to like maybe plan out like the types of tests we should have. Okay. Let's do this first one and focus, you know, like really guide it a little bit, treat it like it's, it's a junior that's kind of new to this stuff. Don't just, you wouldn't tell the junior, like we need to refactor the architecture. I'll see you next week. You're like, well, what are you going to + +01:05:35 get back? You don't know. Yeah. Well, maybe, maybe like three quick, like lightning round tips. + +01:05:40 Yeah. One of one, one of them mostly just, I thought about because I do this with, I have been doing this for years where if I'm working on different projects or different part of the code base, I will just use different terminal backgrounds, different colors. This is my blue terminal. So blue is where I've got Tmux and Vim, and I'm running a bunch of stuff. Exactly. + +01:06:02 And then, yeah, I think my colleague, Amit, tweeted about this. And then black will be, I'm running the server, I'm running my task queue, all the stuff that is running right now, local development. Then if I'm tailing a bunch of logs on the server, that'll be in another + +01:06:19 different color. So just mentally- Does this happen automatically? + +01:06:22 No, I actually just open up separate terminals. And then within that, I'll have different team + +01:06:27 accessions running where I can just be like, again, it has nothing to do with the computer. + +01:06:32 It has everything to do with me. A lot of times just doing a ton of context switching. So I've found this to be applicable as a pattern for the agents. If I'm having an agent that works primarily on one part of the code base, then I will just have that be blue. And then another part of the code base have a B black. + +01:06:49 Right. + +01:06:49 So, again, like I also, this maybe goes right into the second tip, which is a lot of people are like still microservices versus monolith. + +01:06:58 Like I actually think that the perfect design pattern right now for a lot of projects is like much closer to monolith, but like you split it up into like based off of the number of agents that you want to run. + +01:07:10 So like, for me, I've got like my web scraping and like data collection stuff. + +01:07:14 And like, I generally just want like one, maybe two agents. + +01:07:18 So like, that is like a separate project. + +01:07:20 And then it interfaces with everything else through APIs, like actual like web APIs. + +01:07:25 And so like, that is actually how I've architected things where I'm like, how many developers, developers, agents do I want working on the code base at once? + +01:07:32 Like, what is my actual capacity to review the code? + +01:07:36 As a single developer, there's only so much capacity that I have. + +01:07:39 And so I will actually architect, or I've started to architect around that. + +01:07:44 But that doesn't mean I have a thousand microservices. + +01:07:46 It typically means that I split the architecture into like three or four components, maybe two beyond the main application. + +01:07:55 And that helps me to just feel like I'm being productive without being overwhelmed. + +01:07:59 Yeah, it's also good advice just for working in small teams anyway. + +01:08:03 Yeah, yeah, exactly. + +01:08:04 And it's just like a little bit more like a pragmatic. + +01:08:05 Some of the teams are automated, right? + +01:08:06 It's crazy. + +01:08:07 Yeah, it's like a pragmatic, like I'm not doing crazy microservices. + +01:08:10 I'm not also doing like monolith everything, right? + +01:08:13 So I think that to me, I'll just kind of be a little bit more, more pragmatic about things. + +01:08:18 Yeah. + +01:08:18 I wonder if you could automate, if you could, if you could AI agentic code, some form of automatic color profile selecting. + +01:08:26 I mean, yeah, they go pretty good on bash scripts. + +01:08:29 I mean, it's possible. + +01:08:31 I've one, one last tip, which kind of bit me. + +01:08:34 Luckily I have, I'm running Sentry, shout out to the Sentry team for, for, for BlushGap. + +01:08:40 and I caught this bug and I, you know, a lot of times I'm like finding bugs, like, oh, it's a 500 error or whatever. + +01:08:46 I'm not the end of the world. + +01:08:47 Like I go and fix it. + +01:08:48 I had definitely written more bugs myself by hand. + +01:08:53 Now that I am like, I have a mixed sort of like toolkit. + +01:08:56 Like I, I've just, it's just something to like be mindful of, which is like, I'm, I'm so used to having like bits and pieces written by an agent. + +01:09:05 And then I review it that when I'm writing code, I have like my, it's almost like over It's like, oh, I review a lot of code. + +01:09:11 I actually almost feel like the agent should be reviewing the code that I'm writing by hand. + +01:09:15 And so I'll just say like, that's one thing that I feel like is a big tip for developers and might bite a lot of people as they spend somewhat less time writing code, especially like low level kind of like rote detail kind of stuff. + +01:09:29 And then they have to, then they are like, oh, I'll go do that real quick. + +01:09:31 And then they suddenly like introduce a bug and they're like, oh, geez, I didn't. + +01:09:35 Yeah. + +01:09:36 Yeah. + +01:09:36 haven't done this like yeah by hand with the clickety click for a while yeah exactly exactly yeah i um i think that we need to we need to embrace and be mindful like you're saying um a certain level of like student mind when we're doing this kind of stuff i mean you say you've been using clock code for like six months and these are starting to show up like imagine 10 years down the line right you know what i mean it could be seriously an issue yeah i did help one person do a bunch of HNNC coding and won't name it because I'm not sure they want to be named, but they really were like, wow, this is incredible. And the thing that surprises me, they said, was as it's going through maybe 30 or 40 things it does to process a request, it's explaining why it's doing it. It says, okay, now I need to make sure that this runs in this way. So let's do this like Django, this manage pi command or something like that. + +01:10:32 Right. + +01:10:33 And they're like, I'm actually, if I'm paying attention, I'm actually learning new things about how this is supposed to be done. + +01:10:38 And in the end it says, here's why I made these changes. + +01:10:41 I think it's so easy to just go great next. + +01:10:44 You know what I mean? + +01:10:44 Yeah. + +01:10:45 But if we just slow down a little bit and you can learn a lot to learn, you learn, I've learned a ton. + +01:10:50 Like I can just hit escape and I'm like, tell me why you're doing that. + +01:10:53 I'm not really sure why, why you're doing that. + +01:10:56 And it'll explain it. + +01:10:57 And then you can, you can, it's the back and forth. + +01:11:00 I mean, it is like having the conversation about code that you often can't have with another developer. + +01:11:07 It's really expensive to take your whole team's time to talk through some change to the code base. + +01:11:12 You can have all those conversations. + +01:11:14 And I personally find as someone who is relatively introverted, it is not taxing on me to just have that conversation with the computer. + +01:11:20 So I can still have all the conversations with the humans and not be drained by the conversation with the computer. + +01:11:26 Yeah. + +01:11:27 Yeah, that's awesome. + +01:11:27 And think of the opportunity for people who either are sole developers or they live in places where they can't go to user group meetup type things. + +01:11:35 Yeah. + +01:11:36 Yeah. + +01:11:36 Or they're like busy in meetings all day and they still want to be a productive developer and be hands on with this stuff. + +01:11:42 And not, you know, I think a big part of it is like not getting caught up too much with the whole hype cycle. + +01:11:47 I am firmly in the camp that while this is a shift, it is not that big of a shift compared to when they said, oh, software developers are going to go away because we have COBOL. Software developers are going to go away because we have code generation off UML diagrams. Software developers are going to go away because of low code, no code. + +01:12:08 I get the sense, having used these tools, that a lot of companies will use them in the wrong way of thinking they can just replace developers. + +01:12:16 And actually what they're doing is they're building up a massive amount of technical debt and they're going to need software developers to fix it. + +01:12:22 Yeah. + +01:12:22 So that, I think it's building further layers of abstraction and more code. + +01:12:26 I think that you and I and everyone else out there who is very competent coders pre-AI, we're going to be a little bit like COBOL programmers, I think. + +01:12:35 There's going to be places that are just like, look, we've just got to call some people that know how to write code and figure this out. + +01:12:41 Like this stuff, the crazy analogy, the crazy analogy is like, okay, this is what I read one. + +01:12:47 Like, so that like the U boats during German U boats during world war one have like a pre they have steel in them that is pre or it's not been affected by like nuclear, the nuclear bombs that were set off around, you know, during and after world war two. + +01:13:04 So like they will extract the steel from those because they're not affected by the radiation. + +01:13:10 I guess we're like the old U-boats. + +01:13:12 They're going to extract us from retirement and be like, you still know how to program + +01:13:17 in this esoteric language. + +01:13:18 The LLMs can't do it. + +01:13:19 I don't know. + +01:13:20 Maybe that's what's going to happen. + +01:13:21 Take the mothballs out. + +01:13:22 That's right. + +01:13:22 Yeah. + +01:13:22 And we're going back into service. + +01:13:25 Boy, it's an interesting time. + +01:13:27 And there's a lot of opportunity as you pointed out. + +01:13:30 So I want to leave people with a message of don't be scared, embrace this, see what it can do for you. + +01:13:35 There's a lot of opportunity here, but Matt, you get the final word. + +01:13:38 Yeah. + +01:13:39 I mean, I agree. + +01:13:39 I think it's, again, it's like the same philosophy we started with, which is like, I don't see this as that big of a shift compared to like a lot of the other stuff that's happened in industry. + +01:13:49 Like just, I would just learn these things. + +01:13:51 And frankly, like, then you are empowered to say like, I'm not going to use that because you've tried it. + +01:13:57 And so like, then you have an informed opinion on it. + +01:14:00 And I think that's really what matters as software developers, not to just dismiss something simply because it seems like a hype cycle, but actually just to try it. + +01:14:07 And if it doesn't work and you're like, well, I have an informed opinion. + +01:14:10 I will say, though, this is actually rapidly evolving. + +01:14:13 And as we've talked about through our conversation, I would actually try this multiple times over several months because it does get better and it does change with the tools. + +01:14:22 And so it's not just like I tried it once and I'm going to toss it away. + +01:14:26 I would give it a try every few months and see if it works for, you know, + +01:14:29 kind of clicks for you and kind of works for your use case. + +01:14:31 Yeah, if we had this conversation pre-agentic code agents. + +01:14:34 Yeah. + +01:14:35 Could be very, very different, but six months later. + +01:14:38 It's just like when open source was like nascent. + +01:14:42 Oh, I tried open source once. + +01:14:43 If you haven't tried it in 15 years, like, I don't know what to say, right? + +01:14:47 I think it's a similar thing, but it's just a little bit faster evolution. + +01:14:50 Yeah. + +01:14:51 Yeah, it's crazy times. + +01:14:52 I mean, I'm honestly thinking maybe we should ship this episode like this week without recording, not the two weeks out that it's scheduled to be recording because who knows what's going to happen. + +01:15:01 No, I'm just kidding. + +01:15:02 But they are changing fast. + +01:15:04 Well, thanks for coming back on the show. + +01:15:06 Always great to catch up with you. + +01:15:07 Thanks, Michael. + +01:15:08 It's been great. + +01:15:08 Yeah, bye. + +01:15:10 This has been another episode of Talk Python To Me. + +01:15:13 Thank you to our sponsors. + +01:15:15 Be sure to check out what they're offering. + +01:15:16 It really helps support the show. + +01:15:18 This episode is sponsored by Posit Connect from the makers of Shiny. + +01:15:22 Publish, share, and deploy all of your data projects that you're creating using Python. + +01:15:26 Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards, and APIs. + +01:15:33 Posit Connect supports all of them. + +01:15:35 Try Posit Connect for free by going to talkpython.fm/posit, P-O-S-I-T. + +01:15:41 Want to level up your Python? + +01:15:43 We have one of the largest catalogs of Python video courses over at Talk Python. + +01:15:47 Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:15:52 And best of all, there's not a subscription in sight. + +01:15:54 Check it out for yourself at training.talkpython.fm. + +01:15:58 Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:16:02 We should be right at the top. + +01:16:04 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +01:16:13 We're live streaming most of our recordings these days. + +01:16:16 If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:16:24 This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. + +01:16:28 Now get out there and write some Python code. + +01:16:42 *music* + diff --git a/transcripts/517-agentic-ai.vtt b/transcripts/517-agentic-ai.vtt new file mode 100644 index 0000000..1707a76 --- /dev/null +++ b/transcripts/517-agentic-ai.vtt @@ -0,0 +1,2312 @@ +WEBVTT + +00:00:00.120 --> 00:00:28.740 +Agentic AI programming is what happens when coding assistants stop acting like autocomplete and start collaborating on real work. In this episode, we cut through the hype and incentives to define agentic, then get hands-on with how tools like Cursor, CloudCode, and Langchain actually behave inside an established code base. Our guest, Matt McKay, now the Vice President of Developer Relations at DigitalOcean, the creator of FullStackPython, and PlushCap, shares hard-won tactics. + +00:00:29.140 --> 00:00:38.000 +We unpack what breaks from brittle, generate a bunch of test requests to agents amplifying technical debt and uneven design patterns. + +00:00:38.720 --> 00:00:43.020 +Plus, we also discuss a sane Git workflow for AI-sized diffs. + +00:00:43.660 --> 00:00:51.900 +You'll hear practical clawed tips, why developers write more bugs when they get less time programming, and where open source agents are headed. + +00:00:52.280 --> 00:00:58.120 +Hint, the destination is humans as editors of systems, not just typists of code. + +00:00:58.780 --> 00:01:03.840 +This is Talk Python To Me, episode 517, recorded July 29th, 2025. + +00:01:20.180 --> 00:01:23.120 +Welcome to Talk Python To Me, a weekly podcast on Python. + +00:01:23.700 --> 00:01:25.280 +This is your host, Michael Kennedy. + +00:01:25.400 --> 00:01:58.120 +Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is sponsored by Posit Connect from the makers of Shiny. Publish, share, and deploy all of your data projects that you're creating using Python. + +00:01:58.720 --> 00:02:04.780 +Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards, and APIs. + +00:02:05.780 --> 00:02:16.660 +Posit Connect supports all of them. Try Posit Connect for free by going to talkpython.fm/posit, P-O-S-I-T. Matt, great to have you back on the show. Welcome back to Talk Python. + +00:02:17.040 --> 00:02:19.380 +Thanks, Michael. Been a little while. Good to be back. + +00:02:19.420 --> 00:02:21.220 +It has been a little while. + +00:02:21.680 --> 00:02:32.920 +As I was entering your show details into the backend system for Talk Python to say what episodes are coming up in what order and so on, I have to enter the guest ID into that thing. + +00:02:33.200 --> 00:02:35.660 +Yes, it could be a drop-down list, but then there's so much scrolling. + +00:02:35.860 --> 00:02:36.500 +I just can't do it. + +00:02:37.500 --> 00:02:38.520 +And it's multi-select. + +00:02:38.980 --> 00:02:40.980 +So your guest ID is 29. + +00:02:41.440 --> 00:02:49.420 +We first did an episode together about Fullstack Python when I was sitting on the couch in Stuttgart, Germany, back when I lived there. + +00:02:49.470 --> 00:02:50.020 +How about that? + +00:02:50.140 --> 00:02:50.520 +Wow. + +00:02:50.940 --> 00:02:51.180 +All right. + +00:02:51.200 --> 00:02:52.040 +Yeah, it's been a while. + +00:02:52.360 --> 00:02:54.220 +I feel like that's like that credibility. + +00:02:54.820 --> 00:02:58.580 +You have that like only a double digit guest ID. + +00:02:59.020 --> 00:02:59.460 +Exactly. + +00:02:59.800 --> 00:03:01.620 +That's a low double digits. + +00:03:01.860 --> 00:03:02.020 +Yes. + +00:03:02.160 --> 00:03:05.700 +This is like a pro number in racing or something. + +00:03:05.830 --> 00:03:05.980 +Indeed. + +00:03:06.460 --> 00:03:08.060 +Well, that was the past. + +00:03:08.580 --> 00:03:10.700 +We talked about some awesome things then. + +00:03:11.180 --> 00:03:16.680 +The idea of full stack Python, I believe, was a lot of the focus, but it's honestly been 10 years. + +00:03:16.700 --> 00:03:19.100 +I don't remember exactly what we covered in detail. + +00:03:19.600 --> 00:03:23.920 +However, I can tell you, this is definitely not a backward looking episode. + +00:03:24.540 --> 00:03:25.600 +This is cutting edge stuff. + +00:03:25.980 --> 00:03:26.220 +Yeah. + +00:03:26.820 --> 00:03:29.920 +I mean, so much of this has changed over even three months. + +00:03:30.340 --> 00:03:40.040 +I mean, we'll talk about it all, but there's like the philosophy behind all this, the holy religious wars around what developers should be doing with it. + +00:03:40.780 --> 00:03:43.140 +And then let's dig into the details + +00:03:43.210 --> 00:03:44.140 +because that's the fun stuff. + +00:03:44.380 --> 00:03:44.680 +Absolutely. + +00:03:45.430 --> 00:03:45.540 +Absolutely. + +00:03:46.140 --> 00:03:52.080 +I would say it's certainly one of the most controversial topics in dev space these days. + +00:03:52.500 --> 00:04:05.640 +Well, it's almost like, if you remember when Django and Ruby on Rails came out and they were the new frameworks, even before all the JavaScript frameworks and everything like that, it was backend server, server-side code, and you could do so much more. + +00:04:05.730 --> 00:04:08.660 +I remember this was back when I was an early professional developer. + +00:04:08.790 --> 00:04:15.260 +I was working in Java with servlets in a framework called Tapestry and everything was so hard to do. + +00:04:15.540 --> 00:04:17.299 +And they tried to add so many layers of abstraction. + +00:04:17.320 --> 00:04:20.019 +It was like peak enterprise, Java development. + +00:04:20.160 --> 00:04:22.540 +How many layers of dependency injection were at play? + +00:04:22.600 --> 00:04:22.840 +Oh, yeah. + +00:04:23.140 --> 00:04:26.400 +And so then I would go home at night and I'm like learning Django. + +00:04:26.480 --> 00:04:34.380 +And I was like, I can't believe how much more I can get done in an hour than I did eight, nine, 10 hours a day working in the Java code. + +00:04:34.740 --> 00:04:42.600 +Not because there's literally nothing wrong with Java, but it was just that the frameworks were so well built for what we were trying to do. + +00:04:43.160 --> 00:04:50.740 +So that's the only parallel I personally have to like what is happening right now, which, and this is like 10X, 100X of what that is. + +00:04:50.900 --> 00:05:02.180 +It's like, if you use the tools properly for the right purposes at the right time period, because like a year ago versus today is very different, you can be wildly more productive. + +00:05:03.040 --> 00:05:04.040 +There's also the downsides, + +00:05:04.440 --> 00:05:09.760 +but there is substantial improvement to be had in certain areas of software development. + +00:05:09.960 --> 00:05:13.900 +And I feel like that is actually really the big takeaway. + +00:05:15.000 --> 00:05:16.320 +Among all the hype, break everything out. + +00:05:16.320 --> 00:05:26.440 +It's like, there's tactical things you can use this, agentic tools for LLMs that will 10X, 100X certain things that are super annoying about software development right now + +00:05:26.800 --> 00:05:28.420 +in a way that was previously impossible. + +00:05:28.800 --> 00:05:30.160 +Right. Summarize my architecture. + +00:05:30.600 --> 00:05:38.160 +Have you tried that compared to like, I have a hundred thousand lines of code and I've got to like study the crap out of it just to figure out what piece connects to what. + +00:05:38.260 --> 00:05:40.160 +I love the roast my code. + +00:05:40.560 --> 00:05:43.380 +Like, don't just tell me about it. + +00:05:43.560 --> 00:05:44.860 +Like, give it to me straight here. + +00:05:45.140 --> 00:05:48.000 +Like, how could this function be improved? + +00:05:48.480 --> 00:05:49.040 +Roast it? + +00:05:49.460 --> 00:05:51.580 +Okay, now tell me how I can improve it. + +00:05:51.880 --> 00:05:51.980 +Okay. + +00:05:53.600 --> 00:05:57.700 +And at the end, just give me a little bit of like ego boost because I need it after all the roasting. + +00:05:58.240 --> 00:05:58.820 +Yes, exactly. + +00:05:59.540 --> 00:06:01.100 +Well, it's like, okay, you roast me. + +00:06:01.200 --> 00:06:06.520 +Anytime you talk me down, Just include a little spirit lifting comment as well. + +00:06:06.820 --> 00:06:08.880 +Well, the best part is like, okay, roast me. + +00:06:09.060 --> 00:06:11.300 +Okay, if you're so smart, go fix it for me. + +00:06:11.920 --> 00:06:13.100 +Yeah, you're like, oh, that's right. + +00:06:13.620 --> 00:06:14.040 +It did work. + +00:06:14.180 --> 00:06:15.000 +Sometimes it does actually. + +00:06:15.120 --> 00:06:15.220 +Sometimes. + +00:06:16.380 --> 00:06:16.940 +Which is insane. + +00:06:17.360 --> 00:06:28.480 +Yeah, I think probably the best way I could say we should frame, people should frame this conversation to at least to get started with in their mind is a balanced take, right? + +00:06:28.620 --> 00:06:30.800 +It's not necessarily vibe coding. + +00:06:31.040 --> 00:06:35.860 +while that's hilarious and has spawned many amazing YouTube videos, right? + +00:06:37.300 --> 00:06:42.460 +But it's also, it doesn't necessarily make sense to imagine that these tools don't exist. + +00:06:43.180 --> 00:06:46.680 +And just out of principle say, well, I'm never going to touch them + +00:06:47.120 --> 00:06:49.700 +because I'm not going to replace coders with this. + +00:06:49.860 --> 00:06:54.320 +It's like saying, I was just playing a bunch with rough, you know, the formatting tool, right? + +00:06:54.660 --> 00:06:59.640 +And it's ripping through 500 files at a time on these projects I'm working on. + +00:06:59.960 --> 00:07:04.320 +Ruff format, rough check fix, bam, 37 files repaired. + +00:07:04.640 --> 00:07:09.020 +I could have done that by hand and flexed my dev skills, but should I? + +00:07:09.260 --> 00:07:10.280 +No, of course I shouldn't. + +00:07:10.360 --> 00:07:10.660 +It takes microseconds. + +00:07:10.660 --> 00:07:12.220 +Would that be the best use of your time? + +00:07:12.480 --> 00:07:14.120 +It certainly does not. + +00:07:14.320 --> 00:07:19.280 +And so if these tools exist, well, then I think it comes down to like, well, what is the right way to use them? + +00:07:19.520 --> 00:07:20.260 +How do they go well? + +00:07:20.320 --> 00:07:21.120 +How do they go poorly? + +00:07:21.520 --> 00:07:21.680 +Yeah. + +00:07:21.900 --> 00:07:22.680 +Are they too expensive? + +00:07:23.060 --> 00:07:23.480 +That's spot on. + +00:07:23.540 --> 00:07:34.760 +I mean, what I find fascinating about this shift is like most professional developers have spent their entire adult lives just like learning new technologies. + +00:07:35.240 --> 00:07:37.060 +I actually don't know why this is any different. + +00:07:37.580 --> 00:07:40.400 +Like the whole point of being a developer + +00:07:40.760 --> 00:07:45.200 +is like keeping an open mind, an open philosophy of like, what can I learn? + +00:07:45.620 --> 00:07:46.760 +How should I be learning? + +00:07:47.580 --> 00:07:59.780 +Just that is what appeals to me so much about being a software developer is like this mindset of I go into this year not even knowing what tools I may be using by the end of the year because they may not exist yet. + +00:08:00.180 --> 00:08:04.640 +And I want to go learn them. And some of them I am going to throw immediately in the wastebasket + +00:08:05.080 --> 00:08:40.140 +and other ones are going to stick with me for potentially decades. And in fact, when it comes to LLMs, I had put out a post on X and I was like, I feel like Olama is one of those tools that I've added to my tool belt, like Django or Vim or Teamux that will be with me potentially for a couple of decades because it is just so straightforward and it makes it so easy to use these open-weighted models. And so that's incredible. 18 months ago, I hadn't even heard of this tool. And actually, it didn't exist before, I don't know, 18, 24 months ago. + +00:08:40.719 --> 00:08:53.920 +And here's something that I'm like, wow, this is going to stick with me. So I think that's what's maybe getting lost in a little bit of the hype cycle is we're developers. All we do is learn new stuff. Why is this any different from learning an open source project that you just found on + +00:08:54.040 --> 00:09:15.120 +GitHub? A hundred percent. A hundred percent. I'm definitely here with you on that. I'd see in other areas like education, I see it, you know, like write this essay for me. That's very problematic if you're in 10th grade and you're supposed to learn how to write an essay, but your job is to create software that works, add features, make it awesome. Like there's not a test. Yeah. + +00:09:15.880 --> 00:09:16.880 +Other than shipping. + +00:09:45.220 --> 00:09:54.740 +you if you're not a senior enough developer you're not experienced enough or you're not willing to dig deep enough you suddenly are stopped there's nothing you can do from there you're like no fix + +00:09:54.900 --> 00:09:58.380 +it fix it it doesn't matter how many times you tell the lom to fix it or the tool to fix it please + +00:09:58.660 --> 00:10:04.280 +or you go to jail yeah exactly yes there's people on the train tracks you must fix this right now + +00:10:05.679 --> 00:10:10.639 +my grandma's life depends upon it right you gotta make this database query work yeah well + +00:10:11.360 --> 00:10:38.800 +I think one thing that maybe is preventing people from embracing some of this stuff is, I actually don't even think that a lot of the terminology is clear. So if you'll allow me to be pedantic for a minute, I actually think that this is often the most insightful thing that I work with people on or tell people about, which is just like, what is the difference between an AI model, an LLM, an AI agent, and some of these other things? Because actually, people use these interchangeably. + +00:10:38.900 --> 00:10:43.940 +People will say Gen AI, generative AI, when they mean an agent or vice versa or an LLM. + +00:10:44.320 --> 00:10:48.700 +These are not drop and replace terminology for each other. + +00:10:48.730 --> 00:10:49.940 +They have very specific meanings. + +00:10:50.050 --> 00:11:02.660 +And I think particularly when I've seen the traditional software companies try to all of a sudden AI all the things, this is part of why developers get so annoyed with the AI ecosystem. + +00:11:02.790 --> 00:11:07.360 +Because it's like saying Django is not a... + +00:11:07.490 --> 00:11:08.520 +You wouldn't say web framework. + +00:11:08.760 --> 00:11:11.000 +you would say like, oh, it's an AI agent coding tool. + +00:11:11.040 --> 00:11:11.880 +It's like, no, it's not. + +00:11:12.020 --> 00:11:12.640 +What are you talking about? + +00:11:12.760 --> 00:11:14.580 +Immediately you lose your credibility, right? + +00:11:14.720 --> 00:11:19.480 +It's an AI generative web framework builder. + +00:11:20.140 --> 00:11:20.620 +Right, right. + +00:11:20.620 --> 00:11:21.460 +Thank goodness it's not. + +00:11:21.560 --> 00:11:26.720 +Thank goodness Django, the creators and the maintainers know what they're good at. + +00:11:26.740 --> 00:11:29.440 +And they're not trying to be a part of the hype cycle. + +00:11:30.560 --> 00:11:37.280 +Ironically, I think Simon Wilson, one of the original Django folks, is one of the leading people in AI. + +00:11:38.060 --> 00:11:39.520 +I've learned so much from his writing. + +00:11:39.600 --> 00:11:40.100 +Yeah, absolutely. + +00:11:40.460 --> 00:11:40.560 +It's unreal. + +00:11:40.560 --> 00:11:40.800 +It's amazing. + +00:11:41.100 --> 00:11:42.840 +I, I didn't think that, go ahead. + +00:11:42.880 --> 00:11:42.960 +Yeah. + +00:11:43.270 --> 00:11:46.020 +I don't know how, I don't know how, I don't understand how he can be so prolific. + +00:11:46.160 --> 00:11:46.920 +He is incredible. + +00:11:47.240 --> 00:11:54.720 +Like he's, he's just like a true gift, to software development, having been on the leading edge of so many things. + +00:11:54.940 --> 00:11:55.240 +It's amazing. + +00:11:55.600 --> 00:11:56.120 +Yeah, absolutely. + +00:11:56.600 --> 00:12:04.380 +So if I could just lay out real quick, maybe for, for folks like a, an AI model was an AI model is typically when people are talking about that. + +00:12:04.390 --> 00:12:13.140 +Now, again, like there can be a little bit of nuance and gray areas with some of these definitions, but typically an AI model is something that is trained on some sort of training set. + +00:12:13.840 --> 00:13:52.740 +And it's been typically trained on the transformer architecture, which is a fairly recent, last eight years kind of development. And so this is really what's allowed the breakthrough when we talk about AI now versus AI a couple of decades ago when it was like complete AI winter and no one wanted to talk about it, is we've had this breakthrough in architecture and it's not just like one breakthrough. It is a tremendous number of breakthroughs around attention and the way that things are weighted and things like that. But essentially, to boil that down, you have these AI models. Now, an AI model is not equivalent to a large language model, an LLM. An LLM is one type of AI model. So when I was at Assembly AI, they were training or they are training state-of-the-art speech-to-text models is not an LLM. So a lot of times people will say like AI model LLM, but those are not equivalent. An AI model is a superset of the subset, which is like an LLM, which is one type of typically working on text modality. Although there are things called multimodal models, which could have different image inputs and image outputs and text inputs, text outputs, and stuff like that. But I think that's one thing where a lot of companies and maybe even developers who are learning about the space get confused. It's like, is an AI model and LLM are the exact same thing? No, there's a relationship there, but they're not the same. So then you have like generative AI, which a lot of companies just kind of like sprinkle that into everything that they talk about. It's like generative AI is really using AI models, typically LLMs, but also image generation and some other forms to create some sort of output. So it's the generative component of it. So you have some sort of input and then there's a non-deterministic set of outputs to come out the other side. + +00:13:52.780 --> 00:13:57.640 +So it'll be, for example, like draw me a red dragon breathing fire for the image generation. + +00:13:57.920 --> 00:14:03.820 +And that generative AI is basically using an AI model to produce that image out the other side. + +00:14:04.100 --> 00:14:06.460 +So those are some of the common terms. + +00:14:06.640 --> 00:14:16.100 +And then you have AI agents, which is a lot of what we talk about or we're going to talk about, which is it is using typically an LLM, but it's typically using some sort of AI model. + +00:14:16.440 --> 00:14:18.540 +And that is kind of almost think about it as like the core. + +00:14:19.060 --> 00:14:22.880 +There's inputs into the system and non-deterministic outputs that come out the other side. + +00:14:22.900 --> 00:14:35.520 +So you'll say something like, write me a bunch of unit tests and in Cloud Code or in Cursor or in Windsurf, and then it will interpret those inputs and then produce code or some sort of output out the other side. + +00:14:35.700 --> 00:14:40.400 +So I think for developers who are trying to get into, how do I even understand kind of the AI space? + +00:14:40.700 --> 00:14:53.240 +It's actually really important to get that terminology correct, because otherwise you won't even know when you're reading, particularly for companies or people that aren't as familiar with the space, like what they're even talking about. + +00:14:53.700 --> 00:15:00.420 +So I always like to kind of like- A lot of the companies, yeah, their job, it's to their benefit to obscure + +00:15:00.420 --> 00:15:01.280 +and make it just seem like it's everything. + +00:15:01.320 --> 00:15:03.420 +Yeah, they want to seem like- + +00:15:03.760 --> 00:15:05.020 +Yeah, totally. + +00:15:05.020 --> 00:15:13.300 +There's financial incentives by some companies to like obscure what they're doing and make it seem much more complicated so that they can sell this solution. + +00:15:13.300 --> 00:15:17.380 +Oh, it takes this really complicated problem and streamlines it down to like some sort of simple solution. + +00:15:17.380 --> 00:15:18.700 +And that's like often not the case. + +00:15:18.760 --> 00:15:22.500 +Like when you peel it back as a developer, like, you're not really doing that. + +00:15:22.700 --> 00:15:22.800 +Right. + +00:15:23.420 --> 00:15:31.960 +So I think that's often where I think a lot of developers would get frustrated with the current state of the industry because you're like, no, it's not what you're doing. + +00:15:32.380 --> 00:15:34.480 +You know, like you're not, you're not doing generative AI. + +00:15:34.700 --> 00:15:40.380 +You're not doing actual agents because agents are, you know, and then there's like autonomous agents, which are like operating independently. + +00:15:40.760 --> 00:15:49.060 +So that's one thing I think developers can like take away from the conversation is just like, is the company accurately describing what they are doing? + +00:15:49.360 --> 00:15:55.300 +Yeah, I a hundred percent agree. And let me do a little sidebar rant and I'd love to get your + +00:15:55.480 --> 00:16:01.780 +thoughts on this. Okay. Yeah. So when I hear people say, I've tried AI, it's a bunch of AI + +00:16:02.100 --> 00:16:19.740 +slop. It just makes up a bunch of mistakes. I noticed a couple of things that are often the case when I hear those and have people avoid that, right? So one, a lot of times I feel like people are not getting the best experience. They say like, I tried this. It really wasn't for me. + +00:16:20.100 --> 00:17:36.900 +It just messes up more than it provides. They're using the cheapest free models that they can find, right? If you use, you know, Cloud's Opus model versus some free model, you know, like a 3 billion parameter local model they're not even in the same category the type of accuracy and like insight and like the context do they they understand everything rather than well they only understand the last file they read like that that is like one half of the problem and then i think the other for people who are not being successful yet with this kind of stuff it has to do with not providing enough information and context and stuff in the prompt so i'll see like oh refactor this function it's like well, hold on. What, where, where do you even want it to go? Right. It's just going to like, start randomly doing stuff. Here's a function. It is similar to these. And I need to move this to this kind of design pattern, keeping in mind that I'm using this ORM and like, give it like the same amount of description you would to a junior developer who's not super familiar with your project. So you want to give that thing like a possibly a multi-page write-up of what it needs to do and then ask it to plan it out and then start working through the plan, not just refactor + +00:17:36.930 --> 00:17:39.400 +to be, you know, to do this or whatever. + +00:17:39.500 --> 00:17:42.100 +And I think those two things are both problems. + +00:17:42.430 --> 00:17:48.060 +They often go together because the cheaper models can't accept that kind of information and keep it working. + +00:17:48.600 --> 00:17:48.660 +Right. + +00:17:48.780 --> 00:17:48.960 +Yes. + +00:17:49.160 --> 00:17:49.960 +What do you think about that? + +00:17:50.200 --> 00:17:51.460 +I mean, I totally agree. + +00:17:51.640 --> 00:17:56.660 +I think also to like these models, you cannot just like mix and match and replace things. + +00:17:56.820 --> 00:18:20.560 +So you may have an experience with one model and also recognize that these models, even though Claude for Opus is the public name of that model, they are tweaking and tuning that model on the backend to figure out how they can more, maybe not profitably, but serve this model up at scale in a window of resources. + +00:18:20.700 --> 00:18:29.040 +So even within a given model, unless it is an open weighted model that you've downloaded, like you have under complete control, these models are changing over time. + +00:18:29.540 --> 00:18:35.520 +So this is like, I think that's actually been one of the most concerning parts for me as a developer. + +00:18:35.560 --> 00:18:39.040 +I was like, what if I rely on a tool and it changes? + +00:18:39.620 --> 00:18:40.440 +And I wake up one day. + +00:18:40.500 --> 00:18:41.320 +The tests passed last week. + +00:18:41.420 --> 00:18:42.220 +They don't pass this week. + +00:18:42.220 --> 00:18:42.280 +Right. + +00:18:42.560 --> 00:18:43.600 +I have no control or visibility. + +00:18:43.620 --> 00:18:43.740 +Right. + +00:18:44.060 --> 00:18:48.240 +I was productive last week or I was productive yesterday and I can no longer be productive today. + +00:18:49.140 --> 00:18:51.580 +And that's setting aside any downtime or API stuff, right? + +00:18:52.419 --> 00:19:03.180 +So I think the thing is, that's why I very much appreciate the vibe check, the whole concept of a vibe check, which is you get a new model or get access to a new API. + +00:19:03.820 --> 00:19:06.540 +And what are the individual components that you want to test? + +00:19:06.940 --> 00:19:11.760 +So even to your example of refactor, here's the ORM, that sort of thing. + +00:19:11.980 --> 00:19:16.420 +I started very, when I have a model, and I use a lot of cloud code. + +00:19:16.440 --> 00:19:19.160 +I use a lot of Opus and so on now, but I'm sure this will evolve. + +00:19:19.310 --> 00:19:22.000 +I would love to try it with some open weighted models soon. + +00:19:22.540 --> 00:19:33.760 +And I will say something like in this URLs.py, because I work a lot with Django on the backend, I'll say update just the, I'm trying to add a new page. + +00:19:34.320 --> 00:19:35.340 +It's going to be at this route. + +00:19:35.740 --> 00:19:39.880 +Here is the, please update the URLs.py file for me. + +00:19:40.090 --> 00:19:41.520 +And like, here's roughly what I expect, right? + +00:19:42.000 --> 00:19:42.760 +Super specific. + +00:19:42.950 --> 00:20:09.040 +And if it can't even get that part right, the chances of you saying, okay, now build a new page, now build a new view, now do X, Y, and Z are very small. So you kind of have like the smallest unit that you would work with and then start building up. And it is a two-way street. Like you have to build confidence in that model over time. Like what are its true capabilities? And I will say like it's a lot of work, but it's a lot of work being a software developer and learning a new open source project. + +00:20:09.440 --> 00:20:14.360 +So it's actually not that different from just like, okay, pip install, new library, reading the + +00:20:14.320 --> 00:20:19.420 +documentation, all those things. But it's a different format of going back and forth with + +00:20:19.440 --> 00:20:28.700 +the computer. I think it encourages people to be less serious because it feels like a little chat, a little back and forth. Just, hey, I just asked a little question and it came with a cool answer. + +00:20:28.960 --> 00:20:41.879 +It doesn't speak, you need to study this and you need to really, this is a skill. It's like I'm having just a chat with someone you know or whatever. Well, it does open up, even if you set + +00:20:41.900 --> 00:21:22.440 +aside code generation. It opens up new capabilities that I would argue are useful for every single developer, regardless of how soon you are. So just being able to run Claude and in plan mode, ask it questions about like, where might I have security vulnerabilities in my code? Or where could I refactor database queries in order to compress the number of database queries within this view or where could I add caching that would be most impactful and don't even have it touch your code other than to read the code. That was actually like my big kind of breakthrough with LLMs was I was like, I'm just going to use them in read only mode. I don't need them to modify my code for me. + +00:21:22.560 --> 00:21:40.860 +I'm comfortable doing that myself as a developer. But once I got confident in the models being able to read the code, I was kind of like, eh, just like dip my toe in the water, like maybe modifying some things. And especially in Python, I've read a lot of scripts. I'm like updating data and I just, I don't know. It's not my favorite part of coding. So having models that can write for me, + +00:21:41.030 --> 00:21:52.640 +even if it's not perfect, and then I can modify it. I need to export this from the database in that format. And if it works at all, it's perfect. If it's not going to work at all, right? It's a little risk. Yeah. And an export from a database is, + +00:21:53.280 --> 00:22:08.540 +as long as you're not accidentally dropping a table, it is just a read-only, kind of like, tell me about my code. There's all these things out there that in software development are like, you have to do the analysis yourself, but if you can shortcut it with an LLM, that actually seems like a big win to me. + +00:22:08.960 --> 00:22:11.320 +And I don't actually see any downside to doing that. + +00:22:11.520 --> 00:22:17.640 +Like if, again, if it's, if it's a read only and you're not touching the code, that to me feels like only a win. + +00:22:17.900 --> 00:22:22.200 +It was, it's time that I've saved that otherwise I would have to invest myself. + +00:22:23.860 --> 00:22:27.100 +This portion of Talk Python To Me is brought to you by the folks at Posit. + +00:22:27.660 --> 00:22:30.780 +Posit has made a huge investment in the Python community lately. + +00:22:31.440 --> 00:22:36.120 +Known originally for RStudio, they've been building out a suite of tools and services for Team Python. + +00:22:36.980 --> 00:22:41.940 +Today, I want to tell you about a new way to share your data science assets, Posit Connect Cloud. + +00:22:42.940 --> 00:22:48.160 +Posit Connect Cloud is an online platform that simplifies the deployment of data applications and documents. + +00:22:48.740 --> 00:22:51.340 +It might be the simplest way to share your Python content. + +00:22:51.980 --> 00:22:53.720 +Here's how it works in three easy steps. + +00:22:53.990 --> 00:22:57.780 +One, push your Python code to a public or private GitHub repo. + +00:22:58.460 --> 00:23:02.020 +Two, tell Posit Connect Cloud which repo contains your source code. + +00:23:02.520 --> 00:23:03.520 +Three, click Deploy. + +00:23:04.020 --> 00:23:04.340 +That's it. + +00:23:04.740 --> 00:23:10.200 +Posit Connect Cloud will clone your code, build your asset, and host it online at a URL for you to share. + +00:23:10.840 --> 00:23:15.560 +Best of all, Posit Connect Cloud will update your app as you push code changes to GitHub. + +00:23:16.300 --> 00:23:22.480 +If you've dreamed of Git-based continuous deployment for your projects, Posit Connect Cloud is here to deliver. + +00:23:23.060 --> 00:23:25.900 +Any GitHub user can create a free Posit Connect Cloud account. + +00:23:26.280 --> 00:23:28.760 +You don't even need a special trial to see if it's a good fit. + +00:23:29.140 --> 00:23:34.300 +So if you need a fast, lightweight way to share your data science content, try Posit Connect Cloud. + +00:23:34.980 --> 00:23:40.600 +And as we've talked about before, if you need these features, but on-prem, check out Posit Connect. + +00:23:41.320 --> 00:23:44.400 +Visit talkpython.fm/connect-cloud. + +00:23:45.180 --> 00:23:46.100 +See if it's a good fit. + +00:23:46.600 --> 00:23:49.420 +That's talkpython.fm/connect-cloud. + +00:23:49.700 --> 00:23:51.680 +The link is in your podcast player's show notes. + +00:23:52.260 --> 00:23:54.280 +Thank you to Posit for supporting Talk Python To Me. + +00:23:55.520 --> 00:23:58.100 +Let me give people an example of something I recently did. + +00:23:58.290 --> 00:24:03.200 +And I want to kind of tie this back to like really emphasize the agentic side of things. + +00:24:03.660 --> 00:24:09.320 +So I have, I just checked, I have 530 GitHub repositories on my hard drive. + +00:24:10.300 --> 00:24:11.220 +I have three computers. + +00:24:11.490 --> 00:24:16.580 +I have like this streaming course live recording computer that we're talking on now. + +00:24:16.780 --> 00:24:17.740 +M2 Mac mini. + +00:24:17.990 --> 00:24:20.660 +I have an M4 Mac mini, which I usually work on, but I also have my laptop. + +00:24:21.120 --> 00:25:25.140 +And so every now and then, usually I'm on top of things, but every now and then I'll up a project and i'll start making changes i'm like oh i forgot to do git pull oh no oh this might be this might be bad you know especially if it's like a binary file like a powerpoint that i'm going to use for a course it's like there's no fixing it right you just yeah put them side by side and rename one and copy it over but it's it's not great and so i was sitting at my my um kitchen table on my laptop about to take my daughter to school and i opened up cursor with cloud sonnet in agentic mode and i described what i want i said what i would like is for you to create a script that i can run as a single command that will recursively find all the git repositories in a certain whatever folder i'm in downward find that do a git pull on every single one of them and report out which ones have changed. I drove my daughter to school, which is five minutes, one way, five minutes back. I sit down. The thing was there. It had created little test directories. + +00:25:25.460 --> 00:25:29.800 +It ran it. It said, it looks like it's totally good. And I literally just now have that as a + +00:25:29.960 --> 00:25:33.500 +utility. I can just, when I sit down, I'm like, oh, it's been like a week since I worked on + +00:25:33.550 --> 00:26:05.780 +security. Just get refreshed and just give it a moment. And then boom, you can see a report of all the stuff that's changed across all the parts. Even on something amazing is like chat CPT. You agentic. So maybe like, let's, that's a long winded way of like saying, tell us about the magic of like, what is this agent and tool using aspect for it? Cause I think when I said my rant, it's not working for people. I think a lot of times people are not doing agentic AI. They're asking LLM to write functions or stuff, which is great, but it's not the same. Yeah. Well, I think, I would say + +00:26:05.860 --> 00:26:18.280 +like kind of the biggest thing, and I mean, there's like, there's multiple attributes to it. Like, again, going back to some of the definitions, it's like you have an LLM and that LLM, if it's running chatGPT on your web browser, it's not going to have access to all the stuff in your codebase. + +00:26:19.360 --> 00:26:26.800 +Unless you have a public GitHub repository or something. But generally, when you're working in your local development environment, it's not going to have access to that stuff. + +00:26:28.040 --> 00:26:46.540 +To be fair, there are some tools that will take enough context and upload it to chatGPT. But again, what you're starting to do is you're starting to get really far away from a natural workflow and into one in which you have to bend to how the LLM is set up. + +00:26:46.760 --> 00:26:53.480 +So to me, I think the simplest way that I would look at it as agentic is like, it's something that is running side by side with you. + +00:26:53.760 --> 00:27:04.000 +It's almost like, I don't want to say copilot because copilot has overloaded terms with like kind of GitHub copilot, but it is essentially like, think about it almost as like pair programming with like more junior developers. + +00:27:04.760 --> 00:27:06.100 +Other people have used that analogy. + +00:27:06.480 --> 00:27:10.440 +It's not perfect, but it's kind of the one that I have that kind of sticks with me. + +00:27:10.440 --> 00:27:11.960 +It's as close as we got, I think, honestly. + +00:27:11.980 --> 00:27:13.100 +It's as close as we got, right? + +00:27:13.100 --> 00:27:16.160 +You can tell it to go do stuff, and it's not just going to only read. + +00:27:16.400 --> 00:27:17.300 +It'll look around. + +00:27:18.419 --> 00:27:19.220 +It'll experiment. + +00:27:19.640 --> 00:27:20.340 +It'll try something. + +00:27:20.500 --> 00:27:21.240 +It'll see it's wrong. + +00:27:21.340 --> 00:27:21.920 +It'll try something. + +00:27:22.900 --> 00:27:23.760 +It's kind of independent. + +00:27:24.040 --> 00:27:35.740 +And I think the easiest one for a lot of folks to have tried, because it was the one that I frankly got the most comfortable with at first, was in Cursor, you have which is essentially VS Code. + +00:27:36.020 --> 00:27:38.240 +So, you know, cursor is like embedded within that. + +00:27:38.300 --> 00:27:45.040 +And then you have different ways of interacting with like an agent mode where you're just asking to do stuff, right? + +00:27:45.120 --> 00:27:49.960 +Like asking stuff about the code base or please to write this function, whatever it is. + +00:27:50.260 --> 00:27:52.940 +So I do think that like that works for some folks. + +00:27:53.540 --> 00:27:55.300 +For me, it was not kind of the light bulb moment. + +00:27:55.300 --> 00:28:05.080 +It was kind of like where I started initially using it was I would, if I was like, oh, I need to like use this new API or I need to like kind of like develop a script or whatever it is. + +00:28:05.160 --> 00:28:12.400 +It was kind of like my, because I don't normally work in VS Code, I'm like a Vim, Tmux, like for a long time, like that's kind of my natural environment. + +00:28:12.610 --> 00:28:14.980 +And I was never going to like kind of adapt my own workflow. + +00:28:15.090 --> 00:28:16.340 +And I think a lot of people are like that, right? + +00:28:16.390 --> 00:28:19.520 +Like you're in JetBrains or you're in Xcode or whatever it is. + +00:28:19.930 --> 00:28:28.220 +You don't, you, you, what there's a breakthrough for you is like to build it in a way to your work, your development workflow that just is, is natural. + +00:28:28.590 --> 00:28:33.999 +And I think that's kind of, not the canonical definition, but to me is kind of most representative + +00:28:34.020 --> 00:29:10.120 +of kind of like agentic programming is like it it's just completely a part of your workflow and you don't have to adapt so um so again like that that cursor mode is kind of like okay i'd use it for like one-off stuff for me the breakthrough was using was using cloud code and i'll actually instead of talking about cloud code i'll say here was the bet that i made my feeling was there was enough value from cloud code uh based off of what i've been the videos that I've been watching, what I've been reading, and a lot of folks have been using it, that I was like, I'm a little worried about getting too reliant upon this tool, which is an API. + +00:29:10.560 --> 00:29:12.360 +And I will give Anthropic a lot of credit. + +00:29:12.540 --> 00:29:24.120 +They have done a lot of good work on these models, but I've also used Anthropic stuff in the past, which either had bad uptime or they were changing the model too frequently on the backside or things like that, that I was worried. + +00:29:24.560 --> 00:29:29.320 +And I think that's a natural thing as a software developer, should I really use this API? + +00:29:30.000 --> 00:29:38.900 +So I was like, my bet was, okay, even if I use this and it's amazing, if it goes away, I still need to be comfortable just like moving back to my workflow. + +00:29:39.460 --> 00:29:42.100 +Or there could be open weighted models in the future. + +00:29:42.350 --> 00:29:52.000 +Like I can just run Olama and I could run either Cloud Code or Open Code or some sort of CLI that would allow me to just do the same things, right? + +00:29:52.180 --> 00:29:54.280 +I mean, may not exactly, but roughly the same things. + +00:29:54.680 --> 00:29:55.800 +So that was kind of the bet that I made. + +00:29:55.900 --> 00:30:01.020 +That was like the philosophical mindset shift for myself was I'm already very comfortable as a software developer. + +00:30:01.470 --> 00:30:04.500 +Let me add this in a way that doesn't break my coding workflow. + +00:30:04.570 --> 00:30:08.040 +I'm not adapting to these tools in a way that is unnatural. + +00:30:09.030 --> 00:30:22.480 +And then I will use them in ways that I feel like are actually going to be productive as opposed to forcing them to be used in ways that almost like all the hype cycle and the news media is talking about, you will never write another line of code again. + +00:30:22.720 --> 00:30:24.700 +I don't actually really believe that is true. + +00:30:25.170 --> 00:30:25.660 +I don't know. + +00:30:26.200 --> 00:30:44.800 +I feel like anybody who's saying that is not actually using these tools. And I actually don't think it's going that direction. So I don't know that that kind of sets the stage rather than like how I use the tools, like what was going through my mind as a software, as an experienced software developer of like, should I even make this shift? I don't know if that if you had to undergo that + +00:30:45.080 --> 00:30:56.720 +as well. I did. And I guess I haven't been as introspective about it as you. But for me, The real shift for me was I'm already a good coder. + +00:30:57.060 --> 00:30:59.300 +I feel very competent flying around. + +00:30:59.440 --> 00:31:03.200 +Our tool chain, I know yours and mine is like kind of quite different, right? + +00:31:03.300 --> 00:31:04.640 +You're very Tmux based. + +00:31:04.980 --> 00:31:10.540 +I'm very much often in PyCharm jumping around there, but we all are very fluid and capable, right? + +00:31:10.620 --> 00:31:13.640 +So you're feeling like this is like, I'm really productive and competent. + +00:31:13.840 --> 00:31:17.260 +And the last thing I want to do is just like spend my days in a chat engine. + +00:31:17.680 --> 00:31:18.220 +You know what I mean? + +00:31:18.740 --> 00:31:18.900 +Right. + +00:31:19.000 --> 00:31:20.360 +Like that is, that is not it. + +00:31:20.700 --> 00:31:28.140 +And I guess one of the main things, I sat down with a friend of mine, we were out having beer and he had his laptop, a friend named Mark, and he had his laptop. + +00:31:28.250 --> 00:31:31.260 +I said, well, let's just, man, I've been doing some really cool stuff with Agenda. + +00:31:31.460 --> 00:31:33.140 +Let me just, let's just do an example. + +00:31:33.720 --> 00:31:42.660 +And over like a beer, we built the most amazing app that would literally take weeks or a week at least, you know, knowing what you're doing. + +00:31:42.950 --> 00:31:45.940 +I got not, not, I'm going to learn these frameworks and then do it. + +00:31:46.220 --> 00:31:50.160 +But even with, like that actually changes my perspective. + +00:31:50.620 --> 00:32:00.880 +Did it use open source libraries and frameworks in order to, like, did the tool pull in a bunch of stuff and it actually stood up and built this application off of all the open source code, right? + +00:32:01.060 --> 00:32:01.200 +Yes. + +00:32:01.400 --> 00:32:02.220 +And it was ridiculous. + +00:32:02.840 --> 00:32:09.140 +It was a FastAPI app using SQLAlchemy to talk to SQLite. + +00:32:09.430 --> 00:32:12.120 +The web design was done in Bulma CSS. + +00:32:12.780 --> 00:32:18.060 +It used simple MDE for live markdown editing. + +00:32:18.680 --> 00:32:20.700 +And it just, it kept going. + +00:32:21.160 --> 00:32:32.860 +And one of the things I think is really interesting, and I know you have some thoughts on this as well, is it's the more you work with like new fancy HIP frameworks, the less well off you are, right? + +00:32:33.060 --> 00:32:37.340 +Like I could have said Tailwind, but Tailwind's got all these build steps and all these little nuances. + +00:32:37.820 --> 00:32:39.140 +Bulma is just include a CSS. + +00:32:39.720 --> 00:32:43.760 +And so it didn't have, that was like a thing it didn't have to worry about and so on, you know what I mean? + +00:32:43.900 --> 00:32:44.040 +Yeah. + +00:32:44.380 --> 00:32:46.080 +So I find it's like, + +00:32:46.200 --> 00:32:51.920 +I find myself trending towards more simple code than more towards frameworks, which is the opposite. + +00:32:52.230 --> 00:32:52.500 +I think. + +00:32:52.640 --> 00:32:56.400 +Yeah, well, there's I'll even take it a step further, which is I very much appreciate. + +00:32:56.800 --> 00:32:59.940 +I very I agree with you on the like less build steps. + +00:33:00.440 --> 00:33:02.860 +And in fact, like the simpler is often better. + +00:33:02.950 --> 00:33:07.500 +But I will say that I very much appreciate the opinionated tools and frameworks. + +00:33:07.920 --> 00:33:15.060 +And that's why I've actually had a better experience using Cloud Code with Django. + +00:33:15.660 --> 00:33:27.460 +And a big piece is also, I've written thousands, if not tens of thousands of lines of code already in Plush Cap, which is typically what I'm building, if not some side scripts and stuff like that. + +00:33:28.060 --> 00:33:29.420 +And I will have- + +00:33:29.420 --> 00:33:30.640 +Let's do a really quick sidebar + +00:33:30.640 --> 00:33:33.700 +and let you introduce those projects. + +00:33:33.700 --> 00:33:35.180 +Just because I know it's really nice + +00:33:35.180 --> 00:33:40.680 +to be able to reference back and say, when I was adding this feature or this is the, so give people something concrete, you know, + +00:33:40.760 --> 00:33:43.180 +like tell us about full stack Python and Plush Cap. + +00:33:43.420 --> 00:33:44.280 +- So this is full stack Python. + +00:33:44.510 --> 00:33:46.180 +I wrote full stack Python. + +00:33:46.300 --> 00:33:59.180 +It's like over 150,000 words, all about the Python ecosystem, ranging from like the Python language itself to web frameworks, to deployment options, and content delivery networks, APIs, all sorts. + +00:33:59.480 --> 00:34:05.540 +So it was kind of like all the things around Python and it was broken down by conceptual ideas. + +00:34:05.800 --> 00:34:16.060 +So like data visualization and implementations, which I feel like is something that is not particularly obvious to people that are learning to program is like you have conceptual ideas and those have specific implementations. + +00:34:16.159 --> 00:34:29.980 +So a web framework is a conceptual idea across many different programming languages, but the specific implementations like a Ruby on Rails, like Rails, the framework, or Django, the framework, are the specific implementations within those programming ecosystems. + +00:34:30.440 --> 00:34:33.820 +And so that's essentially how I built out this site over 10 years. + +00:34:34.040 --> 00:34:38.740 +I will say that I only really work on one side project at a time. + +00:34:39.260 --> 00:34:43.080 +And Fullstack Python, I felt like, kind of run its course over 10 years. + +00:34:43.200 --> 00:34:45.320 +So I really haven't updated it in a few years. + +00:34:45.800 --> 00:34:52.600 +I still think it's actually relevant for most of it, but some of the links are a little bit dated and things like that. + +00:34:52.629 --> 00:35:00.320 +And it's not been brought into the conversation that we're having around coding agents and things like that. + +00:35:00.540 --> 00:35:07.300 +It is still relevant, but I would say not as relevant as it was when I was updating it literally every day. + +00:35:07.400 --> 00:35:07.480 +Right. + +00:35:07.640 --> 00:35:10.280 +Back in 2016 or something like that. + +00:35:10.670 --> 00:35:10.760 +Yeah. + +00:35:10.770 --> 00:35:10.860 +Yeah. + +00:35:11.080 --> 00:35:22.020 +And I also, I struggle a little bit because I would love to go back to working on full stack Python, but what I struggle with is you can ask LLMs about all of this stuff and it will give you very good answers. + +00:35:22.410 --> 00:35:32.160 +And so a lot of what I was doing was just bringing like as an experienced software developer, like writing almost like essays on like, you can see here, like, why are web frameworks useful? + +00:35:32.540 --> 00:35:59.420 +You can ask an LLM like why web frameworks are useful. And actually it'll probably give you a better answer than what I've written here, because it's going to incorporate tons of different sources. So that's where I've struggled a little bit with like, as you know, the chat models based on LLMs, especially as they've added search have gotten better. I'm not really sure how to add enough value on top of what an LLM can tell you to justify a substantial investment in writing. + +00:36:00.320 --> 00:36:39.260 +And then also, the one challenge with Full Stack Python is it's a statically generated site. It is primarily Markdown built with Pelican, which is a great static site generator in Python, but it was a lot of writing and it wasn't as much coding as I wanted. The site is kind of the site And then you add a ton of content to it. To me, I really wanted to get back to, especially now as a VP for the last several years of my career in executive level, I don't get a lot of time day in and day out to code. And so I really wanted something that I was coding every single day on nights and weekends and just doing in my spare time. + +00:36:39.360 --> 00:36:43.640 +So that's kind of where I shifted to this project, PlushCap, which is at plushcap.com. + +00:36:43.900 --> 00:36:48.600 +So this is like a landscape of developer-focused companies with self-service motions. + +00:36:49.140 --> 00:36:58.440 +So the hypothesis behind this was like when I was at Twilio, we were tracking developer content and events and other competitors. + +00:36:58.960 --> 00:37:05.660 +And it wasn't back then, but now YouTube is a really important source for teaching developers. + +00:37:05.920 --> 00:37:09.880 +And so I want to just like to aggregate a lot of this data. + +00:37:10.220 --> 00:37:10.880 +So that's what I've done. + +00:37:11.020 --> 00:37:21.280 +It's essentially 500 developer-focused companies across all different stages from like pre-seed all the way to publicly traded companies, where they stand kind of in their competitive position. + +00:37:21.830 --> 00:37:27.940 +And then like leaderboards that will show you like how many videos and views and subscribers do different companies have. + +00:37:28.800 --> 00:37:52.340 +And so like there's just a lot of data analysis, a lot of data visualization, a lot that essentially just goes into, like, if you go to, if you scroll down on Airbyte and you go down to, on this page, if you scroll down to the website structure and click on the blog, like, sorry, if you click on, go back and just click on the blog or just go up a little bit, the blog posts. + +00:37:52.420 --> 00:37:53.220 +Gotcha. I see. + +00:37:53.780 --> 00:38:34.100 +Under content category, this lays out the number of blog posts that was published per month by the company. And so just being able to visualize like their content patterns and trends is like, been helpful for me as I talk to companies about their go-to-market motions with developers and developer marketing, things like that. So anyway, this is a Django project running on DigitalOcean, Cloudflare as a CDN, a ton of Python, a ton of database stuff on the back end. So for me, I just love digging into this and I use coding agents and they've really greatly accelerated what I can actually do with this. + +00:38:34.260 --> 00:38:36.520 +Because what I'll do is I'm in meetings all day. + +00:38:37.220 --> 00:38:41.020 +So in the start of the day, I'll just tee up like a bunch of stuff that I want to get done. + +00:38:41.480 --> 00:38:45.320 +And then throughout the day, as I'm like getting out of meetings, I can just hit like, okay, okay, like good. + +00:38:45.600 --> 00:38:47.900 +And I just run it all on my own computer. + +00:38:48.080 --> 00:38:52.220 +And then I've got my work computer, I got my own computer where I'm like running all this stuff. + +00:38:52.360 --> 00:38:55.440 +And like periodically when I'm taking a break, I'm gonna be like, okay, yep, that looks good. + +00:38:55.440 --> 00:38:57.060 +Or no, I don't do that thing, right? + +00:38:57.440 --> 00:39:15.320 +So to me, it is like having a development team, even just a small development team, because I'm not doing like huge stuff to actually implement things. So that's really where I use all these tools. And I have an established code base. I wrote the code base myself over three years by hand. So now I have all the design patterns. + +00:39:15.570 --> 00:39:34.240 +I have an opinionated framework with Django. I've already chosen the vast majority of the open source libraries that I need. And it is mostly about the coding tools, pattern matching against what I've already done to create a new visualization or to create a new scraper because I'll scrape all the content from all these sites. + +00:39:34.820 --> 00:39:48.700 +So if I add a new company in the database, I have to write some custom web scrapers in order to get all their data. And that stuff is annoying. I've already written hundreds of those. So I'd rather just have- You're done right now. + +00:39:48.700 --> 00:39:52.440 +And then out the other side comes something that I can actually use. + +00:39:52.440 --> 00:39:55.580 +And I don't have to do all the grunt work because I've done a lot of that myself. + +00:39:55.640 --> 00:40:00.440 +Yeah. There's no more lessons to learn by making you- figure out their nav structure. + +00:40:01.120 --> 00:40:01.480 +Yeah. + +00:40:01.820 --> 00:40:16.720 +Or like, you know, I'm pulling like, okay, which CSS class corresponds to like the author name of the blog post so that I can get some structured data out of what is otherwise like unstructured data because it's every single company has a different blog format, right? + +00:40:16.980 --> 00:40:17.060 +So. + +00:40:17.860 --> 00:40:18.460 +Yeah, I think this, + +00:40:18.880 --> 00:40:20.660 +you touched on two really interesting things here. + +00:40:20.880 --> 00:40:22.300 +First, well, three, I guess. + +00:40:22.320 --> 00:40:23.520 +One, PlushCap is super cool. + +00:40:23.760 --> 00:40:24.060 +Thanks. + +00:40:24.360 --> 00:40:24.460 +Yeah. + +00:40:24.800 --> 00:40:32.220 +Two, having a low stakes project that you can just kind of go mad on with agentic AI + +00:40:32.370 --> 00:40:35.820 +and it's not going to end things or end badly. + +00:40:36.260 --> 00:40:39.300 +And three, I think this is really interesting. + +00:40:39.730 --> 00:40:41.800 +And I've noticed the same trends for me as well. + +00:40:42.160 --> 00:40:55.440 +But having already created the project, having really carefully thought about design structures, like, okay, it's going to be in this folder structure and it's going to follow this design pattern over and over using these three libraries I like. + +00:40:55.450 --> 00:40:59.260 +Not the most popular ones, necessarily, but the ones that I think I want to work with. + +00:40:59.500 --> 00:41:02.680 +Putting that in place really limits the, + +00:41:02.980 --> 00:41:05.080 +oh, it just, why is it using NPM? + +00:41:05.260 --> 00:41:06.000 +This is a Python project. + +00:41:06.120 --> 00:41:06.720 +What is it up to? + +00:41:06.780 --> 00:41:13.360 +You know, like it's way, way more focused and more likely to be aligned with what you were hoping it would do in the first place. + +00:41:13.580 --> 00:41:17.320 +And you need to choose the tools that you can debug the best. + +00:41:18.160 --> 00:41:21.300 +And so for me, like this is a Bootstrap front end. + +00:41:21.460 --> 00:41:29.560 +There's a framework called Tabler, which is built kind of on top of Bootstrap, which makes it even easier to build kind of these admin style interfaces and pulls in. + +00:41:29.590 --> 00:41:31.080 +I think it's I think it's chart.js. + +00:41:32.140 --> 00:41:33.040 +Apex might be Apex. + +00:41:33.070 --> 00:41:34.560 +I was using a few different charting frameworks. + +00:41:35.230 --> 00:41:36.800 +So it might be like Apex or something now. + +00:41:37.020 --> 00:42:25.320 +But the whole the whole point of like choosing your tools and being opinionated about your tools, I think actually helps a lot because then you don't have to get up to speed on the framework or the library before you debug the problem that the LLM created, because the lms are typically gonna they're gonna get you often 95 of the way there and you can keep prompting them but it's often it's often better to just like go fix it yourself there might be like a one-line tweak or something like that um that you need to do as opposed to like trying to get it to like sometimes it feels like getting the last like five to ten percent of polish is like finding a needle in the haystack because it's changing too much often whereas if you can just go in and and modify it yourself, you're not going to, you're not going to introduce additional noise into the development process. + +00:42:25.700 --> 00:42:25.900 +Yeah. + +00:42:26.520 --> 00:42:29.800 +I have a, but seems like an out of left field question, but it's not. + +00:42:30.100 --> 00:42:31.560 +What about Git and source control? + +00:42:32.020 --> 00:42:32.660 +Oh, yeah. + +00:42:33.400 --> 00:42:49.900 +So, I mean, that's the thing that we haven't talked about, which is like wildly important, which is like after when I'm letting something run, it is off of either it's either in a separate Git branch from running a bunch of things, or I actually like a grant, I have some simplest simplifying assumptions with my own code base. + +00:42:50.140 --> 00:42:53.760 +I've split the code base into multiple parts. + +00:42:54.340 --> 00:42:58.640 +So I have scripts where the scripts are like web scrapers. + +00:42:59.120 --> 00:43:02.960 +And if I'm updating things through an API, like I use the YouTube API. + +00:43:03.120 --> 00:43:06.860 +I use the PyPy API. + +00:43:07.200 --> 00:43:10.080 +I use a bunch of different APIs to get data into the database. + +00:43:10.700 --> 00:43:13.880 +So I can have an agent running on that thing. + +00:43:14.340 --> 00:43:15.020 +And that's separate. + +00:43:15.040 --> 00:43:22.380 +I actually only have those things interface with an API that is running with Django framework with the main production application database. + +00:43:23.190 --> 00:43:27.660 +So I've created a layer of abstraction where I can have an agent running on this thing over here. + +00:43:28.350 --> 00:43:32.020 +Then I can also have it running on the main application code base. + +00:43:32.460 --> 00:43:39.680 +And as long as we're not changing anything about the API, then I know that they can make their changes separately and that they're not going to interfere with each other. + +00:43:41.119 --> 00:43:46.580 +And all of this, going back to your original question about Git, I'm just constantly just doing Git add, Git commit. + +00:43:47.460 --> 00:43:53.080 +The one thing I've definitely noticed about some of these tools they will go off and create a bunch of different files. + +00:43:53.530 --> 00:44:00.060 +And so you do have to get pretty careful with like, I used to just do, you know, git commit, all the things, + +00:44:01.020 --> 00:44:02.880 +git commit dot, or no, sorry, + +00:44:03.100 --> 00:44:05.420 +git add dot, git commit with a little message. + +00:44:06.080 --> 00:44:10.120 +And I've learned like, oh, geez, like there's some debugging files that it actually didn't remove. + +00:44:10.580 --> 00:44:12.420 +I got to like manually remove those. + +00:44:12.940 --> 00:44:18.080 +Now, again, like I think a lot of these tools, you can configure them to like get rid of those problems. + +00:44:18.420 --> 00:44:21.680 +I think that that's the thing that's really improving quickly now. + +00:44:21.770 --> 00:44:35.180 +So if you ask me, like, what's the thing now that's really annoying that will likely be much easier a year from now, it will be like, you're going to have a standard like claw.md file that gives instructions that's just going to be like standardized. + +00:44:35.640 --> 00:44:38.760 +It's like how everybody has a git ignore file. + +00:44:39.740 --> 00:44:41.980 +Like you just kind of copy it from somewhere, right? + +00:44:42.220 --> 00:44:42.300 +Yeah. + +00:44:42.540 --> 00:44:43.780 +And then you modify it a little bit. + +00:44:44.040 --> 00:44:45.600 +But like the vast majority of it's the same. + +00:44:45.960 --> 00:44:47.260 +Like that's where we're going to go, right? + +00:44:47.540 --> 00:44:47.660 +Right. + +00:44:48.010 --> 00:44:48.140 +Right. + +00:44:48.240 --> 00:44:50.540 +You're like, oh, this we've all tended towards this. + +00:44:50.900 --> 00:44:50.980 +Yeah. + +00:44:51.400 --> 00:44:55.740 +And so that goes back to my, you're not prompting enough sort of things, right? + +00:44:55.880 --> 00:45:01.180 +Like if, if you're opinionated about how your project is built, put that in there. + +00:45:01.340 --> 00:45:11.440 +And a lot of times these agentic tools have like a rules file, like cursor has a dot cursor rules and stuff that you might put in there is my preferred web framework is FastAPI. + +00:45:11.880 --> 00:45:18.720 +When you install things or create a virtual environment, use uv, not Python's native, right? + +00:45:19.080 --> 00:45:20.300 +And you'll see it when it's working. + +00:45:20.500 --> 00:45:27.920 +It'll say things like, I know that I'm supposed to use uv, so it'll run uv pip list when it's checking if something got installed right. + +00:45:28.210 --> 00:45:30.660 +And it'll even like sort of remind itself from what you've said. + +00:45:30.730 --> 00:45:33.980 +And it's those kinds of things that keep sit on track, right? + +00:45:34.100 --> 00:45:43.940 +So I would say one of the things people should actually put more effort into than maybe they initially think is like setting up these rules about just your general preferences yes and uh i think + +00:45:43.940 --> 00:46:44.420 +it was armin roniker had a really great uh tip the other day where essentially like you can just use i believe it's a hook in cloud code and it's like basically if the um the lm tries to use a pip install something it will actually throw an error and it will know based off of that error that it should be using uv uh and so you could add enough of these hooks and so like what i anticipate is like over time, number one, like people will develop some level of standardization for these, like right now, the configuration files are very fragmented across the different agentic tools. So you can't quickly swap from like one to the other, like, and like there's a Gemini CLI and there's a cloud, you know, cloud that has a configuration, both for project specific and for all your development workspaces. My guess is like a lot of this stuff will start to get standardized, just like MCP was standardized and an open standard. And then it will make it much easier for people to just transpose their development preferences from one tool to the other. + +00:46:44.680 --> 00:47:54.980 +So my guess is that's probably what's coming over the next 12 to 18 months. And if the big companies don't want that to happen, they'll be left behind by the open source tools that will make that much easier. I think the other thing is I actually don't... I don't know. Maybe I'm one of the balls with them. I don't, I have a VMRC, like for my configuration, I don't have like an insane number of extensions or, you know, or I guess plugins, or like a huge VMRC or anything like that. Like I actually try to keep things relatively standard. And then just focus on the patterns that are most important or the configurations that are most important. And I I'm still kind of like that with, with using cloud code, I do have, you know, cloud and defy on stuff. But I also found that it's not 100% accurate. And so I think there's going to be a lot of development around making sure that the tools adhere to the practices that you really want. Because right now, I feel like if you want reliability in code formatting, you need to run the code formatter as a separate step as a part of your build workflow. The agentic tools are just not... Maybe I'm not doing it right. + +00:47:55.380 --> 00:47:56.380 +I'm reliable now. + +00:47:56.440 --> 00:48:04.120 +I've gotten, cloud sonnet to, to know that it's supposed to run ruff format and rough check --fix whenever it finishes anything. + +00:48:04.560 --> 00:48:12.060 +And so it'll, at the end it'll say, and now I'm supposed to do, you know, this to make sure it's tidy and like the style you like according to your rough Tom, all right. + +00:48:12.320 --> 00:48:12.460 +Yeah. + +00:48:12.800 --> 00:48:13.580 +And that, that makes sense. + +00:48:13.730 --> 00:48:16.240 +I mean, it's, it's not, sometimes it doesn't. + +00:48:16.380 --> 00:48:16.480 +Right. + +00:48:17.180 --> 00:48:18.880 +But you usually do what do you, what's wrong with you? + +00:48:19.180 --> 00:48:19.480 +I know. + +00:48:19.840 --> 00:48:20.020 +Yeah. + +00:48:20.260 --> 00:48:39.060 +And also I feel like everything that you add to the agent, like the more and more steps than you add, actually, again, this will change over time, but I feel like it actually gets less reliable. And also the number of steps, I would rather just have it run a script that will handle all those things before I'm ready to commit, rather than having it run all the time. + +00:48:39.180 --> 00:48:56.980 +Because I've also found that sometimes I want to add a tool for code formatting or something, and then it slows down the whole process too much for me. And then the main thing with these tools is I want them to be extremely fast. And that is probably the biggest piece of advice I can get to any of these companies is like why Claude Code over others? + +00:48:57.070 --> 00:49:03.240 +Like it is insanely fast, even though the LLMs require, you know, processing time and stuff like it's responsive. + +00:49:03.530 --> 00:49:07.400 +And so like my guess is like that's also where a lot of the open source tools will go as well. + +00:49:07.520 --> 00:49:10.120 +They'll just be really fast and responsive, just like Vim and T-Monks are. + +00:49:10.320 --> 00:49:10.480 +Yeah. + +00:49:10.830 --> 00:49:13.520 +Leave it to a Vim user to say it's not fast enough. + +00:49:13.570 --> 00:49:13.900 +Just kidding. + +00:49:15.940 --> 00:49:16.260 +Yeah, exactly. + +00:49:16.600 --> 00:49:21.000 +So I think maybe just like some quick shout outs to some of the tools, right? + +00:49:21.060 --> 00:49:23.940 +So Cloud Code obviously is a really interesting one. + +00:49:24.200 --> 00:49:25.200 +You've been talking about that a lot. + +00:49:25.360 --> 00:49:26.760 +I've mentioned Cursor a few times. + +00:49:26.990 --> 00:49:32.800 +I think as much as I'm a much more fan of the PyTarm style, I find it to be very effective. + +00:49:33.160 --> 00:49:37.240 +I've done some stuff with Juni, which is kind of a new project from JetBrains. + +00:49:37.440 --> 00:49:42.320 +Like JetBrains has had an AI agent that was kind of like a super duper autocompleter. + +00:49:42.740 --> 00:49:44.360 +And maybe we could talk about that difference as well. + +00:49:44.590 --> 00:49:46.360 +But it's like a super duper autocompleter. + +00:49:46.400 --> 00:49:56.680 +but then they have a separate completely different install thing that is an agentic ai called juni and i think it's making good steps it like really integrates with pycharm if you're you're into that + +00:49:57.010 --> 00:50:26.720 +um yeah well and that's been the evolution right like we had yeah github copilot which was essentially like a very fancy auto complete it kind of it would complete your code for you but it was not an it wasn't an agent in any meaningful capacity like that's what kind of kicked off some of this stuff of like wow it's really great autocomplete and i think a lot of the tools follow that pattern. Even when you would use cursor say a year ago, it was very much like auto-complete, at least the standard pattern, right? Before they had the agent mode. + +00:50:26.920 --> 00:50:39.760 +For sure. And that never really connected with me. I'm like, I don't need that. I can just, I can auto-complete it word by word. I don't really, because half the time it'll auto-complete like three or four lines. I'm like, yes, but not that line. And how do I get it to auto-complete + +00:50:39.860 --> 00:50:43.180 +the next? I guess I got to accept it then go edit that part. You know, like it's just, + +00:50:43.500 --> 00:50:48.040 +I never really vibed with that, but the agentic mode, that is next level stuff. + +00:50:48.690 --> 00:50:55.280 +Also, if you're a PyCharm or VS Code user, the cloud code integrates in both of them, right? + +00:50:55.580 --> 00:50:55.740 +Yes. + +00:50:57.120 --> 00:51:03.580 +So maybe talk, I know you're using cloud code, so maybe tell people what it's like and why you like that one. + +00:51:03.760 --> 00:51:11.020 +Okay, so I started using cloud code like six months ago, but I was using it off of the API credits, which was before they came out with the plans. + +00:51:11.580 --> 00:51:12.680 +Started evaluating it. + +00:51:12.680 --> 00:51:15.000 +I was like, this is going to be way too expensive. + +00:51:15.270 --> 00:51:18.480 +Like this is going to be- Yes, it is very expensive very soon. + +00:51:18.480 --> 00:51:25.020 +I was like, based off of my usage patterns, like just like monthly, it's probably going to cost me over $1,000 a month. + +00:51:25.020 --> 00:51:28.440 +I was like, this is just not, it's not worth that, right? + +00:51:28.440 --> 00:51:32.540 +So then they came out with the max plan, which was I think originally $200 a month. + +00:51:32.540 --> 00:51:35.440 +Now they have $100 and $200 a month, which I thought was interesting, right? + +00:51:35.440 --> 00:51:37.720 +But I was like, I talked to enough people that did use it. + +00:51:37.720 --> 00:51:40.900 +And I was like, that's interesting, but I'm not that comfortable with this workflow yet. + +00:51:41.120 --> 00:51:43.180 +They came out with the pro plan, which was $20 a month. + +00:51:43.240 --> 00:51:44.720 +They said, okay, I'll try it for a month. + +00:51:45.260 --> 00:51:47.860 +It was almost like a, to me, it was almost like a free trial. + +00:51:48.020 --> 00:51:54.640 +Like I'm willing to invest $20 into a course or whatever it is like 200 is like a pretty high bar, but 20 bucks. + +00:51:54.720 --> 00:51:56.120 +I was like, yeah, okay, fine. + +00:51:56.260 --> 00:51:58.880 +Even if it, even if this sucks, I'll just, I'll try it. + +00:51:59.259 --> 00:52:01.220 +So I maxed out the pro plan. + +00:52:01.220 --> 00:52:08.420 +I was like, I'm gonna get every drop out of the pro plan that I possibly can for my 20 bucks, and then I'll make a decision as to whether I want to upgrade or not. + +00:52:08.660 --> 00:52:10.740 +I was like, this is good enough. + +00:52:11.279 --> 00:52:13.340 +And that was only with the Sonnet model. + +00:52:13.340 --> 00:52:14.880 +It wasn't the higher Opus model. + +00:52:14.990 --> 00:52:17.580 +So there's a lot of stuff that was a little bit more limiting about it. + +00:52:17.580 --> 00:52:18.620 +We have to give a lot more instructions. + +00:52:19.360 --> 00:52:20.980 +It wasn't as good as architecting things. + +00:52:21.460 --> 00:52:22.300 +But I got enough out of it. + +00:52:22.300 --> 00:52:24.380 +I was like, okay, I'd be willing to upgrade. + +00:52:25.420 --> 00:52:28.620 +And so I'm actually currently now on the $200 a month plan. + +00:52:28.940 --> 00:52:29.720 +I think it's worth it. + +00:52:29.870 --> 00:52:37.020 +I will say that it also, for me personally, it forces me to be a little bit more addicted to using it. + +00:52:37.160 --> 00:52:38.560 +I want to get my money's worth. + +00:52:38.620 --> 00:52:49.440 +and so there's a, there's a, a tool called CC usage, which, was originally just for the API, but now, is it'll just give you your usage patterns. + +00:52:49.760 --> 00:52:58.020 +Even if you're on the, the, you know, subscription plan, I will say like, my usage yesterday was, if it was by API would have been $105. + +00:52:58.800 --> 00:53:06.140 +So, you know, over a course of a month, if I was actually consistent, I'd be spending over $3,000 a month off of API credits, which is just not, not sustainable. + +00:53:06.300 --> 00:53:06.440 +Right. + +00:53:06.860 --> 00:53:16.020 +Yeah, there might be people out there listening who think, you know what, Matt is actually crazy because $200 a month to some random AI thing when I'm a developer is insane. + +00:53:16.580 --> 00:53:29.800 +But if you had somebody who worked for you, who you could give detailed instruction to and have some reasonable expectation that they can get it done independently, and you said, well, that person you hired is $200 a month, that would be insane, right? + +00:53:29.940 --> 00:53:34.120 +And this, you can really turn this thing loose and get a lot done. + +00:53:34.130 --> 00:53:41.840 +But I think one of the mind shifts, at least I had to make was if I'm going to pay for one of these AIs at a level like that, you better make use of it. + +00:53:42.050 --> 00:53:45.940 +And you better have a plan that like, this thing is, this is how I'm going to take advantage + +00:53:46.010 --> 00:53:46.300 +of it. + +00:53:46.470 --> 00:53:49.220 +But when you do all of a sudden, you're like, wow, this is actually a bargain. + +00:53:49.560 --> 00:53:49.800 +Yeah. + +00:53:50.220 --> 00:53:54.160 +Well, and let's, let's compare it to like another like AI tool that I use, which is Descript. + +00:53:54.360 --> 00:53:55.560 +And I use Descript quite a bit. + +00:53:55.820 --> 00:53:58.200 +Descript is for creating, you just record video. + +00:53:58.720 --> 00:54:02.260 +And then you can edit off of the transcript and it'll like edit the video. + +00:54:02.420 --> 00:54:04.580 +It's not perfect, but it's very, very good. + +00:54:05.160 --> 00:54:09.040 +And I'm on like, I don't know, it's like 150 bucks a year or something like that. + +00:54:09.200 --> 00:54:10.880 +I use it a lot for everyone. + +00:54:11.540 --> 00:54:20.000 +Like my teams are remote distributed and I like to record videos just so they can hear from me like in five to 10 minute increments of like, here's what I'm thinking about or shout outs or whatever it is. + +00:54:20.000 --> 00:54:20.080 +Right. + +00:54:20.480 --> 00:54:26.100 +So I use it like quite a bit internally, but like if I don't use it for a week because I'm traveling, I don't feel bad about it. + +00:54:27.000 --> 00:54:29.840 +And so like, but that's 150 bucks a year. + +00:54:30.280 --> 00:54:35.180 +Like I get enough usage out of it at 150 bucks a year that if I don't use it for a week, it's not that big of a deal. + +00:54:35.520 --> 00:54:37.180 +200 bucks a month is like a really high bar. + +00:54:37.240 --> 00:54:43.640 +And so my bet there was just like with the $20 a month plan was, if I don't use this enough, I'm going to cancel it. + +00:54:44.580 --> 00:54:50.660 +And the other bet is I think eventually that the open weighted models are going to get really interesting here. + +00:54:51.420 --> 00:54:55.920 +And so I just want to be on the edge of seeing this ahead of time. + +00:54:56.340 --> 00:55:00.900 +And so I look at it as a little bit of an investment in my own learning. + +00:55:02.300 --> 00:55:08.540 +And so to me, there's just no replacing hands-on time with the tools, right? + +00:55:08.870 --> 00:55:11.600 +And so that to me is really what this is about. + +00:55:11.900 --> 00:55:17.480 +And to be fair, I've used other tools, other APIs, developer tools, where I paid for them. + +00:55:18.040 --> 00:55:21.080 +It's actually a browser-based, which is like a web scraping tool, is awesome. + +00:55:21.780 --> 00:55:24.600 +I have paid 40 bucks a month here and there to use it. + +00:55:24.660 --> 00:55:27.500 +And then when I don't use it enough, I will just downgrade my plan. + +00:55:27.780 --> 00:55:35.720 +And you kind of have your short list of things that you're using as a developer, as a time, as opposed to thinking that it is like indefinitely, I'm going to be paying for this. + +00:55:36.040 --> 00:55:36.260 +Yeah. + +00:55:36.460 --> 00:55:36.600 +Yeah. + +00:55:36.760 --> 00:55:55.660 +I definitely think it's important for people to maybe not try the $200 plan, but like the $20 plan or something, if they're interested in these things, because it's downloading a free model that runs on your local machine, that 3 billion parameters and saying, well, I tried AI and it doesn't work is not the same as, as trying this, this type of thing. + +00:55:56.080 --> 00:55:56.160 +Right. + +00:55:56.160 --> 00:55:56.240 +Right. + +00:55:56.420 --> 00:55:57.860 +It's, it's, it's really, really different. + +00:55:58.220 --> 00:55:58.340 +Yeah. + +00:55:58.500 --> 00:56:03.560 +Well, and also think it's, if you have a, I think that's why my situation, okay, let's, + +00:56:03.620 --> 00:56:07.020 +let's maybe like set up a little framing of who I think this is most valuable for. + +00:56:07.200 --> 00:56:12.980 +If you were like an elite developer and you're working on things like embedded systems or things that absolutely cannot break. + +00:56:13.520 --> 00:56:15.840 +I actually think there is less value for you here. + +00:56:15.940 --> 00:56:18.520 +I think the read only mode is valuable in that. + +00:56:18.580 --> 00:56:49.240 +like analyze my C code, where could there be like, you know, like memory, you know, buffer, buffer or bullet attacks or whatever it is. Right. But I think if you were building like side projects, like I'm not trying to like monetize like plush cap really, but like, you know, maybe someday I will. I just really love building plush cap. Like I think if you have a side project and for you, like you're building courses, you have your website, like there's a lot there that if you just had more bandwidth, you could build so much more cool stuff. + +00:56:49.410 --> 00:57:02.860 +Like, yeah, like when you have a backlog of ideas and you don't have time to build them, that's actually where I think these tools are most valuable because you can build a lot of those ideas and some of those ideas might be stupid and then you can just throw them away. + +00:57:03.280 --> 00:57:09.920 +But the ones that you, if you can just clear your backlog and come up with new ideas, because this thing is just, you're able to ship so much faster. + +00:57:10.420 --> 00:57:12.840 +Even if you have to refactor by hand, that's amazing. + +00:57:13.240 --> 00:57:13.360 +Yeah. + +00:57:13.700 --> 00:57:20.580 +I have three projects that are like sort of side project-ish, but are like related to Talk Python in various ways. + +00:57:21.080 --> 00:57:24.640 +And yeah, it's just turn the stuff loose on it and see if it's a good idea. + +00:57:24.690 --> 00:57:31.860 +Who knows if it'll see the light of day, but it goes from, can I dedicate three months to that to, well, let me try an afternoon. + +00:57:32.660 --> 00:57:32.800 +Yeah. + +00:57:33.180 --> 00:57:34.520 +See how it comes out, you know? + +00:57:34.720 --> 00:57:38.960 +Let me show you like something that I built purely off of like Claude Code. + +00:57:39.080 --> 00:58:34.180 +if you go to if you go to plush cap i have a um a leaderboard uh of youtube channels and by company so you've got i actually have you know all almost 500 companies in there but it cuts off at a thousand so yeah if you go to leaderboards youtube um you know there's there's 231 companies listed here and all 231 are um uh have over a thousand subscribers now the thing that's interesting out like i was just showing this to someone and they're like oh you know it'd be really interesting is like you have like subscribers, you've got videos, you've got views. And if you click into like views, like if you click into the number of views, this is total views over time. It's a visualization of like, you know, what, how many views, open AI has on their YouTube channel over time. They were like, what would this look like? You know, cause they're, they're producing more and more videos. So are, is the average number of views going up over time or down? + +00:58:34.280 --> 00:58:36.520 +Like, are they becoming more efficient or less efficient? + +00:58:37.000 --> 00:58:42.260 +So I was like, I wonder, like I already have subscribers, videos and views. + +00:58:42.530 --> 00:58:43.180 +I have the visualization. + +00:58:43.290 --> 00:58:45.560 +I literally have the pattern for what I would like this to be. + +00:58:45.640 --> 00:58:54.380 +And yeah, there now you can see actually they're becoming less efficient over time as they add more videos and there's different ways you can slice this data. + +00:58:54.390 --> 00:58:58.220 +So I've actually got like a bunch more ideas just from creating this view. + +00:58:58.600 --> 00:59:01.680 +But this view was based off of the pattern that I had already created. + +00:59:02.020 --> 00:59:08.140 +And so it was just basically like using the Django ORM to pull the data, you know, create a new web page. + +00:59:08.600 --> 00:59:11.660 +And I could tell whether it was correct or not and then like tweak it. + +00:59:11.940 --> 00:59:18.400 +So it had the pattern and it just needed to mimic that in a way that was a new view and visualization. + +00:59:19.640 --> 00:59:20.840 +And I feel like that's what this is. + +00:59:20.840 --> 00:59:23.540 +Pull new data, make a new page, but it's pretty much the same. + +00:59:23.780 --> 00:59:24.060 +Exactly. + +00:59:24.580 --> 00:59:24.680 +Yes. + +00:59:25.200 --> 00:59:28.140 +So this isn't like, you know, rocket science or anything like that. + +00:59:28.200 --> 00:59:30.320 +But for me, like I wanted to build this view. + +00:59:30.620 --> 00:59:45.140 +I actually, the interesting part about the story is like, I talked to somebody in the morning and I shipped this like when I had a quick break and I was like, I sent it over to them and I was like, Hey, I built this thing. and actually no, it was my, my boss at work at DigitalOcean. + +00:59:45.200 --> 00:59:55.779 +He was like, cause we're looking at our YouTube channel and he's like, I wonder what the average views are over time. And I like, I was like a few hours later, I was like, I had a break, I shipped it. And he's like, this is crazy. Cause we use, you know, we use this data as a part of + +00:59:55.800 --> 01:00:00.680 +running my, my org, like Debra and stuff. So yeah, absolutely. Because whenever we have an idea, + +01:00:00.770 --> 01:00:03.620 +we're like, what does the data look like? Or what, especially when you can compare it across + +01:00:03.940 --> 01:00:07.680 +companies, that's really helpful. Yeah. It's the perfect type of thing. I absolutely love it. + +01:00:08.080 --> 01:00:35.340 +So let's close this out. We got maybe take a few more minutes and let's, let's give people some concrete tips. Like for example, when we talked about Git, you know, check in off to let it finish a job, save it and check that in. And don't let it just go wild for a while. Cause if it goes of bonkers and you're like oh i hate that you're like but that's also hours of work that's gone you know what i mean like use use source control as a um um sort of a save along the way + +01:00:35.500 --> 01:01:08.260 +um you've got a bunch of tips and stuff very give us tips yes i know okay number one thing i actually feel like i can clear uh i have better clarity around technical debt and how important it is to get rid of technical debt now because if an l if you're using the like an agent and it it like is looking to pattern match to what you have in your code base, and it finds the area that you know is basically technical debt, and it copies that, and you're not really on top of looking through the code, it will often copy design patterns that you really do not want in your code base. + +01:01:08.780 --> 01:01:24.940 +And so I actually feel like it allows you to make the argument that cleaning up technical debt is more important when you use LLMs and agents than when you were doing things by hand where you knew So as a developer, I'm just going to ignore that section of the code and I'm not going to use it. + +01:01:25.180 --> 01:01:35.220 +So that's like a very specific tip, which is like, if you are going to pattern match against something you're doing in your code base, make sure it's not copying the area that you know is a problem. + +01:01:35.760 --> 01:01:37.300 +And then just like shipping that. + +01:01:37.320 --> 01:01:38.680 +I think that's, that's one really. + +01:01:38.700 --> 01:01:41.420 +You don't want to supercharge your bad habits. + +01:01:41.900 --> 01:01:42.120 +Yeah. + +01:01:42.440 --> 01:01:42.500 +Yeah. + +01:01:42.740 --> 01:01:46.980 +I would say on top of this, you absolutely do want to clean up technical debt. + +01:01:47.240 --> 01:02:20.400 +you can, when it's done, say, great work, that's valid, but now please apply some dry principles and some design patterns that you know that we're already using and just polish this up one more time and you'll get pretty good results. On the other side is if you've got a project that's got a ton of technical debt, you can kind of just have this conversation with the agentic AI and say, help me understand, where are the long functions? Where is there too much coupling? Let's start breaking that down. And when it's in that mindset, it's, it's pretty good at like addressing those + +01:02:20.570 --> 01:04:00.579 +problems. Right. And that's, that's the flip side of like the technical debt is like, it should be easier, arguably to clean up technical debt, because if you fix something or you're, you're asking the L like the agent, like the introspective, like how could this be improved? What are the design patterns? And particularly like, at least in clock code, there's like a plan mode. I use plan mode. It's like a shift tab. I use it all the time because then it's not, it's going to just like tell you what it's going to do, not go and do it, not going to make any changes. And so I will ask it, like, how can I optimize these database queries? How can I simplify this code without oversimplifying? That type of thing. And that's actually really helpful for just identifying what changes actually should be made. Then I can make the decision, are you going to go ahead and implement those? Or should I just go ahead and implement them because they're easy enough for me to make the changes myself. Awesome. Next tip. Yeah. So I feel like you've already talked a little bit about this. Like the context is really important and the prompting is important. I will say that like where I've failed in the past and I like was not happy with the LLMs was like, I'd say something like, can you just write a bunch of unit tests for this thing? And like, of course, it's not going to have enough. Like I give it the code that I want it to be testing. But now what I found is like I, when I, after I have it write some code, I will say like, okay, now write me like a few happy paths and I'll like evaluate the happy paths and I'll be like, write the unhappy paths. And I'll like, look at those and how it's failing. And then I'm like, what are the edge cases? And so you just have to, even if you're like, a lot of people are like, oh, here's how you write the perfect prompt. Like I actually really hate doing that because what if I wrote the perfect prompt and it still didn't give me what I wanted at the other side? I just tend to + +01:04:00.600 --> 01:04:04.860 +have to tell it it's going to go to jail please you know no i just tend to be really incremental + +01:04:05.180 --> 01:05:00.380 +i'm like really specific like just like again going back to like that junior developer mentality it's like just write me like three happy path tests to test this function yeah like give me some inputs and the expected outputs okay now i want you to write the things that like basically would break it and make sure that it doesn't break so what that does is like rather than i i have not found success or it's been a lot harder with like write me a bunch of tests both happy path the non-happy path and then it's kind of you know depending which model you're using or what day you get like it may or may not be correct and so i brought it's also think about like for yourself and your own sanity when you're doing reviewing the changes that it's making uh you really want it to just like be consumable like okay those are the happy path and mentally you're like okay happy path tests or mentally not that um you should if you instruct it in the way that you want to consume the information that it's created out the other side, that can actually be really helpful as well. + +01:05:00.640 --> 01:05:09.620 +Yeah. I find it's terrible if you just, this, this project needs tests, write tests. It ends up testing like just the smallest arbitrary aspects of the code. And you're like, what is this? + +01:05:09.880 --> 01:05:35.320 +Like now I got to maintain this junk that like one plus one is two. Like, no, this is not what I want from you. Right. You got to be really specific and do bite size. And it's a little bit of the plan to like maybe plan out like the types of tests we should have. Okay. Let's do this first one and focus, you know, like really guide it a little bit, treat it like it's, it's a junior that's kind of new to this stuff. Don't just, you wouldn't tell the junior, like we need to refactor the architecture. I'll see you next week. You're like, well, what are you going to + +01:05:35.380 --> 01:05:40.020 +get back? You don't know. Yeah. Well, maybe, maybe like three quick, like lightning round tips. + +01:05:40.599 --> 01:06:02.180 +Yeah. One of one, one of them mostly just, I thought about because I do this with, I have been doing this for years where if I'm working on different projects or different part of the code base, I will just use different terminal backgrounds, different colors. This is my blue terminal. So blue is where I've got Tmux and Vim, and I'm running a bunch of stuff. Exactly. + +01:06:02.850 --> 01:06:19.760 +And then, yeah, I think my colleague, Amit, tweeted about this. And then black will be, I'm running the server, I'm running my task queue, all the stuff that is running right now, local development. Then if I'm tailing a bunch of logs on the server, that'll be in another + +01:06:19.720 --> 01:06:22.780 +different color. So just mentally- Does this happen automatically? + +01:06:22.860 --> 01:06:27.800 +No, I actually just open up separate terminals. And then within that, I'll have different team + +01:06:27.800 --> 01:06:32.800 +accessions running where I can just be like, again, it has nothing to do with the computer. + +01:06:32.800 --> 01:06:48.920 +It has everything to do with me. A lot of times just doing a ton of context switching. So I've found this to be applicable as a pattern for the agents. If I'm having an agent that works primarily on one part of the code base, then I will just have that be blue. And then another part of the code base have a B black. + +01:06:49.130 --> 01:06:49.240 +Right. + +01:06:49.760 --> 01:06:58.120 +So, again, like I also, this maybe goes right into the second tip, which is a lot of people are like still microservices versus monolith. + +01:06:58.420 --> 01:07:10.000 +Like I actually think that the perfect design pattern right now for a lot of projects is like much closer to monolith, but like you split it up into like based off of the number of agents that you want to run. + +01:07:10.310 --> 01:07:14.380 +So like, for me, I've got like my web scraping and like data collection stuff. + +01:07:14.680 --> 01:07:17.680 +And like, I generally just want like one, maybe two agents. + +01:07:18.360 --> 01:07:20.180 +So like, that is like a separate project. + +01:07:20.760 --> 01:07:24.720 +And then it interfaces with everything else through APIs, like actual like web APIs. + +01:07:25.240 --> 01:07:32.640 +And so like, that is actually how I've architected things where I'm like, how many developers, developers, agents do I want working on the code base at once? + +01:07:32.760 --> 01:07:36.200 +Like, what is my actual capacity to review the code? + +01:07:36.500 --> 01:07:39.660 +As a single developer, there's only so much capacity that I have. + +01:07:39.740 --> 01:07:43.820 +And so I will actually architect, or I've started to architect around that. + +01:07:44.000 --> 01:07:45.900 +But that doesn't mean I have a thousand microservices. + +01:07:46.110 --> 01:07:54.520 +It typically means that I split the architecture into like three or four components, maybe two beyond the main application. + +01:07:55.170 --> 01:07:58.760 +And that helps me to just feel like I'm being productive without being overwhelmed. + +01:07:59.120 --> 01:08:03.100 +Yeah, it's also good advice just for working in small teams anyway. + +01:08:03.460 --> 01:08:04.140 +Yeah, yeah, exactly. + +01:08:04.370 --> 01:08:05.520 +And it's just like a little bit more like a pragmatic. + +01:08:05.520 --> 01:08:06.720 +Some of the teams are automated, right? + +01:08:06.900 --> 01:08:07.260 +It's crazy. + +01:08:07.280 --> 01:08:10.400 +Yeah, it's like a pragmatic, like I'm not doing crazy microservices. + +01:08:10.490 --> 01:08:13.460 +I'm not also doing like monolith everything, right? + +01:08:13.580 --> 01:08:17.480 +So I think that to me, I'll just kind of be a little bit more, more pragmatic about things. + +01:08:18.140 --> 01:08:18.339 +Yeah. + +01:08:18.500 --> 01:08:25.900 +I wonder if you could automate, if you could, if you could AI agentic code, some form of automatic color profile selecting. + +01:08:26.230 --> 01:08:29.380 +I mean, yeah, they go pretty good on bash scripts. + +01:08:29.450 --> 01:08:30.680 +I mean, it's possible. + +01:08:31.460 --> 01:08:34.140 +I've one, one last tip, which kind of bit me. + +01:08:34.680 --> 01:08:40.160 +Luckily I have, I'm running Sentry, shout out to the Sentry team for, for, for BlushGap. + +01:08:40.299 --> 01:08:46.339 +and I caught this bug and I, you know, a lot of times I'm like finding bugs, like, oh, it's a 500 error or whatever. + +01:08:46.740 --> 01:08:47.500 +I'm not the end of the world. + +01:08:47.660 --> 01:08:48.440 +Like I go and fix it. + +01:08:48.960 --> 01:08:52.200 +I had definitely written more bugs myself by hand. + +01:08:53.279 --> 01:08:56.120 +Now that I am like, I have a mixed sort of like toolkit. + +01:08:56.620 --> 01:09:05.380 +Like I, I've just, it's just something to like be mindful of, which is like, I'm, I'm so used to having like bits and pieces written by an agent. + +01:09:05.589 --> 01:09:11.440 +And then I review it that when I'm writing code, I have like my, it's almost like over It's like, oh, I review a lot of code. + +01:09:11.880 --> 01:09:15.540 +I actually almost feel like the agent should be reviewing the code that I'm writing by hand. + +01:09:15.569 --> 01:09:28.779 +And so I'll just say like, that's one thing that I feel like is a big tip for developers and might bite a lot of people as they spend somewhat less time writing code, especially like low level kind of like rote detail kind of stuff. + +01:09:29.390 --> 01:09:31.660 +And then they have to, then they are like, oh, I'll go do that real quick. + +01:09:31.900 --> 01:09:35.180 +And then they suddenly like introduce a bug and they're like, oh, geez, I didn't. + +01:09:35.359 --> 01:09:35.620 +Yeah. + +01:09:36.020 --> 01:09:36.080 +Yeah. + +01:09:36.819 --> 01:10:32.560 +haven't done this like yeah by hand with the clickety click for a while yeah exactly exactly yeah i um i think that we need to we need to embrace and be mindful like you're saying um a certain level of like student mind when we're doing this kind of stuff i mean you say you've been using clock code for like six months and these are starting to show up like imagine 10 years down the line right you know what i mean it could be seriously an issue yeah i did help one person do a bunch of HNNC coding and won't name it because I'm not sure they want to be named, but they really were like, wow, this is incredible. And the thing that surprises me, they said, was as it's going through maybe 30 or 40 things it does to process a request, it's explaining why it's doing it. It says, okay, now I need to make sure that this runs in this way. So let's do this like Django, this manage pi command or something like that. + +01:10:32.640 --> 01:10:32.720 +Right. + +01:10:33.060 --> 01:10:38.680 +And they're like, I'm actually, if I'm paying attention, I'm actually learning new things about how this is supposed to be done. + +01:10:38.930 --> 01:10:40.940 +And in the end it says, here's why I made these changes. + +01:10:41.420 --> 01:10:43.580 +I think it's so easy to just go great next. + +01:10:44.180 --> 01:10:44.800 +You know what I mean? + +01:10:44.860 --> 01:10:44.940 +Yeah. + +01:10:45.260 --> 01:10:50.400 +But if we just slow down a little bit and you can learn a lot to learn, you learn, I've learned a ton. + +01:10:50.760 --> 01:10:53.960 +Like I can just hit escape and I'm like, tell me why you're doing that. + +01:10:53.990 --> 01:10:56.380 +I'm not really sure why, why you're doing that. + +01:10:56.780 --> 01:10:57.680 +And it'll explain it. + +01:10:57.840 --> 01:10:59.900 +And then you can, you can, it's the back and forth. + +01:11:00.000 --> 01:11:07.400 +I mean, it is like having the conversation about code that you often can't have with another developer. + +01:11:07.660 --> 01:11:12.440 +It's really expensive to take your whole team's time to talk through some change to the code base. + +01:11:12.440 --> 01:11:13.500 +You can have all those conversations. + +01:11:14.040 --> 01:11:20.660 +And I personally find as someone who is relatively introverted, it is not taxing on me to just have that conversation with the computer. + +01:11:20.790 --> 01:11:26.380 +So I can still have all the conversations with the humans and not be drained by the conversation with the computer. + +01:11:26.640 --> 01:11:26.780 +Yeah. + +01:11:27.010 --> 01:11:27.560 +Yeah, that's awesome. + +01:11:27.660 --> 01:11:35.200 +And think of the opportunity for people who either are sole developers or they live in places where they can't go to user group meetup type things. + +01:11:35.910 --> 01:11:36.020 +Yeah. + +01:11:36.020 --> 01:11:36.160 +Yeah. + +01:11:36.320 --> 01:11:42.040 +Or they're like busy in meetings all day and they still want to be a productive developer and be hands on with this stuff. + +01:11:42.210 --> 01:11:47.160 +And not, you know, I think a big part of it is like not getting caught up too much with the whole hype cycle. + +01:11:47.640 --> 01:12:07.760 +I am firmly in the camp that while this is a shift, it is not that big of a shift compared to when they said, oh, software developers are going to go away because we have COBOL. Software developers are going to go away because we have code generation off UML diagrams. Software developers are going to go away because of low code, no code. + +01:12:08.560 --> 01:12:16.380 +I get the sense, having used these tools, that a lot of companies will use them in the wrong way of thinking they can just replace developers. + +01:12:16.550 --> 01:12:21.920 +And actually what they're doing is they're building up a massive amount of technical debt and they're going to need software developers to fix it. + +01:12:22.180 --> 01:12:22.280 +Yeah. + +01:12:22.420 --> 01:12:26.300 +So that, I think it's building further layers of abstraction and more code. + +01:12:26.840 --> 01:12:35.400 +I think that you and I and everyone else out there who is very competent coders pre-AI, we're going to be a little bit like COBOL programmers, I think. + +01:12:35.700 --> 01:12:41.260 +There's going to be places that are just like, look, we've just got to call some people that know how to write code and figure this out. + +01:12:41.740 --> 01:12:47.740 +Like this stuff, the crazy analogy, the crazy analogy is like, okay, this is what I read one. + +01:12:47.910 --> 01:13:03.460 +Like, so that like the U boats during German U boats during world war one have like a pre they have steel in them that is pre or it's not been affected by like nuclear, the nuclear bombs that were set off around, you know, during and after world war two. + +01:13:04.080 --> 01:13:09.960 +So like they will extract the steel from those because they're not affected by the radiation. + +01:13:10.820 --> 01:13:12.620 +I guess we're like the old U-boats. + +01:13:12.730 --> 01:13:16.960 +They're going to extract us from retirement and be like, you still know how to program + +01:13:17.130 --> 01:13:18.360 +in this esoteric language. + +01:13:18.570 --> 01:13:19.880 +The LLMs can't do it. + +01:13:19.910 --> 01:13:20.280 +I don't know. + +01:13:20.400 --> 01:13:21.140 +Maybe that's what's going to happen. + +01:13:21.140 --> 01:13:22.000 +Take the mothballs out. + +01:13:22.180 --> 01:13:22.500 +That's right. + +01:13:22.680 --> 01:13:22.780 +Yeah. + +01:13:22.820 --> 01:13:24.860 +And we're going back into service. + +01:13:25.860 --> 01:13:26.960 +Boy, it's an interesting time. + +01:13:27.050 --> 01:13:30.100 +And there's a lot of opportunity as you pointed out. + +01:13:30.200 --> 01:13:34.980 +So I want to leave people with a message of don't be scared, embrace this, see what it can do for you. + +01:13:35.260 --> 01:13:38.380 +There's a lot of opportunity here, but Matt, you get the final word. + +01:13:38.580 --> 01:13:38.740 +Yeah. + +01:13:39.000 --> 01:13:39.520 +I mean, I agree. + +01:13:39.570 --> 01:13:48.500 +I think it's, again, it's like the same philosophy we started with, which is like, I don't see this as that big of a shift compared to like a lot of the other stuff that's happened in industry. + +01:13:49.940 --> 01:13:51.440 +Like just, I would just learn these things. + +01:13:51.530 --> 01:13:56.360 +And frankly, like, then you are empowered to say like, I'm not going to use that because you've tried it. + +01:13:57.160 --> 01:13:59.440 +And so like, then you have an informed opinion on it. + +01:14:00.020 --> 01:14:07.660 +And I think that's really what matters as software developers, not to just dismiss something simply because it seems like a hype cycle, but actually just to try it. + +01:14:07.860 --> 01:14:10.240 +And if it doesn't work and you're like, well, I have an informed opinion. + +01:14:10.660 --> 01:14:13.240 +I will say, though, this is actually rapidly evolving. + +01:14:13.340 --> 01:14:22.700 +And as we've talked about through our conversation, I would actually try this multiple times over several months because it does get better and it does change with the tools. + +01:14:22.880 --> 01:14:25.420 +And so it's not just like I tried it once and I'm going to toss it away. + +01:14:26.060 --> 01:14:29.120 +I would give it a try every few months and see if it works for, you know, + +01:14:29.160 --> 01:14:31.000 +kind of clicks for you and kind of works for your use case. + +01:14:31.360 --> 01:14:34.540 +Yeah, if we had this conversation pre-agentic code agents. + +01:14:34.980 --> 01:14:35.120 +Yeah. + +01:14:35.420 --> 01:14:37.660 +Could be very, very different, but six months later. + +01:14:38.380 --> 01:14:41.480 +It's just like when open source was like nascent. + +01:14:42.040 --> 01:14:43.360 +Oh, I tried open source once. + +01:14:43.740 --> 01:14:47.720 +If you haven't tried it in 15 years, like, I don't know what to say, right? + +01:14:47.840 --> 01:14:50.280 +I think it's a similar thing, but it's just a little bit faster evolution. + +01:14:50.700 --> 01:14:50.760 +Yeah. + +01:14:51.020 --> 01:14:52.320 +Yeah, it's crazy times. + +01:14:52.500 --> 01:15:01.260 +I mean, I'm honestly thinking maybe we should ship this episode like this week without recording, not the two weeks out that it's scheduled to be recording because who knows what's going to happen. + +01:15:01.620 --> 01:15:02.100 +No, I'm just kidding. + +01:15:02.400 --> 01:15:03.960 +But they are changing fast. + +01:15:04.400 --> 01:15:06.100 +Well, thanks for coming back on the show. + +01:15:06.200 --> 01:15:07.300 +Always great to catch up with you. + +01:15:07.500 --> 01:15:07.880 +Thanks, Michael. + +01:15:08.180 --> 01:15:08.420 +It's been great. + +01:15:08.700 --> 01:15:09.160 +Yeah, bye. + +01:15:10.440 --> 01:15:12.800 +This has been another episode of Talk Python To Me. + +01:15:13.620 --> 01:15:14.560 +Thank you to our sponsors. + +01:15:15.040 --> 01:15:16.260 +Be sure to check out what they're offering. + +01:15:16.480 --> 01:15:17.700 +It really helps support the show. + +01:15:18.280 --> 01:15:21.980 +This episode is sponsored by Posit Connect from the makers of Shiny. + +01:15:22.460 --> 01:15:26.340 +Publish, share, and deploy all of your data projects that you're creating using Python. + +01:15:26.940 --> 01:15:33.040 +Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards, and APIs. + +01:15:33.940 --> 01:15:35.560 +Posit Connect supports all of them. + +01:15:35.730 --> 01:15:41.160 +Try Posit Connect for free by going to talkpython.fm/posit, P-O-S-I-T. + +01:15:41.900 --> 01:15:42.780 +Want to level up your Python? + +01:15:43.240 --> 01:15:46.880 +We have one of the largest catalogs of Python video courses over at Talk Python. + +01:15:47.360 --> 01:15:52.040 +Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:15:52.420 --> 01:15:54.660 +And best of all, there's not a subscription in sight. + +01:15:54.740 --> 01:15:57.580 +Check it out for yourself at training.talkpython.fm. + +01:15:58.280 --> 01:16:02.460 +Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:16:02.880 --> 01:16:03.780 +We should be right at the top. + +01:16:04.280 --> 01:16:13.160 +You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +01:16:13.820 --> 01:16:16.040 +We're live streaming most of our recordings these days. + +01:16:16.380 --> 01:16:23.900 +If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:16:24.940 --> 01:16:28.440 +This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. + +01:16:28.800 --> 01:16:30.380 +Now get out there and write some Python code. + +01:16:42.880 --> 01:16:45.680 +*music* + From a2b4137ba565d7a71caf7812c07989c6a1c4eccd Mon Sep 17 00:00:00 2001 From: Michael Kennedy Date: Tue, 19 Aug 2025 15:34:09 -0700 Subject: [PATCH 5/5] transcripts --- .../069_excellent_programming_blog.txt | 2 +- .../069_excellent_programming_blog.vtt | 2 +- transcripts/070_pythonic_music.txt | 2 +- transcripts/070_pythonic_music.vtt | 2 +- transcripts/096-exploring-awesome-python.txt | 2 +- transcripts/096-exploring-awesome-python.vtt | 2 +- ...thon-environment-to-visual-studio-code.txt | 2 +- ...thon-environment-to-visual-studio-code.vtt | 2 +- ...pyllvm-and-mongodb-for-data-scientists.txt | 2 +- ...pyllvm-and-mongodb-for-data-scientists.vtt | 2 +- ...and-running-old-python-faster-at-intel.txt | 2 +- ...and-running-old-python-faster-at-intel.vtt | 2 +- transcripts/118-serverless-software.txt | 2 +- transcripts/118-serverless-software.vtt | 2 +- transcripts/121-microservices-in-python.txt | 2 +- transcripts/121-microservices-in-python.vtt | 2 +- ...ome-assistant-pythonic-home-automation.txt | 2 +- ...ome-assistant-pythonic-home-automation.vtt | 2 +- transcripts/123-100-python-interviews.txt | 2 +- transcripts/123-100-python-interviews.vtt | 2 +- transcripts/124-python-for-ai-research.txt | 2 +- transcripts/124-python-for-ai-research.vtt | 2 +- .../126-kubernetes-for-pythonistas.txt | 2 +- .../126-kubernetes-for-pythonistas.vtt | 2 +- transcripts/130-ten-pythonic-books.txt | 2 +- transcripts/130-ten-pythonic-books.vtt | 2 +- transcripts/131-ten-data-science-libs.txt | 2 +- transcripts/131-ten-data-science-libs.vtt | 2 +- .../132-contributing-to-open-source.txt | 2 +- .../132-contributing-to-open-source.vtt | 2 +- ...dual-typing-of-production-applications.txt | 2 +- ...dual-typing-of-production-applications.vtt | 2 +- ...r-stories-of-the-developer-evangelists.txt | 4 +- ...r-stories-of-the-developer-evangelists.vtt | 4 +- .../192-python-year-in-review-2018.txt | 2 +- .../192-python-year-in-review-2018.vtt | 2 +- .../255-talking-to-cars-with-python.txt | 2 +- .../255-talking-to-cars-with-python.vtt | 2 +- transcripts/400-ruff-linter.txt | 2 +- transcripts/400-ruff-linter.vtt | 2 +- transcripts/415-pydantic-20-and-fastapi.txt | 2 +- transcripts/415-pydantic-20-and-fastapi.vtt | 2 +- transcripts/430-gradio.txt | 2 +- transcripts/430-gradio.vtt | 2 +- transcripts/440-jupyter-ai.txt | 6 +- transcripts/440-jupyter-ai.vtt | 6 +- .../456-building-custom-gpts-with-python.txt | 16 +- .../456-building-custom-gpts-with-python.vtt | 16 +- transcripts/468-python-trends-of-2024.txt | 2 +- transcripts/468-python-trends-of-2024.vtt | 2 +- .../476-unified-python-packaging-with-uv.txt | 6 +- .../476-unified-python-packaging-with-uv.vtt | 6 +- .../482-pre-commit-hooks-for-python-devs.txt | 2 +- .../482-pre-commit-hooks-for-python-devs.vtt | 2 +- transcripts/488-lancedb.txt | 2 +- transcripts/488-lancedb.vtt | 2 +- transcripts/504-developer-trends.txt | 2 +- transcripts/504-developer-trends.vtt | 2 +- ...07-agentic-ai-workflows-with-langgraph.txt | 8 +- ...07-agentic-ai-workflows-with-langgraph.vtt | 8 +- transcripts/517-agentic-ai.txt | 16 +- transcripts/517-agentic-ai.vtt | 16 +- transcripts/518-django-birthday.txt | 1738 +++++++++++ transcripts/518-django-birthday.vtt | 2624 +++++++++++++++++ youtube_transcripts/356-ml-startups.vtt | 2 +- youtube_transcripts/459-ai-coding.vtt | 8 +- ...07-agentic-ai-workflows-with-langgraph.vtt | 6 +- .../517-agentic-ai-youtube.vtt | 16 +- 68 files changed, 4484 insertions(+), 122 deletions(-) create mode 100644 transcripts/518-django-birthday.txt create mode 100644 transcripts/518-django-birthday.vtt diff --git a/transcripts/069_excellent_programming_blog.txt b/transcripts/069_excellent_programming_blog.txt index e5316eb..4d8f494 100644 --- a/transcripts/069_excellent_programming_blog.txt +++ b/transcripts/069_excellent_programming_blog.txt @@ -30,7 +30,7 @@ 00:01:37 Hey, everyone. Before we get to the main interview, I actually have a special little segment for you and a -00:01:43 pretty awesome announcement. I'm here with Matt McKay. Remember him from the Full Stack Guide to +00:01:43 pretty awesome announcement. I'm here with Matt Makai. Remember him from the Full Stack Guide to 00:01:48 Deployment episode back in episode 26. So Matt and I have a pretty awesome announcement. Matt, diff --git a/transcripts/069_excellent_programming_blog.vtt b/transcripts/069_excellent_programming_blog.vtt index 5f8b827..4167aa5 100644 --- a/transcripts/069_excellent_programming_blog.vtt +++ b/transcripts/069_excellent_programming_blog.vtt @@ -49,7 +49,7 @@ Twitter, where they're at snap underscore CI and at Intel software. Hey, everyone. Before we get to the main interview, I actually have a special little segment for you and a 00:01:43.480 --> 00:01:48.420 -pretty awesome announcement. I'm here with Matt McKay. Remember him from the Full Stack Guide to +pretty awesome announcement. I'm here with Matt Makai. Remember him from the Full Stack Guide to 00:01:48.420 --> 00:01:54.160 Deployment episode back in episode 26. So Matt and I have a pretty awesome announcement. Matt, diff --git a/transcripts/070_pythonic_music.txt b/transcripts/070_pythonic_music.txt index e6df9a5..0bd5663 100644 --- a/transcripts/070_pythonic_music.txt +++ b/transcripts/070_pythonic_music.txt @@ -60,7 +60,7 @@ 00:02:12 That's pretty awesome. -00:02:14 And it's a good sign that I think Matt McKay and I are on to doing something cool for the community. +00:02:14 And it's a good sign that I think Matt Makai and I are on to doing something cool for the community. 00:02:18 I really, truly appreciate all of you who are contributing to it and helping me bring this course to life. diff --git a/transcripts/070_pythonic_music.vtt b/transcripts/070_pythonic_music.vtt index 67792a1..a5bdac4 100644 --- a/transcripts/070_pythonic_music.vtt +++ b/transcripts/070_pythonic_music.vtt @@ -94,7 +94,7 @@ The campaign was fully funded in just an hour and 15 minutes. That's pretty awesome. 00:02:14.280 --> 00:02:18.360 -And it's a good sign that I think Matt McKay and I are on to doing something cool for the community. +And it's a good sign that I think Matt Makai and I are on to doing something cool for the community. 00:02:18.360 --> 00:02:23.400 I really, truly appreciate all of you who are contributing to it and helping me bring this course to life. diff --git a/transcripts/096-exploring-awesome-python.txt b/transcripts/096-exploring-awesome-python.txt index 9bf2f89..2880a1a 100644 --- a/transcripts/096-exploring-awesome-python.txt +++ b/transcripts/096-exploring-awesome-python.txt @@ -20,7 +20,7 @@ 00:00:37 How about cryptography or raw socket programming and so on? -00:00:40 This week, Matt McKay is back with me. +00:00:40 This week, Matt Makai is back with me. 00:00:43 Remember him talking about deployments on episode 26 last year? diff --git a/transcripts/096-exploring-awesome-python.vtt b/transcripts/096-exploring-awesome-python.vtt index 5893e82..b5f4319 100644 --- a/transcripts/096-exploring-awesome-python.vtt +++ b/transcripts/096-exploring-awesome-python.vtt @@ -34,7 +34,7 @@ What options do you have for admin backends on web apps? How about cryptography or raw socket programming and so on? 00:00:40.920 --> 00:00:43.000 -This week, Matt McKay is back with me. +This week, Matt Makai is back with me. 00:00:43.000 --> 00:00:46.420 Remember him talking about deployments on episode 26 last year? diff --git a/transcripts/101-adding-a-full-featured-python-environment-to-visual-studio-code.txt b/transcripts/101-adding-a-full-featured-python-environment-to-visual-studio-code.txt index c53d9fe..c777d79 100644 --- a/transcripts/101-adding-a-full-featured-python-environment-to-visual-studio-code.txt +++ b/transcripts/101-adding-a-full-featured-python-environment-to-visual-studio-code.txt @@ -730,7 +730,7 @@ 00:19:26 Do you want to launch an online business? -00:19:28 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:19:28 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:19:32 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/101-adding-a-full-featured-python-environment-to-visual-studio-code.vtt b/transcripts/101-adding-a-full-featured-python-environment-to-visual-studio-code.vtt index e632683..474bb2b 100644 --- a/transcripts/101-adding-a-full-featured-python-environment-to-visual-studio-code.vtt +++ b/transcripts/101-adding-a-full-featured-python-environment-to-visual-studio-code.vtt @@ -1129,7 +1129,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML d Do you want to launch an online business? 00:19:28.180 --> 00:19:32.380 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:19:32.380 --> 00:19:37.620 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/103-compiling-python-through-pyllvm-and-mongodb-for-data-scientists.txt b/transcripts/103-compiling-python-through-pyllvm-and-mongodb-for-data-scientists.txt index 959d205..f4b2ad5 100644 --- a/transcripts/103-compiling-python-through-pyllvm-and-mongodb-for-data-scientists.txt +++ b/transcripts/103-compiling-python-through-pyllvm-and-mongodb-for-data-scientists.txt @@ -646,7 +646,7 @@ 00:19:29 Do you want to launch an online business? -00:19:30 Well, Matt McKay and I built an Entrepreneur's Playbook with Python for Entrepreneurs. +00:19:30 Well, Matt Makai and I built an Entrepreneur's Playbook with Python for Entrepreneurs. 00:19:35 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/103-compiling-python-through-pyllvm-and-mongodb-for-data-scientists.vtt b/transcripts/103-compiling-python-through-pyllvm-and-mongodb-for-data-scientists.vtt index d86a210..0abd5a8 100644 --- a/transcripts/103-compiling-python-through-pyllvm-and-mongodb-for-data-scientists.vtt +++ b/transcripts/103-compiling-python-through-pyllvm-and-mongodb-for-data-scientists.vtt @@ -985,7 +985,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML d Do you want to launch an online business? 00:19:30.920 --> 00:19:35.120 -Well, Matt McKay and I built an Entrepreneur's Playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an Entrepreneur's Playbook with Python for Entrepreneurs. 00:19:35.120 --> 00:19:40.360 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/113-dedicated-ai-chips-and-running-old-python-faster-at-intel.txt b/transcripts/113-dedicated-ai-chips-and-running-old-python-faster-at-intel.txt index 3abb13f..305b22c 100644 --- a/transcripts/113-dedicated-ai-chips-and-running-old-python-faster-at-intel.txt +++ b/transcripts/113-dedicated-ai-chips-and-running-old-python-faster-at-intel.txt @@ -620,7 +620,7 @@ 00:25:59 Do you want to launch an online business? -00:26:01 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:26:01 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:26:06 This 16 hour course will teach you everything you need to launch your web based business with Python. diff --git a/transcripts/113-dedicated-ai-chips-and-running-old-python-faster-at-intel.vtt b/transcripts/113-dedicated-ai-chips-and-running-old-python-faster-at-intel.vtt index 5a9b913..5f934c9 100644 --- a/transcripts/113-dedicated-ai-chips-and-running-old-python-faster-at-intel.vtt +++ b/transcripts/113-dedicated-ai-chips-and-running-old-python-faster-at-intel.vtt @@ -937,7 +937,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON and XML da Do you want to launch an online business? 00:26:01.520 --> 00:26:06.520 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:26:06.520 --> 00:26:11.520 This 16 hour course will teach you everything you need to launch your web based business with Python. diff --git a/transcripts/118-serverless-software.txt b/transcripts/118-serverless-software.txt index dd7b1d7..e3b08f1 100644 --- a/transcripts/118-serverless-software.txt +++ b/transcripts/118-serverless-software.txt @@ -1068,7 +1068,7 @@ 00:42:16 Do you want to launch an online business? -00:42:18 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:42:18 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:42:22 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/118-serverless-software.vtt b/transcripts/118-serverless-software.vtt index a1e5b29..f93759f 100644 --- a/transcripts/118-serverless-software.vtt +++ b/transcripts/118-serverless-software.vtt @@ -1606,7 +1606,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML d Do you want to launch an online business? 00:42:18.620 --> 00:42:22.820 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:42:22.820 --> 00:42:28.060 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/121-microservices-in-python.txt b/transcripts/121-microservices-in-python.txt index f6aaa91..a9c7ade 100644 --- a/transcripts/121-microservices-in-python.txt +++ b/transcripts/121-microservices-in-python.txt @@ -1394,7 +1394,7 @@ 00:42:45 Do you want to launch an online business? -00:42:47 Well, Matt McKay and I built an entrepreneur's playbook with Python for entrepreneurs. +00:42:47 Well, Matt Makai and I built an entrepreneur's playbook with Python for entrepreneurs. 00:42:51 This 16 hour course will teach you everything you need to launch your web based business with diff --git a/transcripts/121-microservices-in-python.vtt b/transcripts/121-microservices-in-python.vtt index 85c0ce6..d9aa402 100644 --- a/transcripts/121-microservices-in-python.vtt +++ b/transcripts/121-microservices-in-python.vtt @@ -2095,7 +2095,7 @@ You'll learn to work with the restful HTTP services as well as SOAP, JSON and XM Do you want to launch an online business? 00:42:47.960 --> 00:42:51.960 -Well, Matt McKay and I built an entrepreneur's playbook with Python for entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for entrepreneurs. 00:42:51.960 --> 00:42:55.960 This 16 hour course will teach you everything you need to launch your web based business with diff --git a/transcripts/122-home-assistant-pythonic-home-automation.txt b/transcripts/122-home-assistant-pythonic-home-automation.txt index 56a035d..fcf1ce5 100644 --- a/transcripts/122-home-assistant-pythonic-home-automation.txt +++ b/transcripts/122-home-assistant-pythonic-home-automation.txt @@ -1052,7 +1052,7 @@ 00:36:56 Do you want to launch an online business? -00:36:58 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:36:58 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:37:02 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/122-home-assistant-pythonic-home-automation.vtt b/transcripts/122-home-assistant-pythonic-home-automation.vtt index aa89271..14aa5d5 100644 --- a/transcripts/122-home-assistant-pythonic-home-automation.vtt +++ b/transcripts/122-home-assistant-pythonic-home-automation.vtt @@ -1582,7 +1582,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML d Do you want to launch an online business? 00:36:58.820 --> 00:37:02.980 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:37:02.980 --> 00:37:08.220 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/123-100-python-interviews.txt b/transcripts/123-100-python-interviews.txt index 1b4c064..fb68a4f 100644 --- a/transcripts/123-100-python-interviews.txt +++ b/transcripts/123-100-python-interviews.txt @@ -618,7 +618,7 @@ 00:23:31 Do you want to launch an online business? -00:23:33 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:23:33 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:23:37 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/123-100-python-interviews.vtt b/transcripts/123-100-python-interviews.vtt index 09a461e..b576e8a 100644 --- a/transcripts/123-100-python-interviews.vtt +++ b/transcripts/123-100-python-interviews.vtt @@ -943,7 +943,7 @@ You'll learn to work with RESTful HTTP services, as well as SOAP, JSON, and XML Do you want to launch an online business? 00:23:33.160 --> 00:23:37.320 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:23:37.320 --> 00:23:42.560 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/124-python-for-ai-research.txt b/transcripts/124-python-for-ai-research.txt index 575e768..c51b842 100644 --- a/transcripts/124-python-for-ai-research.txt +++ b/transcripts/124-python-for-ai-research.txt @@ -1110,7 +1110,7 @@ 00:42:05 Do you want to launch an online business? -00:42:06 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:42:06 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:42:11 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/124-python-for-ai-research.vtt b/transcripts/124-python-for-ai-research.vtt index cf00877..ca98987 100644 --- a/transcripts/124-python-for-ai-research.vtt +++ b/transcripts/124-python-for-ai-research.vtt @@ -1669,7 +1669,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML d Do you want to launch an online business? 00:42:06.980 --> 00:42:11.140 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:42:11.140 --> 00:42:16.380 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/126-kubernetes-for-pythonistas.txt b/transcripts/126-kubernetes-for-pythonistas.txt index ac14e73..6cc7297 100644 --- a/transcripts/126-kubernetes-for-pythonistas.txt +++ b/transcripts/126-kubernetes-for-pythonistas.txt @@ -1510,7 +1510,7 @@ 00:41:49 Do you want to launch an online business? -00:41:51 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:41:51 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:41:55 This 16-hour course will teach you everything you need to launch your web-based business with diff --git a/transcripts/126-kubernetes-for-pythonistas.vtt b/transcripts/126-kubernetes-for-pythonistas.vtt index 201c7a1..e4b289e 100644 --- a/transcripts/126-kubernetes-for-pythonistas.vtt +++ b/transcripts/126-kubernetes-for-pythonistas.vtt @@ -2278,7 +2278,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML d Do you want to launch an online business? 00:41:51.160 --> 00:41:55.380 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:41:55.380 --> 00:42:00.220 This 16-hour course will teach you everything you need to launch your web-based business with diff --git a/transcripts/130-ten-pythonic-books.txt b/transcripts/130-ten-pythonic-books.txt index fdbbcdb..ba0748c 100644 --- a/transcripts/130-ten-pythonic-books.txt +++ b/transcripts/130-ten-pythonic-books.txt @@ -1508,7 +1508,7 @@ 00:43:34 Do you want to launch an online business? -00:43:36 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:43:36 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:43:40 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/130-ten-pythonic-books.vtt b/transcripts/130-ten-pythonic-books.vtt index a291390..2b6f30f 100644 --- a/transcripts/130-ten-pythonic-books.vtt +++ b/transcripts/130-ten-pythonic-books.vtt @@ -2272,7 +2272,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML d Do you want to launch an online business? 00:43:36.360 --> 00:43:40.560 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:43:40.560 --> 00:43:45.800 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/131-ten-data-science-libs.txt b/transcripts/131-ten-data-science-libs.txt index e4cf752..ae69e97 100644 --- a/transcripts/131-ten-data-science-libs.txt +++ b/transcripts/131-ten-data-science-libs.txt @@ -878,7 +878,7 @@ 00:40:27 as well as SOAP, JSON and XML data formats. Do you want to launch an online business? Well, -00:40:32 Matt McKay and I built an entrepreneur's playbook with Python for entrepreneurs. This 16 hour course will +00:40:32 Matt Makai and I built an entrepreneur's playbook with Python for entrepreneurs. This 16 hour course will 00:40:38 teach you everything you need to launch your web-based business with Python. And finally, diff --git a/transcripts/131-ten-data-science-libs.vtt b/transcripts/131-ten-data-science-libs.vtt index f84c62f..bad4ccc 100644 --- a/transcripts/131-ten-data-science-libs.vtt +++ b/transcripts/131-ten-data-science-libs.vtt @@ -1321,7 +1321,7 @@ Try my brand new consuming HTTP services in Python. You'll learn to work with RE as well as SOAP, JSON and XML data formats. Do you want to launch an online business? Well, 00:40:32.660 --> 00:40:38.160 -Matt McKay and I built an entrepreneur's playbook with Python for entrepreneurs. This 16 hour course will +Matt Makai and I built an entrepreneur's playbook with Python for entrepreneurs. This 16 hour course will 00:40:38.160 --> 00:40:42.460 teach you everything you need to launch your web-based business with Python. And finally, diff --git a/transcripts/132-contributing-to-open-source.txt b/transcripts/132-contributing-to-open-source.txt index edfd304..dd7e9d8 100644 --- a/transcripts/132-contributing-to-open-source.txt +++ b/transcripts/132-contributing-to-open-source.txt @@ -1224,7 +1224,7 @@ 00:37:13 Do you want to launch an online business? -00:37:15 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:37:15 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:37:19 This 16-hour course will teach you everything you need to launch your web-based business with diff --git a/transcripts/132-contributing-to-open-source.vtt b/transcripts/132-contributing-to-open-source.vtt index 70fbdd8..aed39b4 100644 --- a/transcripts/132-contributing-to-open-source.vtt +++ b/transcripts/132-contributing-to-open-source.vtt @@ -1840,7 +1840,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML d Do you want to launch an online business? 00:37:15.600 --> 00:37:19.440 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:37:19.440 --> 00:37:24.300 This 16-hour course will teach you everything you need to launch your web-based business with diff --git a/transcripts/151-gradual-typing-of-production-applications.txt b/transcripts/151-gradual-typing-of-production-applications.txt index 35fe8aa..1a868ec 100644 --- a/transcripts/151-gradual-typing-of-production-applications.txt +++ b/transcripts/151-gradual-typing-of-production-applications.txt @@ -898,7 +898,7 @@ 00:37:01 Do you want to launch an online business? -00:37:03 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +00:37:03 Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:37:07 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/151-gradual-typing-of-production-applications.vtt b/transcripts/151-gradual-typing-of-production-applications.vtt index 73c3b7c..8a02a0b 100644 --- a/transcripts/151-gradual-typing-of-production-applications.vtt +++ b/transcripts/151-gradual-typing-of-production-applications.vtt @@ -1351,7 +1351,7 @@ You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML d Do you want to launch an online business? 00:37:03.240 --> 00:37:07.440 -Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs. +Well, Matt Makai and I built an entrepreneur's playbook with Python for Entrepreneurs. 00:37:07.440 --> 00:37:12.680 This 16-hour course will teach you everything you need to launch your web-based business with Python. diff --git a/transcripts/189-war-stories-of-the-developer-evangelists.txt b/transcripts/189-war-stories-of-the-developer-evangelists.txt index ffbe8af..2718e04 100644 --- a/transcripts/189-war-stories-of-the-developer-evangelists.txt +++ b/transcripts/189-war-stories-of-the-developer-evangelists.txt @@ -6,7 +6,7 @@ 00:00:13 Who are these folks? How did they get this job, and what is it really like to do it from day to -00:00:18 day? Join me, along with Cecil Phillip from Microsoft, Matt McKay from Twilio, and Paul +00:00:18 day? Join me, along with Cecil Phillip from Microsoft, Matt Makai from Twilio, and Paul 00:00:23 Everett from JetBrains to dig into what it means to be a developer advocate and how they each became @@ -1770,7 +1770,7 @@ 00:57:31 This has been another episode of Talk Python To Me. -00:57:33 Our guests in this episode were Cecil Phillip, Matt McKay, and Paul Everett. +00:57:33 Our guests in this episode were Cecil Phillip, Matt Makai, and Paul Everett. 00:57:38 And it's been brought to you by Linode and Rollbar. diff --git a/transcripts/189-war-stories-of-the-developer-evangelists.vtt b/transcripts/189-war-stories-of-the-developer-evangelists.vtt index e230c63..a7d9ffc 100644 --- a/transcripts/189-war-stories-of-the-developer-evangelists.vtt +++ b/transcripts/189-war-stories-of-the-developer-evangelists.vtt @@ -13,7 +13,7 @@ company's booth, or traveling from conference to conference speaking on their sp Who are these folks? How did they get this job, and what is it really like to do it from day to 00:00:18.460 --> 00:00:23.600 -day? Join me, along with Cecil Phillip from Microsoft, Matt McKay from Twilio, and Paul +day? Join me, along with Cecil Phillip from Microsoft, Matt Makai from Twilio, and Paul 00:00:23.600 --> 00:00:28.560 Everett from JetBrains to dig into what it means to be a developer advocate and how they each became @@ -2659,7 +2659,7 @@ Thanks for listening. This has been another episode of Talk Python To Me. 00:57:33.740 --> 00:57:38.200 -Our guests in this episode were Cecil Phillip, Matt McKay, and Paul Everett. +Our guests in this episode were Cecil Phillip, Matt Makai, and Paul Everett. 00:57:38.200 --> 00:57:40.560 And it's been brought to you by Linode and Rollbar. diff --git a/transcripts/192-python-year-in-review-2018.txt b/transcripts/192-python-year-in-review-2018.txt index 2223bf0..14edb2f 100644 --- a/transcripts/192-python-year-in-review-2018.txt +++ b/transcripts/192-python-year-in-review-2018.txt @@ -804,7 +804,7 @@ 00:34:30 be able to do that is wonderful. -00:34:31 Yeah, it is. So you talk of the IOT stuff. I learned something from Matt McKay, recently +00:34:31 Yeah, it is. So you talk of the IOT stuff. I learned something from Matt Makai, recently 00:34:37 on an episode, and I want to share this with you and just see what you think. So you guys click on this, diff --git a/transcripts/192-python-year-in-review-2018.vtt b/transcripts/192-python-year-in-review-2018.vtt index 265329f..15f4f6f 100644 --- a/transcripts/192-python-year-in-review-2018.vtt +++ b/transcripts/192-python-year-in-review-2018.vtt @@ -1213,7 +1213,7 @@ and it used to be, we taught people C and like the barrier to entry for C is hug be able to do that is wonderful. 00:34:31.940 --> 00:34:37.240 -Yeah, it is. So you talk of the IOT stuff. I learned something from Matt McKay, recently +Yeah, it is. So you talk of the IOT stuff. I learned something from Matt Makai, recently 00:34:37.240 --> 00:34:41.520 on an episode, and I want to share this with you and just see what you think. So you guys click on this, diff --git a/transcripts/255-talking-to-cars-with-python.txt b/transcripts/255-talking-to-cars-with-python.txt index 0c151d2..29d871b 100644 --- a/transcripts/255-talking-to-cars-with-python.txt +++ b/transcripts/255-talking-to-cars-with-python.txt @@ -116,7 +116,7 @@ 00:03:43 Yeah, it's not the main topic, but what do you think the qualifications are? -00:03:46 And also, I should maybe just throw out there, back in episode numbers, escaping my mind, I did an episode with Paul Everett, Matt McKay, Cecil Phillip, about being a developer advocate for their different companies. +00:03:46 And also, I should maybe just throw out there, back in episode numbers, escaping my mind, I did an episode with Paul Everett, Matt Makai, Cecil Phillip, about being a developer advocate for their different companies. 00:04:00 So if people really want to dive into it, they can go check that out. diff --git a/transcripts/255-talking-to-cars-with-python.vtt b/transcripts/255-talking-to-cars-with-python.vtt index c834724..8a2ed24 100644 --- a/transcripts/255-talking-to-cars-with-python.vtt +++ b/transcripts/255-talking-to-cars-with-python.vtt @@ -178,7 +178,7 @@ Yeah, just kind of toes in a lot of the communication sides of the work. Yeah, it's not the main topic, but what do you think the qualifications are? 00:03:46.820 --> 00:04:00.280 -And also, I should maybe just throw out there, back in episode numbers, escaping my mind, I did an episode with Paul Everett, Matt McKay, Cecil Phillip, about being a developer advocate for their different companies. +And also, I should maybe just throw out there, back in episode numbers, escaping my mind, I did an episode with Paul Everett, Matt Makai, Cecil Phillip, about being a developer advocate for their different companies. 00:04:00.360 --> 00:04:03.220 So if people really want to dive into it, they can go check that out. diff --git a/transcripts/400-ruff-linter.txt b/transcripts/400-ruff-linter.txt index a1ce6c8..f8757d6 100644 --- a/transcripts/400-ruff-linter.txt +++ b/transcripts/400-ruff-linter.txt @@ -1826,7 +1826,7 @@ 00:44:45 Like I said, you would know that this wasn't just a Python tool you could use. -00:44:49 pip install rough. That's the way to go. +00:44:49 pipinstall ruff. That's the way to go. 00:44:51 But when I installed it, I didn't pip install. diff --git a/transcripts/400-ruff-linter.vtt b/transcripts/400-ruff-linter.vtt index 5becaa1..4fd7c6e 100644 --- a/transcripts/400-ruff-linter.vtt +++ b/transcripts/400-ruff-linter.vtt @@ -2746,7 +2746,7 @@ But let's start with installing it. Like I said, you would know that this wasn't just a Python tool you could use. 00:44:49.920 --> 00:44:51.840 -Pip install rough. That's the way to go. +Pipinstall ruff. That's the way to go. 00:44:51.840 --> 00:44:54.000 But when I installed it, I didn't pip install. diff --git a/transcripts/415-pydantic-20-and-fastapi.txt b/transcripts/415-pydantic-20-and-fastapi.txt index c74bb58..3b8a2af 100644 --- a/transcripts/415-pydantic-20-and-fastapi.txt +++ b/transcripts/415-pydantic-20-and-fastapi.txt @@ -1080,7 +1080,7 @@ 00:25:44 Like here's another project based on Pydantic plus, you know, name your other thing that -00:25:48 it's been crazy, particularly in the like machine learning space where, you know, Langchain, who are one of the like big names right now in these like big language +00:25:48 it's been crazy, particularly in the like machine learning space where, you know, LangChain, who are one of the like big names right now in these like big language 00:25:56 models, large language models, all based on Pydantic, right? diff --git a/transcripts/415-pydantic-20-and-fastapi.vtt b/transcripts/415-pydantic-20-and-fastapi.vtt index 9f2b70c..f05b2c6 100644 --- a/transcripts/415-pydantic-20-and-fastapi.vtt +++ b/transcripts/415-pydantic-20-and-fastapi.vtt @@ -1633,7 +1633,7 @@ Like here's another project based on Pydantic plus, you know, name your other th it's been crazy, particularly in the like machine learning space where, 00:25:51.620 --> 00:25:56.280 -you know, Langchain, who are one of the like big names right now in these like big language +you know, LangChain, who are one of the like big names right now in these like big language 00:25:56.280 --> 00:25:59.480 models, large language models, all based on Pydantic, right? diff --git a/transcripts/430-gradio.txt b/transcripts/430-gradio.txt index 687bdb7..2fda641 100644 --- a/transcripts/430-gradio.txt +++ b/transcripts/430-gradio.txt @@ -816,7 +816,7 @@ 00:19:49 Or something along those lines. -00:19:51 There's Langchain, right? +00:19:51 There's LangChain, right? 00:19:53 Yeah. diff --git a/transcripts/430-gradio.vtt b/transcripts/430-gradio.vtt index e5e2659..6a534e2 100644 --- a/transcripts/430-gradio.vtt +++ b/transcripts/430-gradio.vtt @@ -1231,7 +1231,7 @@ about it. Or something along those lines. 00:19:51.500 --> 00:19:53.400 -There's Langchain, right? +There's LangChain, right? 00:19:53.400 --> 00:19:53.900 Yeah. diff --git a/transcripts/440-jupyter-ai.txt b/transcripts/440-jupyter-ai.txt index 32ba419..725b764 100644 --- a/transcripts/440-jupyter-ai.txt +++ b/transcripts/440-jupyter-ai.txt @@ -1846,7 +1846,7 @@ 00:53:30 And it sounds, for example, your create a notebook that does this sort of thing, like teach me about Matplotlib, is a little bit agent driven. -00:53:38 Thinking of like Langchain and stuff like that, or even GPT engineer. +00:53:38 Thinking of like LangChain and stuff like that, or even GPT engineer. 00:53:43 What's the story? @@ -1968,11 +1968,11 @@ 00:56:00 Anything you want to give a shout out to? -00:56:02 We definitely use Langchain a lot. +00:56:02 We definitely use LangChain a lot. 00:56:04 And it has some pretty fantastic integrations. -00:56:07 And we're actually built on top of Langchain, really. +00:56:07 And we're actually built on top of LangChain, really. 00:56:10 But also Dask. diff --git a/transcripts/440-jupyter-ai.vtt b/transcripts/440-jupyter-ai.vtt index 57acfb0..38c551d 100644 --- a/transcripts/440-jupyter-ai.vtt +++ b/transcripts/440-jupyter-ai.vtt @@ -2776,7 +2776,7 @@ One is there's a lot of interest in these agents. And it sounds, for example, your create a notebook that does this sort of thing, like teach me about Matplotlib, is a little bit agent driven. 00:53:38.280 --> 00:53:43.840 -Thinking of like Langchain and stuff like that, or even GPT engineer. +Thinking of like LangChain and stuff like that, or even GPT engineer. 00:53:43.840 --> 00:53:45.040 What's the story? @@ -2959,13 +2959,13 @@ Some notable PyPI package, maybe something awesome you discovered to help you wr Anything you want to give a shout out to? 00:56:02.480 --> 00:56:04.520 -We definitely use Langchain a lot. +We definitely use LangChain a lot. 00:56:04.520 --> 00:56:07.780 And it has some pretty fantastic integrations. 00:56:07.780 --> 00:56:10.740 -And we're actually built on top of Langchain, really. +And we're actually built on top of LangChain, really. 00:56:10.740 --> 00:56:12.060 But also Dask. diff --git a/transcripts/456-building-custom-gpts-with-python.txt b/transcripts/456-building-custom-gpts-with-python.txt index 30d6f51..b5b49c7 100644 --- a/transcripts/456-building-custom-gpts-with-python.txt +++ b/transcripts/456-building-custom-gpts-with-python.txt @@ -570,7 +570,7 @@ 00:14:54 to switch either for cost reasons or performance reasons or what have you. -00:14:58 And, you know, Langchain, for instance, has a ton of the providers as part of you basically +00:14:58 And, you know, LangChain, for instance, has a ton of the providers as part of you basically 00:15:03 are just switching a few arguments when you're switching between them. @@ -582,9 +582,9 @@ 00:15:20 So you don't have to install all the different libraries that are out there. -00:15:22 And I think Langchain is kind of following a similar approach. +00:15:22 And I think LangChain is kind of following a similar approach. -00:15:25 I think they're coming up with a Langchain core capability where you can just kind of bring +00:15:25 I think they're coming up with a LangChain core capability where you can just kind of bring 00:15:29 in things as you need them. @@ -674,15 +674,15 @@ 00:17:42 All right. -00:17:42 So you talked about Langchain. +00:17:42 So you talked about LangChain. -00:17:44 Just give us a sense real quick of what Langchain is. +00:17:44 Just give us a sense real quick of what LangChain is. 00:17:46 This was a great project from a timing perspective. 00:17:49 I believe they kind of invented it and released it right around the time ChatGPT came out. -00:17:53 It's a very comprehensive library with lots of, I mean, the best part about Langchain to me is the documentation and the code samples. +00:17:53 It's a very comprehensive library with lots of, I mean, the best part about LangChain to me is the documentation and the code samples. 00:18:00 Right. @@ -702,7 +702,7 @@ 00:18:30 They're not close to 100% accurate on these types of tasks. -00:18:33 You know, the idea of agents as well as another thing that you might build with a Langchain. +00:18:33 You know, the idea of agents as well as another thing that you might build with a LangChain. 00:18:38 And the idea there is basically the agent is, you know, getting a task, coming up with a plan of that for that task and then kind of, you know, stepping through those tasks to get the job done. @@ -1374,7 +1374,7 @@ 00:34:31 Right. -00:34:31 And there's all these different, and there's another great place for Lama Index and Langchain. +00:34:31 And there's all these different, and there's another great place for Lama Index and LangChain. 00:34:35 They have chunking strategies where they'll take a big giant document and break it down diff --git a/transcripts/456-building-custom-gpts-with-python.vtt b/transcripts/456-building-custom-gpts-with-python.vtt index ecfaacd..2a91001 100644 --- a/transcripts/456-building-custom-gpts-with-python.vtt +++ b/transcripts/456-building-custom-gpts-with-python.vtt @@ -979,7 +979,7 @@ Like, it's great to do a prototype with one of these models or recognize you mig to switch either for cost reasons or performance reasons or what have you. 00:14:58.120 --> 00:15:03.520 -And, you know, Langchain, for instance, has a ton of the providers as part of you basically +And, you know, LangChain, for instance, has a ton of the providers as part of you basically 00:15:03.520 --> 00:15:07.900 are just switching a few arguments when you're switching between them. @@ -997,10 +997,10 @@ you can just add plugins as you need them, right? So you don't have to install all the different libraries that are out there. 00:15:22.960 --> 00:15:25.040 -And I think Langchain is kind of following a similar approach. +And I think LangChain is kind of following a similar approach. 00:15:25.040 --> 00:15:29.660 -I think they're coming up with a Langchain core capability where you can just kind of bring +I think they're coming up with a LangChain core capability where you can just kind of bring 00:15:29.660 --> 00:15:30.760 in things as you need them. @@ -1135,10 +1135,10 @@ So you can kind of be flexible. All right. 00:17:42.760 --> 00:17:44.280 -So you talked about Langchain. +So you talked about LangChain. 00:17:44.280 --> 00:17:46.660 -Just give us a sense real quick of what Langchain is. +Just give us a sense real quick of what LangChain is. 00:17:46.660 --> 00:17:49.540 This was a great project from a timing perspective. @@ -1147,7 +1147,7 @@ This was a great project from a timing perspective. I believe they kind of invented it and released it right around the time ChatGPT came out. 00:17:53.500 --> 00:18:00.060 -It's a very comprehensive library with lots of, I mean, the best part about Langchain to me is the documentation and the code samples. +It's a very comprehensive library with lots of, I mean, the best part about LangChain to me is the documentation and the code samples. 00:18:00.060 --> 00:18:00.420 Right. @@ -1177,7 +1177,7 @@ They're not going to get close. They're not close to 100% accurate on these types of tasks. 00:18:33.480 --> 00:18:37.860 -You know, the idea of agents as well as another thing that you might build with a Langchain. +You know, the idea of agents as well as another thing that you might build with a LangChain. 00:18:38.100 --> 00:18:47.740 And the idea there is basically the agent is, you know, getting a task, coming up with a plan of that for that task and then kind of, you know, stepping through those tasks to get the job done. @@ -2185,7 +2185,7 @@ questions like, okay, well, how do I chunk up the documents that I have? Right. 00:34:31.900 --> 00:34:35.620 -And there's all these different, and there's another great place for Lama Index and Langchain. +And there's all these different, and there's another great place for Lama Index and LangChain. 00:34:35.620 --> 00:34:39.780 They have chunking strategies where they'll take a big giant document and break it down diff --git a/transcripts/468-python-trends-of-2024.txt b/transcripts/468-python-trends-of-2024.txt index c1ba374..e29979b 100644 --- a/transcripts/468-python-trends-of-2024.txt +++ b/transcripts/468-python-trends-of-2024.txt @@ -158,7 +158,7 @@ 00:06:51 And there are open source packages like that. -00:06:52 Langchain is the most popular. +00:06:52 LangChain is the most popular. 00:06:54 And the downloads of that one have doubled in the last six months. diff --git a/transcripts/468-python-trends-of-2024.vtt b/transcripts/468-python-trends-of-2024.vtt index f62e089..3bdc14a 100644 --- a/transcripts/468-python-trends-of-2024.vtt +++ b/transcripts/468-python-trends-of-2024.vtt @@ -241,7 +241,7 @@ So LLMs are starting to be used as part of more multi-part applications. And there are open source packages like that. 00:06:52.920 --> 00:06:54.280 -Langchain is the most popular. +LangChain is the most popular. 00:06:54.280 --> 00:06:57.360 And the downloads of that one have doubled in the last six months. diff --git a/transcripts/476-unified-python-packaging-with-uv.txt b/transcripts/476-unified-python-packaging-with-uv.txt index 8dca44f..f19f371 100644 --- a/transcripts/476-unified-python-packaging-with-uv.txt +++ b/transcripts/476-unified-python-packaging-with-uv.txt @@ -1210,7 +1210,7 @@ 00:34:10 One is we have this sort of like tool install, kind of like pipX. -00:34:14 So if you want to install rough globally, you can do uv tool install rough and we'll install +00:34:14 So if you want toinstall ruff globally, you can do uv toolinstall ruff and we'll install 00:34:20 it for you. @@ -2420,7 +2420,7 @@ 01:00:35 Like I now use UVX all the time when I want to run things. -01:00:38 So I'll do like UVX rough check and that translates to uv tool run rough, you know, run the check command behind the scenes. +01:00:38 So I'll do like UVX rough check and that translates to uv tool run ruff, you know, run the check command behind the scenes. 01:00:46 What it does is it finds latest version of rough. @@ -2456,7 +2456,7 @@ 01:01:38 I think for things you're going to run it repeatedly, it makes a lot of sense to install them. -01:01:42 also if you have, like, if you ran uv tool install rough, and then you ran like UVX rough, we would use the installed version. +01:01:42 also if you have, like, if you ran uv toolinstall ruff, and then you ran like UVX rough, we would use the installed version. 01:01:51 So if you already have the tool installed, we'll still pull the installed version, assuming it's compatible with the version request that you made. diff --git a/transcripts/476-unified-python-packaging-with-uv.vtt b/transcripts/476-unified-python-packaging-with-uv.vtt index d33e31b..63de0f9 100644 --- a/transcripts/476-unified-python-packaging-with-uv.vtt +++ b/transcripts/476-unified-python-packaging-with-uv.vtt @@ -1831,7 +1831,7 @@ You know, one is like the Python installation. One is we have this sort of like tool install, kind of like pipX. 00:34:14.820 --> 00:34:20.000 -So if you want to install rough globally, you can do uv tool install rough and we'll install +So if you want toinstall ruff globally, you can do uv toolinstall ruff and we'll install 00:34:20.000 --> 00:34:20.400 it for you. @@ -3652,7 +3652,7 @@ and there's a lot of really cool stuff here. Like I now use UVX all the time when I want to run things. 01:00:38.420 --> 01:00:46.420 -So I'll do like UVX rough check and that translates to uv tool run rough, you know, run the check command behind the scenes. +So I'll do like UVX rough check and that translates to uv tool run ruff, you know, run the check command behind the scenes. 01:00:46.420 --> 01:00:48.980 What it does is it finds latest version of rough. @@ -3706,7 +3706,7 @@ UV tool. I think for things you're going to run it repeatedly, it makes a lot of sense to install them. 01:01:42.660 --> 01:01:51.620 -also if you have, like, if you ran uv tool install rough, and then you ran like UVX rough, we would use the installed version. +also if you have, like, if you ran uv toolinstall ruff, and then you ran like UVX rough, we would use the installed version. 01:01:51.620 --> 01:01:58.740 So if you already have the tool installed, we'll still pull the installed version, assuming it's compatible with the version request that you made. diff --git a/transcripts/482-pre-commit-hooks-for-python-devs.txt b/transcripts/482-pre-commit-hooks-for-python-devs.txt index c9685a7..a66215e 100644 --- a/transcripts/482-pre-commit-hooks-for-python-devs.txt +++ b/transcripts/482-pre-commit-hooks-for-python-devs.txt @@ -366,7 +366,7 @@ 00:16:04 So if you want to run more, you basically have to, potentially, write a program which then itself -00:16:10 figures out all the things to do and then delegates to running them. Like if you want to run rough with +00:16:10 figures out all the things to do and then delegates to running them. Like if you want to run ruff with 00:16:16 a fixed formatting issues and you want to run the checker fixer for NumPy doc strings and all those diff --git a/transcripts/482-pre-commit-hooks-for-python-devs.vtt b/transcripts/482-pre-commit-hooks-for-python-devs.vtt index 5a973f8..b5044bf 100644 --- a/transcripts/482-pre-commit-hooks-for-python-devs.vtt +++ b/transcripts/482-pre-commit-hooks-for-python-devs.vtt @@ -556,7 +556,7 @@ TALKPYTHON, all capital, no spaces, it's good for two free months of Sentry's bu So if you want to run more, you basically have to, potentially, write a program which then itself 00:16:10.800 --> 00:16:16.440 -figures out all the things to do and then delegates to running them. Like if you want to run rough with +figures out all the things to do and then delegates to running them. Like if you want to run ruff with 00:16:16.440 --> 00:16:23.860 a fixed formatting issues and you want to run the checker fixer for NumPy doc strings and all those diff --git a/transcripts/488-lancedb.txt b/transcripts/488-lancedb.txt index d50115f..00a0139 100644 --- a/transcripts/488-lancedb.txt +++ b/transcripts/488-lancedb.txt @@ -1330,7 +1330,7 @@ 00:49:47 So the prompt engineering and the calling the completion model once you have the context retrieved, that's not part of the Lance DB API. -00:49:57 That said, so we integrate with, let's say, like a Langchain or a Llama Index. +00:49:57 That said, so we integrate with, let's say, like a LangChain or a Llama Index. 00:50:02 So if you're comfortable with that layer of sort of AI orchestration or RAG orchestration, you can use their APIs and plug in Lance DB into that. diff --git a/transcripts/488-lancedb.vtt b/transcripts/488-lancedb.vtt index 0dd7089..00b1ac5 100644 --- a/transcripts/488-lancedb.vtt +++ b/transcripts/488-lancedb.vtt @@ -2020,7 +2020,7 @@ And then the actual completion that's outside of the scope of Lance DB. So the prompt engineering and the calling the completion model once you have the context retrieved, that's not part of the Lance DB API. 00:49:57.580 --> 00:50:02.120 -That said, so we integrate with, let's say, like a Langchain or a Llama Index. +That said, so we integrate with, let's say, like a LangChain or a Llama Index. 00:50:02.120 --> 00:50:13.260 So if you're comfortable with that layer of sort of AI orchestration or RAG orchestration, you can use their APIs and plug in Lance DB into that. diff --git a/transcripts/504-developer-trends.txt b/transcripts/504-developer-trends.txt index 19ea483..42344d3 100644 --- a/transcripts/504-developer-trends.txt +++ b/transcripts/504-developer-trends.txt @@ -1902,7 +1902,7 @@ 00:59:35 There's so many good open source options, too. -00:59:37 I know a lot of people are excited about like Cloud Code and Copilot, but the Ader chats, the Goose clients, those things are really incredible and they're open source. +00:59:37 I know a lot of people are excited about like Claude Code and Copilot, but the Ader chats, the Goose clients, those things are really incredible and they're open source. 00:59:50 They all run local. diff --git a/transcripts/504-developer-trends.vtt b/transcripts/504-developer-trends.vtt index 85220b9..a6000ca 100644 --- a/transcripts/504-developer-trends.vtt +++ b/transcripts/504-developer-trends.vtt @@ -3553,7 +3553,7 @@ And so nothing's going off my laptop to interoperate with a lot of my tooling. There's so many good open source options, too. 00:59:37.440 --> 00:59:50.620 -I know a lot of people are excited about like Cloud Code and Copilot, but the Ader chats, the Goose clients, those things are really incredible and they're open source. +I know a lot of people are excited about like Claude Code and Copilot, but the Ader chats, the Goose clients, those things are really incredible and they're open source. 00:59:50.770 --> 00:59:51.460 They all run local. diff --git a/transcripts/507-agentic-ai-workflows-with-langgraph.txt b/transcripts/507-agentic-ai-workflows-with-langgraph.txt index 37be1a5..317e0c8 100644 --- a/transcripts/507-agentic-ai-workflows-with-langgraph.txt +++ b/transcripts/507-agentic-ai-workflows-with-langgraph.txt @@ -338,11 +338,11 @@ 00:18:30 Thank you to Posit for supporting the show. -00:18:33 Well, let's talk a bit about, I guess, Langchain first. +00:18:33 Well, let's talk a bit about, I guess, LangChain first. 00:18:37 So there's a lot of things going on here, right? -00:18:39 There's Langchain, the company. +00:18:39 There's LangChain, the company. 00:18:41 There's LangChain, the open source project. @@ -366,7 +366,7 @@ 00:19:43 There are kind of different response formats and features, et cetera. -00:19:47 And so Langchain makes it really easy for developers to use all of those different model providers in a single application. +00:19:47 And so LangChain makes it really easy for developers to use all of those different model providers in a single application. 00:19:53 That's interesting. @@ -382,7 +382,7 @@ 00:20:10 And then if you want to use that LLM and then start to use some other more commercially available ones as well, you can combine those. -00:20:17 And so the name Langchain comes together when we think about chains of systems. +00:20:17 And so the name LangChain comes together when we think about chains of systems. 00:20:21 One common example that we use is RAG, right? diff --git a/transcripts/507-agentic-ai-workflows-with-langgraph.vtt b/transcripts/507-agentic-ai-workflows-with-langgraph.vtt index 70049ed..61a7115 100644 --- a/transcripts/507-agentic-ai-workflows-with-langgraph.vtt +++ b/transcripts/507-agentic-ai-workflows-with-langgraph.vtt @@ -559,13 +559,13 @@ The link is in your podcast player's show notes. Thank you to Posit for supporting the show. 00:18:33.480 --> 00:18:36.660 -Well, let's talk a bit about, I guess, Langchain first. +Well, let's talk a bit about, I guess, LangChain first. 00:18:37.500 --> 00:18:39.380 So there's a lot of things going on here, right? 00:18:39.460 --> 00:18:41.000 -There's Langchain, the company. +There's LangChain, the company. 00:18:41.820 --> 00:18:44.340 There's LangChain, the open source project. @@ -601,7 +601,7 @@ and then the Anthropic API, and then trying to work with Gemini as well. There are kind of different response formats and features, et cetera. 00:19:47.060 --> 00:19:53.680 -And so Langchain makes it really easy for developers to use all of those different model providers in a single application. +And so LangChain makes it really easy for developers to use all of those different model providers in a single application. 00:19:53.940 --> 00:19:54.440 That's interesting. @@ -628,7 +628,7 @@ which is nice just from an extensibility point of view. And then if you want to use that LLM and then start to use some other more commercially available ones as well, you can combine those. 00:20:17.220 --> 00:20:21.240 -And so the name Langchain comes together when we think about chains of systems. +And so the name LangChain comes together when we think about chains of systems. 00:20:21.500 --> 00:20:24.100 One common example that we use is RAG, right? diff --git a/transcripts/517-agentic-ai.txt b/transcripts/517-agentic-ai.txt index 5d6d222..2f75a3f 100644 --- a/transcripts/517-agentic-ai.txt +++ b/transcripts/517-agentic-ai.txt @@ -1,4 +1,4 @@ -00:00:00 Agentic AI programming is what happens when coding assistants stop acting like autocomplete and start collaborating on real work. In this episode, we cut through the hype and incentives to define agentic, then get hands-on with how tools like Cursor, CloudCode, and Langchain actually behave inside an established code base. Our guest, Matt McKay, now the Vice President of Developer Relations at DigitalOcean, the creator of FullStackPython, and PlushCap, shares hard-won tactics. +00:00:00 Agentic AI programming is what happens when coding assistants stop acting like autocomplete and start collaborating on real work. In this episode, we cut through the hype and incentives to define agentic, then get hands-on with how tools like Cursor, Claude Code, and LangChain actually behave inside an established code base. Our guest, Matt Makai, now the Vice President of Developer Relations at DigitalOcean, the creator of FullStackPython, and PlushCap, shares hard-won tactics. 00:00:29 We unpack what breaks from brittle, generate a bunch of test requests to agents amplifying technical debt and uneven design patterns. @@ -298,7 +298,7 @@ 00:14:19 There's inputs into the system and non-deterministic outputs that come out the other side. -00:14:22 So you'll say something like, write me a bunch of unit tests and in Cloud Code or in Cursor or in Windsurf, and then it will interpret those inputs and then produce code or some sort of output out the other side. +00:14:22 So you'll say something like, write me a bunch of unit tests and in Claude Code or in Cursor or in Windsurf, and then it will interpret those inputs and then produce code or some sort of output out the other side. 00:14:35 So I think for developers who are trying to get into, how do I even understand kind of the AI space? @@ -336,7 +336,7 @@ 00:16:02 slop. It just makes up a bunch of mistakes. I noticed a couple of things that are often the case when I hear those and have people avoid that, right? So one, a lot of times I feel like people are not getting the best experience. They say like, I tried this. It really wasn't for me. -00:16:20 It just messes up more than it provides. They're using the cheapest free models that they can find, right? If you use, you know, Cloud's Opus model versus some free model, you know, like a 3 billion parameter local model they're not even in the same category the type of accuracy and like insight and like the context do they they understand everything rather than well they only understand the last file they read like that that is like one half of the problem and then i think the other for people who are not being successful yet with this kind of stuff it has to do with not providing enough information and context and stuff in the prompt so i'll see like oh refactor this function it's like well, hold on. What, where, where do you even want it to go? Right. It's just going to like, start randomly doing stuff. Here's a function. It is similar to these. And I need to move this to this kind of design pattern, keeping in mind that I'm using this ORM and like, give it like the same amount of description you would to a junior developer who's not super familiar with your project. So you want to give that thing like a possibly a multi-page write-up of what it needs to do and then ask it to plan it out and then start working through the plan, not just refactor +00:16:20 It just messes up more than it provides. They're using the cheapest free models that they can find, right? If you use, you know, Claude's Opus model versus some free model, you know, like a 3 billion parameter local model they're not even in the same category the type of accuracy and like insight and like the context do they they understand everything rather than well they only understand the last file they read like that that is like one half of the problem and then i think the other for people who are not being successful yet with this kind of stuff it has to do with not providing enough information and context and stuff in the prompt so i'll see like oh refactor this function it's like well, hold on. What, where, where do you even want it to go? Right. It's just going to like, start randomly doing stuff. Here's a function. It is similar to these. And I need to move this to this kind of design pattern, keeping in mind that I'm using this ORM and like, give it like the same amount of description you would to a junior developer who's not super familiar with your project. So you want to give that thing like a possibly a multi-page write-up of what it needs to do and then ask it to plan it out and then start working through the plan, not just refactor 00:17:36 to be, you know, to do this or whatever. @@ -492,11 +492,11 @@ 00:25:33 security. Just get refreshed and just give it a moment. And then boom, you can see a report of all the stuff that's changed across all the parts. Even on something amazing is like chat CPT. You agentic. So maybe like, let's, that's a long winded way of like saying, tell us about the magic of like, what is this agent and tool using aspect for it? Cause I think when I said my rant, it's not working for people. I think a lot of times people are not doing agentic AI. They're asking LLM to write functions or stuff, which is great, but it's not the same. Yeah. Well, I think, I would say -00:26:05 like kind of the biggest thing, and I mean, there's like, there's multiple attributes to it. Like, again, going back to some of the definitions, it's like you have an LLM and that LLM, if it's running chatGPT on your web browser, it's not going to have access to all the stuff in your codebase. +00:26:05 like kind of the biggest thing, and I mean, there's like, there's multiple attributes to it. Like, again, going back to some of the definitions, it's like you have an LLM and that LLM, if it's running ChatGPT on your web browser, it's not going to have access to all the stuff in your codebase. 00:26:19 Unless you have a public GitHub repository or something. But generally, when you're working in your local development environment, it's not going to have access to that stuff. -00:26:28 To be fair, there are some tools that will take enough context and upload it to chatGPT. But again, what you're starting to do is you're starting to get really far away from a natural workflow and into one in which you have to bend to how the LLM is set up. +00:26:28 To be fair, there are some tools that will take enough context and upload it to ChatGPT. But again, what you're starting to do is you're starting to get really far away from a natural workflow and into one in which you have to bend to how the LLM is set up. 00:26:46 So to me, I think the simplest way that I would look at it as agentic is like, it's something that is running side by side with you. @@ -562,7 +562,7 @@ 00:29:39 Or there could be open weighted models in the future. -00:29:42 Like I can just run Olama and I could run either Cloud Code or Open Code or some sort of CLI that would allow me to just do the same things, right? +00:29:42 Like I can just run Olama and I could run either Claude Code or Open Code or some sort of CLI that would allow me to just do the same things, right? 00:29:52 I mean, may not exactly, but roughly the same things. @@ -650,7 +650,7 @@ 00:33:02 But I will say that I very much appreciate the opinionated tools and frameworks. -00:33:07 And that's why I've actually had a better experience using Cloud Code with Django. +00:33:07 And that's why I've actually had a better experience using Claude Code with Django. 00:33:15 And a big piece is also, I've written thousands, if not tens of thousands of lines of code already in Plush Cap, which is typically what I'm building, if not some side scripts and stuff like that. @@ -936,7 +936,7 @@ 00:49:16 So I think maybe just like some quick shout outs to some of the tools, right? -00:49:21 So Cloud Code obviously is a really interesting one. +00:49:21 So Claude Code obviously is a really interesting one. 00:49:24 You've been talking about that a lot. diff --git a/transcripts/517-agentic-ai.vtt b/transcripts/517-agentic-ai.vtt index 1707a76..7bb683a 100644 --- a/transcripts/517-agentic-ai.vtt +++ b/transcripts/517-agentic-ai.vtt @@ -1,7 +1,7 @@ WEBVTT 00:00:00.120 --> 00:00:28.740 -Agentic AI programming is what happens when coding assistants stop acting like autocomplete and start collaborating on real work. In this episode, we cut through the hype and incentives to define agentic, then get hands-on with how tools like Cursor, CloudCode, and Langchain actually behave inside an established code base. Our guest, Matt McKay, now the Vice President of Developer Relations at DigitalOcean, the creator of FullStackPython, and PlushCap, shares hard-won tactics. +Agentic AI programming is what happens when coding assistants stop acting like autocomplete and start collaborating on real work. In this episode, we cut through the hype and incentives to define agentic, then get hands-on with how tools like Cursor, Claude Code, and LangChain actually behave inside an established code base. Our guest, Matt Makai, now the Vice President of Developer Relations at DigitalOcean, the creator of FullStackPython, and PlushCap, shares hard-won tactics. 00:00:29.140 --> 00:00:38.000 We unpack what breaks from brittle, generate a bunch of test requests to agents amplifying technical debt and uneven design patterns. @@ -463,7 +463,7 @@ And that is kind of almost think about it as like the core. There's inputs into the system and non-deterministic outputs that come out the other side. 00:14:22.900 --> 00:14:35.520 -So you'll say something like, write me a bunch of unit tests and in Cloud Code or in Cursor or in Windsurf, and then it will interpret those inputs and then produce code or some sort of output out the other side. +So you'll say something like, write me a bunch of unit tests and in Claude Code or in Cursor or in Windsurf, and then it will interpret those inputs and then produce code or some sort of output out the other side. 00:14:35.700 --> 00:14:40.400 So I think for developers who are trying to get into, how do I even understand kind of the AI space? @@ -520,7 +520,7 @@ thoughts on this. Okay. Yeah. So when I hear people say, I've tried AI, it's a b slop. It just makes up a bunch of mistakes. I noticed a couple of things that are often the case when I hear those and have people avoid that, right? So one, a lot of times I feel like people are not getting the best experience. They say like, I tried this. It really wasn't for me. 00:16:20.100 --> 00:17:36.900 -It just messes up more than it provides. They're using the cheapest free models that they can find, right? If you use, you know, Cloud's Opus model versus some free model, you know, like a 3 billion parameter local model they're not even in the same category the type of accuracy and like insight and like the context do they they understand everything rather than well they only understand the last file they read like that that is like one half of the problem and then i think the other for people who are not being successful yet with this kind of stuff it has to do with not providing enough information and context and stuff in the prompt so i'll see like oh refactor this function it's like well, hold on. What, where, where do you even want it to go? Right. It's just going to like, start randomly doing stuff. Here's a function. It is similar to these. And I need to move this to this kind of design pattern, keeping in mind that I'm using this ORM and like, give it like the same amount of description you would to a junior developer who's not super familiar with your project. So you want to give that thing like a possibly a multi-page write-up of what it needs to do and then ask it to plan it out and then start working through the plan, not just refactor +It just messes up more than it provides. They're using the cheapest free models that they can find, right? If you use, you know, Claude's Opus model versus some free model, you know, like a 3 billion parameter local model they're not even in the same category the type of accuracy and like insight and like the context do they they understand everything rather than well they only understand the last file they read like that that is like one half of the problem and then i think the other for people who are not being successful yet with this kind of stuff it has to do with not providing enough information and context and stuff in the prompt so i'll see like oh refactor this function it's like well, hold on. What, where, where do you even want it to go? Right. It's just going to like, start randomly doing stuff. Here's a function. It is similar to these. And I need to move this to this kind of design pattern, keeping in mind that I'm using this ORM and like, give it like the same amount of description you would to a junior developer who's not super familiar with your project. So you want to give that thing like a possibly a multi-page write-up of what it needs to do and then ask it to plan it out and then start working through the plan, not just refactor 00:17:36.930 --> 00:17:39.400 to be, you know, to do this or whatever. @@ -754,13 +754,13 @@ utility. I can just, when I sit down, I'm like, oh, it's been like a week since security. Just get refreshed and just give it a moment. And then boom, you can see a report of all the stuff that's changed across all the parts. Even on something amazing is like chat CPT. You agentic. So maybe like, let's, that's a long winded way of like saying, tell us about the magic of like, what is this agent and tool using aspect for it? Cause I think when I said my rant, it's not working for people. I think a lot of times people are not doing agentic AI. They're asking LLM to write functions or stuff, which is great, but it's not the same. Yeah. Well, I think, I would say 00:26:05.860 --> 00:26:18.280 -like kind of the biggest thing, and I mean, there's like, there's multiple attributes to it. Like, again, going back to some of the definitions, it's like you have an LLM and that LLM, if it's running chatGPT on your web browser, it's not going to have access to all the stuff in your codebase. +like kind of the biggest thing, and I mean, there's like, there's multiple attributes to it. Like, again, going back to some of the definitions, it's like you have an LLM and that LLM, if it's running ChatGPT on your web browser, it's not going to have access to all the stuff in your codebase. 00:26:19.360 --> 00:26:26.800 Unless you have a public GitHub repository or something. But generally, when you're working in your local development environment, it's not going to have access to that stuff. 00:26:28.040 --> 00:26:46.540 -To be fair, there are some tools that will take enough context and upload it to chatGPT. But again, what you're starting to do is you're starting to get really far away from a natural workflow and into one in which you have to bend to how the LLM is set up. +To be fair, there are some tools that will take enough context and upload it to ChatGPT. But again, what you're starting to do is you're starting to get really far away from a natural workflow and into one in which you have to bend to how the LLM is set up. 00:26:46.760 --> 00:26:53.480 So to me, I think the simplest way that I would look at it as agentic is like, it's something that is running side by side with you. @@ -859,7 +859,7 @@ So I was like, my bet was, okay, even if I use this and it's amazing, if it goes Or there could be open weighted models in the future. 00:29:42.350 --> 00:29:52.000 -Like I can just run Olama and I could run either Cloud Code or Open Code or some sort of CLI that would allow me to just do the same things, right? +Like I can just run Olama and I could run either Claude Code or Open Code or some sort of CLI that would allow me to just do the same things, right? 00:29:52.180 --> 00:29:54.280 I mean, may not exactly, but roughly the same things. @@ -994,7 +994,7 @@ And in fact, like the simpler is often better. But I will say that I very much appreciate the opinionated tools and frameworks. 00:33:07.920 --> 00:33:15.060 -And that's why I've actually had a better experience using Cloud Code with Django. +And that's why I've actually had a better experience using Claude Code with Django. 00:33:15.660 --> 00:33:27.460 And a big piece is also, I've written thousands, if not tens of thousands of lines of code already in Plush Cap, which is typically what I'm building, if not some side scripts and stuff like that. @@ -1438,7 +1438,7 @@ Yeah, exactly. So I think maybe just like some quick shout outs to some of the tools, right? 00:49:21.060 --> 00:49:23.940 -So Cloud Code obviously is a really interesting one. +So Claude Code obviously is a really interesting one. 00:49:24.200 --> 00:49:25.200 You've been talking about that a lot. diff --git a/transcripts/518-django-birthday.txt b/transcripts/518-django-birthday.txt new file mode 100644 index 0000000..8e70256 --- /dev/null +++ b/transcripts/518-django-birthday.txt @@ -0,0 +1,1738 @@ +00:00:00 20 years after a scrappy newsroom team hacked together a framework to ship stories fast, Django remains the Python web framework that ships real apps responsibly. + +00:00:09 In this anniversary roundtable with its creators and longtime stewards, Simon Wilson, Adrian Holovaty, Will Vincent, Jeff Triplett, and Tiavod Kolas, we trace the path from the Lawrence Journal world to version 1.0, DjangoCon, and the Django Software Foundation. + +00:00:26 Unpack how a BSD license and a culture of docs, tests, and mentorship grew a global community, and revisit lessons from deployments like Instagram. + +00:00:35 We talk modern Django too, ASGI and async, HTMX-friendly patterns, building APIs with DRF and Django Ninja, and how Django pairs with React and Serverless without losing its batteries-included soul. + +00:00:48 You'll hear about Django Girls, Django Nauts, and the Django Fellowship that keeps momentum going, plus where Django fits in today's AI stacks. + +00:00:57 Finally, we look ahead at the next decade of speed, security, and sustainability for Django. + +00:01:02 This is Talk Python To Me, episode 518, recorded August 18th, 2025. + +00:01:19 It's time to unite. We started in pyramid cruising. Welcome to Talk Python To Me, a weekly podcast + +00:01:25 on Python. This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is brought to you entirely by Sentry. It's a bit of an episode takeover, if you will. Sentry has two excellent and exciting services to tell you about. Sear, your agentic AI debugging assistant, which takes all the data already gathered by Sentry to help discover the problems and even propose fixes as GitHub PRs. And the other is AI agent monitoring, which adds deep observability to your AI agents in your app. If you're adding AI and LLM features to your Python apps, you'll want to know about AI agent monitoring. I'll tell you more about both of these later in the episode. And remember, however you happen to sign up for Sentry, if you do, use our code TALKPYTHON, one word, all caps. + +00:02:37 Let's get to the interview. + +00:02:39 Hello, everyone. + +00:02:40 Welcome to Talk Python To Me. + +00:02:41 I'm so excited to be celebrating Django's 20th birthday. + +00:02:46 What a milestone. + +00:02:47 We have Will Vincent, we have Adrian, we have Theobald, Jeff, we have Simon. + +00:02:52 Many of the original people have the origin of Django here to talk about it. + +00:02:56 So, you know, welcome all of you. + +00:02:58 Thanks, Michael. + +00:02:59 Yeah, you bet. + +00:03:00 It's going to be a lot of fun to go back to some stories and maybe also see where things are going after 20 years. + +00:03:06 It's still going strong, to be sure. + +00:03:09 So what an accomplishment. + +00:03:10 Let's just start with a real quick, who am I, introduction from each of you. + +00:03:15 Just give us the elevator pitch on you and your Django life. + +00:03:18 And we'll go around the Brady Bunches in order, I suppose, here on the screen. + +00:03:21 So Will, welcome. + +00:03:23 Okay. + +00:03:23 So we'll go from least to most important. + +00:03:26 So I'm a developer advocate at JetBrains, some Django books, a podcast. + +00:03:31 I think that's enough. + +00:03:31 We have more important people to come. + +00:03:33 All right. + +00:03:34 Adrian, welcome. + +00:03:35 First time to have you on the show. Happy to have you. + +00:03:37 Hi, I'm Adrian, one of the creators of Django many years ago, precisely 20, a long time web developer. + +00:03:45 And yeah, it's basically it. + +00:03:47 I'm Simon. I'm Adrian's intern, effectively. + +00:03:51 We built the first version of Django during my year long paid internship at the Longest Journal World in Kansas. + +00:03:56 It was my university had a year in industry thing, which meant I could get a visa. + +00:04:01 And yeah, we worked on what we called the CMS at the time. + +00:04:05 We didn't know that it was a web framework. + +00:04:07 We thought it was a tool for building local newspaper. + +00:04:09 Yeah. + +00:04:10 Almost got named TPS, which would be amazing. + +00:04:13 Well, I'm a... + +00:04:14 Oh, go ahead. + +00:04:15 Sorry. + +00:04:15 I was going to say, Simon, you're almost more well-known for your AI stuff these days. + +00:04:20 You've been... + +00:04:20 Last couple of years. + +00:04:22 Like, last two years. + +00:04:23 You've been in the news so much. + +00:04:24 I see you everywhere now. + +00:04:25 It's like, congratulations on having two... + +00:04:28 Yeah. + +00:04:29 I mean, that was... + +00:04:30 was. The trick there is I have a blog and not everyone else gave up on blogging, but I kept going. And so when AI started up, I was the only person blogging about AI, which turns out gets you invited to all of the weird Silicon Valley mansion parties and things. So yeah, that's been an + +00:04:44 interesting little twist over the past couple of years. Yeah. Jeff Triplett, I'm so sorry to cut + +00:04:48 you off. Happy to have you. No, no, no. All good. I thought it was pretty free for him, but I came to Lawrence, Kansas two years after these two had left, maybe three years after these two had left To work at the newspaper, I've stuck around. + +00:05:00 Probably one of the few people left that stuck around that worked at the journal world. + +00:05:03 Me and Frank Wiles, who's over here someplace. + +00:05:06 I'm on the Django Software Foundation Board of Directors. + +00:05:08 I've helped with DjangoCon US for, I think, 11 years now. + +00:05:11 And do a newsletter with Will and a bunch of random Django projects. + +00:05:15 So that's me. + +00:05:15 Jeff, were you there for the whole of Media for Media? + +00:05:18 The commercial arm of the newspaper doing Django stuff? + +00:05:22 About three years of it. + +00:05:23 It just had moved across the street to the Little Red building they called Little Red. + +00:05:27 And so I think I moved when I moved here was within probably like a month or two of that starting in the building. + +00:05:33 Awesome. I'd love to I'd love to hear little stories from that era of Django. + +00:05:37 I missed that entirely. + +00:05:38 I'm on the board with Jeff. I'm the president of the foundation, Django Software Foundation for 2025. + +00:05:44 And I've been involved with Django for only 10 years, only 10 years. + +00:05:49 So much less than everyone is here. + +00:05:51 Yeah. Outside that, I'm a dev. + +00:05:53 I'm on the core team for an open source project called Wagtail, which is a CMS as well built on Django. + +00:05:59 Yeah, welcome. + +00:06:00 And Wagtail is a really cool project. + +00:06:03 I think people who are looking to take the CMS idea and level it up even more. + +00:06:08 And you know, I've got a little bit of Wagtail related career history. + +00:06:13 Wagtail came out of a company called Torchbox. + +00:06:15 I worked for Torchbox for about six months to a year quite early on, working on one of the first Django projects that Torchbox worked on. + +00:06:23 And it was a carbon calculator that we were building. + +00:06:26 I have nothing to do with Wagtail at all. + +00:06:27 That was probably 10 years after I left Torchbox. + +00:06:29 But I know the Torchbox team really well. + +00:06:31 We should also, I mean, Jeff, amongst other things, you run Django packages.org right now. + +00:06:36 And I think Wagtail is number one or two with Django REST framework for most downloaded. + +00:06:40 It's something like that. + +00:06:41 It's one of the top three. + +00:06:42 I feel confident saying that. + +00:06:44 Maybe you don't. + +00:06:44 I do. + +00:06:46 Yeah, it's downloaded a lot. + +00:06:47 I don't know if it's top 25. + +00:06:49 Tebow with it. + +00:06:50 I think it's in the top 100. + +00:06:50 We did a post, right? + +00:06:51 So, you know, you're close. + +00:06:52 Okay, Tebow. + +00:06:53 Amongst 20,000 or so plus. + +00:06:56 But, yeah. + +00:06:57 Yeah, yeah, of Django ones. + +00:06:58 Wait, of Django ones? + +00:07:00 Oh. + +00:07:00 What is that? + +00:07:01 There's like 4,000 now, Jeff? + +00:07:02 Something like that? + +00:07:03 I think there's probably like 16,000, 20,000. + +00:07:06 But I think we're tracking like the top 4,000 or 5,000. + +00:07:09 It's what people... + +00:07:11 I never wanted to add code. + +00:07:12 They just pulled everything in just to have a lot of numbers. + +00:07:14 So, it's still like user submitted and ran. + +00:07:18 Oh, it is the most important thing. + +00:07:19 But Tebow, GitHub stars. + +00:07:20 That's the most important thing. + +00:07:21 How many is it for Wagtail? + +00:07:22 But yeah, I think we just like almost at 20,000. + +00:07:25 I'm pretty sure if you're on here listening to this show, stars right now will be at 20,000 in like a week. + +00:07:30 Yeah, it's really high in downloads as well. + +00:07:32 But you know, the Django ecosystem is really big these days and CMS only definitely in a smaller portion of it. + +00:07:37 Oh, there you go. + +00:07:38 Very, very, very popular. + +00:07:40 That's amazing. + +00:07:41 So let's start at the beginning, folks. + +00:07:43 I have on the screen here a picture of the University of Kansas where I also have a connection to. + +00:07:50 I actually went here and got my undergraduate degree in math at this school. + +00:07:54 And I love my time in Lawrence. + +00:07:56 I think it's a lovely town, but it's also the home of Lawrence Journal World, right? + +00:08:02 Tell us who wants to start the story. + +00:08:05 Like what's the origin story here? + +00:08:07 Probably it makes sense for me to start it. + +00:08:09 Yeah, I think so. + +00:08:10 Let's do it. + +00:08:11 Back in 2002 slash 2003, early 2003, I joined that newspaper as a web developer. + +00:08:18 There was the guy who ran it was this guy called Rob Curley, who was seen as this big tech innovator in the news industry. + +00:08:25 And he was sort of recruited me and put together a great team of awesome, awesome web developers, believe it or not, in this tiny little newspaper in the middle of Kansas. + +00:08:35 We were tasked with building interactive database driven apps for the newspaper, which at that time was unheard of. + +00:08:43 Even these days, it's kind of weird for a newspaper to have developers on its staff. + +00:08:48 And at the time I was using PHP and yeah, fast forward, maybe a year into the job, we had an opening to hire another developer and I posted a job entry on my blog and Simon read it all the way in the UK. + +00:09:04 And we followed each other's blogs back when that was like the social media of the day. + +00:09:08 To be fair. + +00:09:09 Much more interesting than social media of today. + +00:09:11 At the time, I felt like there were about 100 people blogging about web development. + +00:09:16 And we were two of those. + +00:09:17 Yeah, and we all followed everyone. + +00:09:20 Did you guys have a blog role? + +00:09:22 Oh, yeah, absolutely. + +00:09:25 I think we were both massive fans of Mark Pilgrim. + +00:09:27 Mark Pilgrim was the web development of Python blogger that we were paying attention to. + +00:09:33 Unfortunately, he's not around anyway. + +00:09:34 I mean, he's alive, but he stopped blogging many, many years ago. + +00:09:38 Yeah, he wrote an online book called Dive into Python. + +00:09:41 And we both read that and we're like, oh, what's this Python thing? + +00:09:44 And thanks to the kindness or possibly ignorance of our boss, we just started using Python one day. + +00:09:52 We didn't really have to get much buy-in or anything. + +00:09:54 And we'd started making little bits and pieces in Python, never really with the intent of making a framework. + +00:10:00 But over time, at some point, we just decided, okay, we're switching to Python. + +00:10:06 No more PHP. + +00:10:07 just going to do everything in this new thing. And I remember the key thing for us is how do + +00:10:12 you deploy Python? Well, there was modpython, the Apache module, and we were just a little bit nervous about it because not many other people were using it. And so we're thinking, okay, well, we can try modpython, but what if it doesn't work? So, okay, well, if we have an abstraction layer between our code and modpython, like a request and a response object, then if modpython doesn't work out, we can swap it out for something else. We didn't, modpython worked great, But that abstraction layer, I think that was the origins of Django, that tiny little layer between R code and mod python, just in case. Classic thing you shouldn't do in programming, right? You're not going to need it. You don't need the abstraction layer, but we built one anyway. And fast forward 20 years and maybe that was a good idea after all. + +00:10:51 Is mod python still around? It's got to be, right? + +00:10:54 I don't know. I mean, modwhiskey, it was replaced by modwhiskey many years ago. + +00:10:59 But yeah, I don't know actually. I've not checked in on the ModPython project in a while. + +00:11:04 It's been last commit eight months ago to ModPython on GitHub. + +00:11:08 Oh, very smart. + +00:11:08 Yeah, yeah. + +00:11:09 That sounds alive. + +00:11:10 Yeah, yeah, it's definitely alive. + +00:11:12 And now we have WSGI and ASGI and that's sort of carrying on the same theme in a way. + +00:11:18 You know, WSGI kind of came out of the work we were doing in that we were working on Django, at the Journal World. The mailing list popped up, the Python web-sig mailing list. This mailing list popped up. We were very active on that initially, trying to pitch our request and response objects. + +00:11:38 We did not win with our response request objects, and Whiskey came out of that instead. But yeah, that was definitely... Whiskey came out of that early era of what we were trying to achieve with with the stuff we were doing. + +00:11:49 So we built up this series of libraries slash tools slash, I guess it turned into a framework. + +00:11:56 And we started thinking, oh, this is kind of interesting. + +00:11:59 Personally, speaking for myself, I was like, oh, I'm probably not going to work here forever, but I would love to keep using this. + +00:12:04 So that sort of started planting the seed about open sourcing it. + +00:12:07 But I think very few people today know that even 20 years ago, there was this meme that there are way too many Python web frameworks. + +00:12:15 So we went to PyCon US and it had to have been 2004. + +00:12:21 And there was a keynote where the speaker said, she showed a slide that was like a tag cloud of all the Python web framework names. + +00:12:29 And there were like 50 of them. + +00:12:31 So Simon and I were sitting there like, oh shoot, we made yet another one. + +00:12:36 We're fools. + +00:12:37 We're so stupid. + +00:12:38 We're embarrassed. + +00:12:39 But I ended up doing a lightning talk at that PyCon and showed what we had. + +00:12:44 It didn't have a name yet. + +00:12:46 And, you know, people came up to me afterward and said, oh, please open source. + +00:12:50 We're very interested in this. + +00:12:51 It looks very promising open source, even though it's yet another Python web framework. + +00:12:55 So I guess extrapolating a lesson from that, it's never too late to make stuff. + +00:13:01 I think another fact there is that Ruby on Rails had come out maybe six months before that PyCon. + +00:13:05 Like Ruby on Rails was the absolute buzz of everything in the web development industry at the time. + +00:13:11 And there was an open question like, what's Python's answer to Ruby on Rails? + +00:13:15 And so it's possible that the Django timing was really good on that front. + +00:13:19 I believe that Rails helped influence the newspaper to open source Django. + +00:13:23 Was that part of the conversation then? + +00:13:25 Well, I remember making a list of 10 to 20 reasons why we should open source it and really thinking hard about it, probably lost some sleep about it. + +00:13:35 And we went into the meeting with Rob to make our pitch. + +00:13:39 And we only got to like five reasons. + +00:13:42 And finally he was like, yeah, let's do it. + +00:13:44 So, I mean, endless, infinite credit to him for being cool. + +00:13:48 And I mean, the company took advantage of open source a lot. + +00:13:52 And I think the higher ups knew it because it also owned a cable company that did a lot of stuff with Linux and all sorts of crazy cable company software that was open source. + +00:14:02 Which was made Frank, right? + +00:14:03 That was Frank Wiles at the cable company driving all of that stuff. + +00:14:07 Yeah, you know, I think it's worth pointing out as well. One, Adrian, you're making me nostalgic just to think about tag clouds. Okay. They were such a thing back in the early 2000s. It's a usability nightmare. I know. Any kind of data or information out of that thing. + +00:14:24 It really is. + +00:14:25 I think I use, oh my God, Delicious a lot, and you'll get tag clouds out of it and all sorts of stuff. + +00:14:31 But that's not why I'm bringing this up. + +00:14:33 More in that timeframe, open sourcing something was not the clear choice it is today. + +00:14:39 At least it was really more of a push, I think. + +00:14:43 Legal liabilities, giving away our IP. + +00:14:45 Like it was a different time. + +00:14:46 There was open source, but not like today. + +00:14:50 This portion of Talk Python To Me is brought to you by Centuries Seer. + +00:14:54 I'm excited to share a new tool from Sentry, Seer. + +00:14:58 Seer is your AI-driven pair programmer that finds, diagnoses, and fixes code issues in your Python app faster than ever. + +00:15:05 If you're already using Sentry, you are already using Sentry, right? + +00:15:10 Then using Seer is as simple as enabling a feature on your already existing project. + +00:15:15 Seer taps into all the rich context Sentry has about an error. + +00:15:19 Stack traces, logs, commit history, performance data, essentially everything. + +00:15:23 Then it employs its agentic AI code capabilities to figure out what is wrong. + +00:15:28 It's like having a senior developer pair programming with you on bug fixes. + +00:15:32 Sear then proposes a solution, generating a patch for your code and even opening a GitHub pull request. + +00:15:38 This leaves the developers in charge because it's up to them to actually approve the PR. + +00:15:43 But it can reduce the time from error detection to fix dramatically. + +00:15:48 Developers who've tried it found it can fix errors in one shot that would have taken them hours to debug. + +00:15:54 SEER boasts a 94.5% accuracy in identifying root causes. + +00:15:59 SEER also prioritizes actionable issues with an actionability score, so you know what to fix first. + +00:16:06 This transforms sentry errors into actionable fixes, turning a pile of error reports into an ordered to-do list. + +00:16:14 If you could use an always-on-call AI agent to help track down errors and propose fixes before you even have time to read the notification, check out Sentry's Seer. + +00:16:24 Just visit talkpython.fm/seer, S-E-E-R. + +00:16:28 The link is in your podcast player's show notes. + +00:16:31 Be sure to use our code, TALKPYTHON. + +00:16:33 One word, all caps. + +00:16:35 Thank you to Sentry for supporting Talk Pythonemy. + +00:16:38 Well, also, the publisher of the newspaper at the time still used a mechanical typewriter to write up his things. + +00:16:45 He was legendary for it. + +00:16:47 But my understanding is that he got into it. + +00:16:49 The argument that was convincing to the newspaper was, "We benefit from this community. + +00:16:54 We should give back to this community." That evidently was part of the pitch. + +00:17:00 It just took off. + +00:17:03 I feel like that newspaper did very well from that one decision that they made back then. + +00:17:07 Yeah, I mean, here we are this many years later talking about it on a podcast, right? Like already, a lot of people know about it. And a lot of people, all of you on the call, but many, many, many others have contributed back to Django to make so much better. If that was just an internal CMS that you guys left there, how many external contributors would you have? Right? Not that many. + +00:17:27 Well, one thing, actually, I'm curious while the two of you are here, I heard a story that there were some efforts to monetize it that didn't pan out. + +00:17:34 Jeff, maybe you were there for that. + +00:17:36 Is that true? + +00:17:37 Were there some efforts to package it up and sell it to newspapers in some capacity as a premium product, I think? + +00:17:43 Am I making that up? + +00:17:44 That was after my time, I think. + +00:17:46 Are you talking about? + +00:17:47 Maybe Frank mentioned that. + +00:17:48 I thought there was something around. + +00:17:50 Yeah, this was Ellington. + +00:17:51 I think we should. + +00:17:52 Oh, I guess. + +00:17:53 Yeah. + +00:17:53 Sorry. + +00:17:53 to Jeff who um yeah this is kind of my era I guess then was um and I still feel like I'm new to Django because I felt like and I've been around for 18 years but like I just I saw these guys code so much and then I came into that era so to me it felt like next gen Star Trek versus now the originals um and so yeah when I started we had hundreds of newspapers were buying Ellington which was basically like what was it the first or second Django app ever written and they basically created a CMS to run your newspaper and then they also had something called marketplace which was almost like an online Craigslist or business directory. + +00:18:26 And so I came in to kind of help manage some of the inventory and do a lot of data imports and stuff for newspapers. + +00:18:32 Like we had all of Massachusetts at one point. + +00:18:35 There was a lot of kind of that, they went that direction with it. + +00:18:38 And I left before that kind of had shuttered and I think they sold it to a big company to buy it. + +00:18:43 And I think Ellington still runs to this day. + +00:18:46 At DjangoCon every two or three years, somebody will approach and say, we want to open source Ellington. + +00:18:50 Would you like to, could we do this at DjangoCon? + +00:18:53 We always say, "Yeah, that'd be really cool." Then six years has gone by and it happens every so often. + +00:18:58 I hope that code gets to see the light of day because it was hundreds of apps by the time we worked on it for hundreds of newspapers. + +00:19:05 I feel like the product was fantastic and the market was great, but it's a terrible market to be selling into. + +00:19:11 In the late 2000s, early 2010s, if you're selling to local newspapers, which are blinking out of existence at a rate of knots. + +00:19:21 My hunch is that the marketplace challenges were down to that. + +00:19:25 It was selling into a shrinking industry, which is a difficult thing to do. + +00:19:29 Yeah, I think they were hitting a brick wall for sure. + +00:19:31 You mentioned Rails earlier and PHP. + +00:19:34 And two things come to mind. + +00:19:35 I have an embarrassing email I sent to Simon at one point because I followed your PHP blog. + +00:19:40 And also Matt Croydon, who later became my boss at Media for Media. + +00:19:43 And I did not. + +00:19:44 I saw a couple of things. + +00:19:45 I think you were so into Django, you weren't really posting about Django. + +00:19:48 And so I think I emailed you once to say, have you seen this thing called Django? + +00:19:52 It looks pretty cool. + +00:19:53 Now that you're doing Python, I think you may like it. + +00:19:55 And so if you ever check your Gmail, it may be there. + +00:19:58 We don't have to do anything live on the air. + +00:20:00 But yeah, I to this day think, oh man, that was kind of wild. + +00:20:04 What brought me to Django though was I did get the Rails book because they very quickly created a Ruby on Rails book. + +00:20:10 It was kind of the main way to learn how to use it because their website kind of sucked as far as documentation went. + +00:20:14 And as I was going through it, so much was boilerplate just to get anything running in Rails. + +00:20:20 And so the Django admin was just so beautifully designed and well done that to me, it was just this win of like, this looks better than anything I could do by hand. + +00:20:28 And it's largely stood the test of time for 20 years. + +00:20:31 I remember like Wilson Miner I had met a couple of times had done some of that work or most of that work. + +00:20:36 He did all of that design work and he designed the original Django website. + +00:20:40 And I think that's one of the main reasons that we got a lot of traction was because the design of the site and the documentation were so good, especially considering the context. + +00:20:50 I mean, that back in those times, it was par for the course for an open source project to have to be completely on style, like a normal, like browser default H1 Craigslist background. + +00:21:02 This design isn't the one that Wilson did. + +00:21:04 The one that's archived that or get to go back. + +00:21:07 But it was pretty timeless at the time, though. + +00:21:09 It was amazing. + +00:21:10 And I feel like the admin has evolved a lot, but it still has that sort of... + +00:21:15 It feels like the Wilson minor design today, even 20 years later. + +00:21:19 Wilson was there the same time I was at the Journal World, and he then went on to early Facebook. + +00:21:25 He was, he's bounced around Silicon Valley. + +00:21:27 Yeah. + +00:21:28 Redesigned Apple.com, I think too. + +00:21:30 He worked with me at EveryBlock for a few years. + +00:21:33 He's now working on Apple Music. + +00:21:35 I didn't know that. + +00:21:35 Excellent. + +00:21:37 As of last time we talked to him, which was recent-ish. + +00:21:40 Yeah, that's quite neat. + +00:21:41 Trying to pull it up here in the Wayback Machine. + +00:21:43 We'll see what I can get. + +00:21:44 I think the other thing I... + +00:21:46 How about this, guys? + +00:21:46 Is this the OG? + +00:21:47 There we go. + +00:21:48 Yep, yep. + +00:21:50 That's the one. + +00:21:51 And that, to be fair, that looks good by today's standard. + +00:21:54 You know, I feel like I would not be at all upset to publish a website look like that in 2025. + +00:22:00 Yeah, that is pretty timeless. + +00:22:01 That's cool. + +00:22:02 Wait, I see the... + +00:22:03 Oh, go ahead, Simon. + +00:22:04 Oh, it's good. The other thing that's worth mentioning with Django, Django's documentation has always been fantastic. And I think that's basically because it was Jacob Kaplan Moss had a, he was a literature major and you have Adrian, who's a journalism major. And it turns out if you put a journalist and somebody who like put that talent on your documentation, you get really + +00:22:23 good documentation. There was, that was a double-edged sword because it took a few years for any publisher to be interested in publishing a book about Django because the standard line was always, oh, the docs are good enough. + +00:22:35 Everyone just, there's no market for it. + +00:22:38 Well, but at the same time, I wanted to ask, I mean, because there was the link there to the Django book, which I have the URL now and Jeff and I run, it lists all the Django books. + +00:22:47 But I mean, the two of you wrote that. + +00:22:49 I thought that was a great book. + +00:22:50 I think you, it was in print, right? + +00:22:52 But it was mainly open source, Adrian? + +00:22:54 Yeah, it was both. + +00:22:55 It was free online and it was published by A-Press. + +00:22:58 There were a couple of editions, yeah. + +00:23:00 The interesting question for Paul Everett out there in the audience is, In the first years, did your team also have to admin the newspaper site? + +00:23:07 Like, did you DevOps before DevOps was a thing? + +00:23:10 Or how did you run it? + +00:23:11 The cloud stuff wasn't that big back then. + +00:23:13 No, we had our own physical servers. + +00:23:15 In the basement. + +00:23:16 In the basement. + +00:23:17 The team was in the basement. + +00:23:18 You could probably hear them. + +00:23:19 Your programmers. + +00:23:20 And yeah, there was a room in the basement with physical servers. + +00:23:24 And oh, I've forgotten their names now. + +00:23:25 I think Frank helped. + +00:23:27 Well, they were named after characters in Beatles songs. + +00:23:30 One was Mr. Mustard. + +00:23:31 One was Pam from Polytheon Pam. + +00:23:33 Yes. Yep. But yeah, and Frank and Nick from Sunflat Cable helped us wire those up. But yeah, it was a fit. There was an Apple server in there as well. That was the media server, + +00:23:46 I think. It was doing like imagery sizing. Yeah, it was like a 2U box or something. I forget. It was actually like a physical server that now, because I'd never seen one before in person. + +00:23:55 XServe. It was an XServe. Yeah. Also, I remember that room had a wire that was the AP News wire, Like I didn't realize there was a physical wire that the news came in on from the Associated Press. + +00:24:07 And when you hear it came in over the wire, that was the wire? + +00:24:10 That was the wire. + +00:24:11 We had the actual wire in our basement. + +00:24:13 Yeah. + +00:24:14 Incredible. + +00:24:15 Well, Simon and I were lucky because we didn't really have any sysadmin experience, but there were some really experienced guys, Frank and Nick, who worked at the company and actually did some training with us. + +00:24:29 I remember going into their dark room. + +00:24:32 All the lights were like turned off. + +00:24:34 You could only see like these Unix shells. + +00:24:36 I was like, whoa, I'm really learning the good stuff. + +00:24:38 And this is the same Frank. + +00:24:40 This is Frank Wiles, who he's been president of the Django Software Foundation in the past. + +00:24:43 And he's still very heavily involved with Django to this day. + +00:24:46 Yeah. + +00:24:47 All right. + +00:24:47 Let's talk about 2008. + +00:24:48 A lot of stuff happened around 2008. + +00:24:50 That was the timeframe a little bit before then Django's open source. + +00:24:55 But maybe you'll have to tell me, maybe a little bit inspired by Ruby on Rails. + +00:25:00 There's a lot of, in Ruby on Rails, a lot of convention over configuration, a lot of magic. + +00:25:05 And Django used to have a little more implicit magic. + +00:25:09 And there was the, when Django went to, around when it went to 1.0, there was the magic removal. + +00:25:14 And it was the first DjangoCon, all those things. + +00:25:17 Tell us about that, whoever wants to take it. + +00:25:18 I can talk to DjangoCon a little bit, the very first DjangoCon. + +00:25:22 I was working for a radio station in London that was doing a big Django project. + +00:25:30 One of my co-workers there was this chap, Rob Lofthouse, who very foolishly stepped up and said, "Well, okay. + +00:25:36 Everyone says there should be a conference and no one's arranging a conference. + +00:25:38 I'll arrange a conference. + +00:25:39 How hard can it be?" The answer is everyone in the school understands it's very, very, very hard. + +00:25:44 He managed to pull it off. + +00:25:46 The big thing there was that Google offered to host it. + +00:25:49 We got the venue and the catering and everything was in Google's campus in Mountain View. + +00:25:53 Was 2008, was that the year of the first DjangoCon? + +00:25:56 I think it was, right? + +00:25:57 I think it was. + +00:25:58 And yeah, we pulled together a conference. + +00:26:03 The videos from that conference were all available on YouTube, actually. + +00:26:06 I rewatched one of the panels quite recently, just out of interest. + +00:26:11 It was really fascinating seeing there are people on those panels, like Russell Keith mcgee was on the database migrations panel and he was just like one of the people who was trying to solve database migrations back then and of course he's been an enormous fixture in the python and django community ever since um but yeah that was that was really fun that was uh like how many of + +00:26:30 us were that jeff were you at the first django conference i was i was barely working at the journal world like maybe six months or a year or something so yeah i just just kind of experienced at all as a nobody. So it was great. And I think my biggest contribution to that conference was, + +00:26:47 so I was working on a migration system called demigrations and Andrew Godwin was working on, I think he was calling it South back then. the, like his migration and we had Russell Keith and I got, I managed to, I made the pitch for Andrew, who was just a student at university at time to be flown to California to present at this conference. I think it may have been his first ever conference talk. And he has been to and spoken at every Django and Python conference since. + +00:27:13 So I like to think I helped just provide that little spark to get his speaking career started. + +00:27:20 But yeah, it was a very, very notable moment, that first DjangoCon. + +00:27:24 I had a mind-blowing moment there. A guy from, I think it was Japan, came up to me. He didn't speak a word of English and he gave me his book that he had written about Django whoa and and he just like bowed and and went away and that's I still think about that and it still kind of brings a tear to me it's like what it's it's mind-blowing that people all around the world that you've never met who speak languages you don't speak use your code like it I still it's been 20 years I still can't even like 100% come to terms with it it's really cool yeah I thought + +00:28:00 you're gonna go a different direction and you're gonna say when Cal Henderson put that sign up of the unicorn like that was going to forever be the moment but i don't know if you want to expand well + +00:28:10 that's the downside of that first jango con there was a q a with me and jacob and i think it was me i said are there any pony requests and that was intended to mean are there any feature ideas sort of open suggestions and evidently that was the um the impetus for the idea of the jango pony which I despise. + +00:28:34 I think it's so dumb and I think that's directly related to me retiring from the project, frankly. + +00:28:38 Oh no! + +00:28:39 I know it's funny in a way but it's also infuriating to me. + +00:28:43 I think it's so stupid and unprofessional and the type of humor is just dumb. + +00:28:48 No wit at all. + +00:28:49 I'll stop my rant. + +00:28:50 That'll be on my own. + +00:28:51 Whenever I start my own podcast I'm going to have the first episode on why the Django pony sucks. + +00:28:56 Wait, what should it be though? + +00:28:57 What would you like it to be? + +00:28:58 Do you have thoughts? + +00:28:59 I think you underestimated. + +00:29:02 I would have guessed it would have went like a it should at least be playing a guitar people love + +00:29:07 celebrating this for that event and occasion qualified people Timo I have one of your stickers on my fridge + +00:29:15 that's one of the things I did for the 20th birthday I designed a new pony + +00:29:20 to distribute to people around the world you might have seen one at the Amsterdam event people really like the mascots + +00:29:27 a bag of the stickers was passed to me and I passed it along. + +00:29:31 To the trash can. No, just kidding. + +00:29:35 This portion of Talk Python To Me is brought to you by Sentry's AI agent monitoring. + +00:29:39 Are you building AI capabilities into your Python applications? + +00:29:43 Whether you're using open AI, local LLMs, or something else, visibility into your AI agent's behavior, performance, and cost is critical. + +00:29:52 You will definitely want to give Sentry's brand new AI agent monitoring a look. + +00:29:58 AI agent monitoring gives you transparent observability into every step of your AI features so you can debug, optimize, and control the cost with confidence. + +00:30:08 You'll get full observability into every step of your AI agents. + +00:30:12 That is model calls, prompts, external tool usage, and custom logic steps. + +00:30:17 AI agent monitoring captures every step of an AI agent's workflow from the user's input to the final response. + +00:30:24 And your app will have a dedicated AI agent's dashboard showing traces and timelines for each agent run. + +00:30:32 You'll get alerts on model errors, latency spikes, token usage surges, and API failures protecting both performance and cost. + +00:30:41 It's plug-and-play Python SDK integration. + +00:30:43 Open AI for now for Django, Flask, and FastAPI apps with more AI platforms coming soon. + +00:30:50 In summary, AI agent monitoring turns the often black box behavior of AI in your app into transparent, debuggable processes. + +00:30:59 If you're adding AI capabilities to your Python app, give Sentry's AI agent monitoring the look. + +00:31:05 Just visit talkpython.fm/sentryagents to get started and be sure to use our code TALKPYTHON, one word, all caps. + +00:31:14 The link is in your podcast player's show notes. + +00:31:16 Thank you to Sentry for supporting Talk Python and me. + +00:31:20 another notable thing that happened in 2008 other than the housing crisis crash that wrecked a bunch of things was the foundation right the jingo software foundation and i think more broadly i think one of the things that makes you know i talked to a lot of different people about different web frameworks and the thing that really makes jingo stick and stand out i think is one the admin thing but the other is the community around jingo and we're talking about the jingo conference the foundation it's just unlike other things there's there's a whole community that you get into when you go into django not just some apis you use you know what i mean so uh who wants to take the + +00:32:00 tell us about how the foundation got started so that was i was there uh so the there was the question of who owns the code um it was i think technically owned by the lawrence journal world correct me if I'm wrong. And so that wasn't going to last forever. So we needed to find for legal, you know, ownership reasons, an entity to own it. We didn't want to go the route of Ruby on Rails, copyright David Hennemeyer Hanson. I think that's just really cringe. So it's good to have an organization as part of it. So yeah, we set up 501c3 if that's the right one that we use. I forget. + +00:32:36 And yeah, that's basically the origin story. From my perspective, I'm sure there's a lot more + +00:32:41 subtlety to it oh gosh do you want to weigh in yeah i was only barely started that time so + +00:32:47 like i definitely relate to you know the community around django you were all telling why you got involved and when and how and for me it was it was whacked as a day job but then what got me to stick around is actually django girls and organizing some of those local workshops and you know having organizations like this like alongside the django project not even the main thing is definitely only possible because we've had such a like long running open source foundation. There's lots of aspects of the foundation we could discuss, but definitely, you know, that kind of longevity + +00:33:16 for community run open source is rare and to be cherished. Yeah. Maybe you could tell us a little bit like what was it when it started and what is it now? Like how has it changed over the time, over the years? I'm going to put in a plug for the Django Fellows program. I feel like the Django + +00:33:33 Fellows program has been an absolutely astonishingly effective part of the mission of the foundation. + +00:33:38 For people who aren't familiar with that, the Django Software Foundation employs fellows who are paid contributors who work on the bits of open source maintenance that aren't necessarily so glamorous. They do bug issue, they do code review and handle security incidents and bug triage and all of these different things. And thanks to them, Django has a release schedule that operates like clockwork. Like Django, there's a new release that comes out when they said it was going to come out. + +00:34:06 Code gets reviewed, bugs get trashed. It's a wonderful, wonderful thing. It's unique because most open source projects that are run like that have a company behind them. There are people who have a full-time job and there's a company that's commercially supporting that. Django does not have a commercial sponsor in that respect. It has this nonprofit that people fund and that employs these these fellows and the fellows keep everything ticking long. I keep on saying to other open source projects, if you can figure out a pattern to use that's similar to the Django fellows, that's absolutely worth doing. The Python Software Foundation has a developer in residence program now, or several developers in residence, which were inspired by the Django fellows program. + +00:34:45 I think it's been really influential. And it's notable because there are very few major open source projects that don't have a company behind them. The other one that comes up a lot is Postgres, Postgres is very much a community-organized program. + +00:34:59 If you look at the way Postgres works, a lot of the key maintainers have their own consultancies. + +00:35:04 So they've got a good way of financially sustaining the work that they're doing. + +00:35:09 I think the Django Fellows Program, which was introduced... + +00:35:13 When was that? + +00:35:13 That was at least 10 years ago the Fellows Program started. + +00:35:16 I think it's been an enormous success. + +00:35:18 Yeah, absolutely. + +00:35:19 Another thing that I think is really interesting, it's a little bit newer, but is the Django Knots program, right? + +00:35:27 And Django Girls as well, shout out to them. + +00:35:30 You know, so many people I've talked to are like, I got into Python because I wanted to do Django, which for you all is probably a little bit mind-blowing as well. + +00:35:37 Like, they didn't use Django because they were Python, but the other way around. + +00:35:40 The first time I met somebody who said, oh yeah, I got into Python because of Django. + +00:35:43 And I was absolutely bowled away. + +00:35:45 I always assumed you'd have to be a proficient Python programmer before you could start picking up something like Django. + +00:35:50 It turns out it was the other way around. + +00:35:51 People would get in from, Because partly because the Django documentation was so good. + +00:35:56 But yeah, that's so many people got their start in the Python community as web developers who dabbled with Django and then got into Python from there. + +00:36:02 Yeah. + +00:36:03 Kenneth, love it on the audience because I learned Python through Django. + +00:36:06 Very cool. + +00:36:06 When I chaired DjangoCon US, I'd always ask the audience. + +00:36:09 For one, like I was always a nervous speaker. + +00:36:11 And so anytime you ask the audience to do something, it like takes my nerves away. + +00:36:14 But I'd always ask who learned Python by learning Django. + +00:36:18 And over half the room every year would always raise their hands up. + +00:36:20 So it was a heck of a gateway to learn Python. + +00:36:23 on. Also, we just hired our third fellow Jacob Walls last week, who I think starts this next Monday. So Django's up to three fellows now. And it's just been, the load is just incredible. So trying to keep up. I think our existing fellows have over a thousand tickets that they're trying to sort through. And if people are watching and want to help fund, contact Tebow and I, + +00:36:44 and we will definitely help you in the USF to support more fellows. + +00:36:47 We'll give you our annual reports, which details everything the foundation does, + +00:36:51 And more on the other side. + +00:36:53 Well, I just want to make one point on that because, I mean, T-Bone and Jeff, you know now, but the finances for a long time, like when I was treasurer in 2020 for three years, the total budget was under $200,000. + +00:37:04 And that was almost all fellows. + +00:37:07 So like the total budget for everything, including grants, legal stuff, hosting. + +00:37:11 I think now maybe it's $250,000, $300,000, but it's, you know, to your point, Simon, it's not funded by a corporation. + +00:37:17 It's tiny. + +00:37:19 And people don't realize that aside from the fellows, everything else is volunteers, right? + +00:37:22 Even the conferences. + +00:37:23 It's not, you know, PyCon, US, EuroPython have a lot of volunteers, but they also have paid employees. + +00:37:28 Like Django Cons are all volunteers. + +00:37:31 Yeah, they're all volunteer based. + +00:37:32 The PSF runs a staff of about, I used to be on the PSF board. + +00:37:35 Simon's on the board. + +00:37:36 I'm on the board. + +00:37:37 Yeah. + +00:37:38 Last time I saw, they probably have about, what, 2 million in salaries a year because of so many people. + +00:37:43 You've got like five or six developer in residence, at least probably what, like 12 to 13 people, maybe 14 people on staff. + +00:37:50 That's about right. + +00:37:50 Yeah. + +00:37:52 It's interesting how much the PSF is an accounting organization because it turns out accounting, especially if you're a nonprofit with like with a whole bunch of organizations who use the PSF as their fiscal sponsor to help them with their nonprofit accounting. + +00:38:06 There's a lot of there's a lot of accounting and logistical works that goes into an organization like the PSF. + +00:38:11 And PyCon US is run by the PSF as well, which is a 2,500 person conference these days. + +00:38:17 That gets very complicated. + +00:38:19 Yeah, so the DSF runs on shoestring size budget compared to the PSF. + +00:38:23 But Django is a pretty good amount of the community, though. + +00:38:25 So that's kind of one of the things we're doing is by getting that third fellow. + +00:38:29 And part of it, too, is we've ran, not to nerd out on expenses for a while, but the DSF for a long time has ran with over $200,000 in the bank. + +00:38:36 And to me, that's just not doing much with our money. + +00:38:38 So I know that we're also doing investments and stuff as well. + +00:38:42 But I think getting that third fellow was something that was just long overdue for the number of people that we have. + +00:38:46 If one fellow goes on vacation for a while and leave, it's just too much for one person to keep the foundation, keep security releases, releases going. + +00:38:55 So it was kind of long overdue, but I'm happy we're able to pull it off this year. + +00:38:58 Yeah, that's great. + +00:38:59 My favorite talk at JaggerCon last year was Jacob Kaplan Mosk gave a keynote about what the DSF would do with a million dollars. + +00:39:06 Like if the DSF could go up to a million dollars a year, what would they spend that money on? + +00:39:10 As also a pitch for why you should be sponsoring the DSF. + +00:39:13 I recommend people watch that if you want to get a feel for how that organization was shaped at least sort of 10 months ago. + +00:39:19 I think there's a huge impact that just having one or two people whose full-time job is to go into the corner, the dark corners of some open source project that's just not that interesting. + +00:39:30 Nobody really wants to go mess with, but just start hacking away on those and, you know, really putting the polish on stuff that is not as exciting as other pieces. + +00:39:39 And so there's not, you know, you said it's not that big relative to the PSF, but put the PSF aside, any other open source Python project, it's a really huge pool that you're working with relative to how many other projects even have fellows. + +00:39:56 Not many, if any. + +00:39:57 Yeah. + +00:39:57 Super, super cool. + +00:39:58 So let's talk about some of the more recent things. + +00:40:02 I know that at least personally, when the Instagram keynote came out a couple of years ago at PyCon, or maybe more than a couple, it was 2017, I think. + +00:40:11 But around that timeframe, just realizing how big some of the deployments of Django were, how much traffic people were serving through them. + +00:40:20 When it first came out, you probably didn't think on that scale. + +00:40:23 No, Instagram, Pinterest, NASA. + +00:40:26 Yeah, no, that was... The goal was to build newspaper websites, right? We wanted to be able to build... I loved Django's original tagline was "Web development on journalism deadlines". That's always been my favorite description of the project. + +00:40:41 Nice. Who else wants to chime in on this idea? Like this is just... It blew my mind. I'm like, wow. + +00:40:45 One thing I will say is a thread, Instagram's Twitter clone came out what last year or the year before. That's a Django app. I met one of the engineers of mine. It's just a classic Django app that Facebook launched just a few years ago. + +00:40:59 I believe technically they have their own fork. They didn't just download Django 4.0. + +00:41:04 I feel like at that scale, if you haven't got your own fork, I'd be very surprised. + +00:41:08 But yeah. For me, it runs on center. Sorry. I think I'm a brief I mean, one interesting thing. + +00:41:14 500 milliseconds behind all of you. + +00:41:16 For me, as far as scale, there's also the number of projects out there. + +00:41:20 I've been doing the counting to figure out our impacts. + +00:41:23 And I think we're at around the 500,000 live Django sites and apps out there from the numbers I could find. + +00:41:30 And also specific types of outreach. + +00:41:34 So governments, for example, seeing how many central governments of all the countries around the world basically use it. + +00:41:41 And when I say use it, it's not just like the one project. + +00:41:43 They have teams whose role is just to manage 10, 20, 50. + +00:41:47 I think right now, Brazil, of all countries, I don't have a connection there, had like 84 sites I could detect in a single environment using Django. + +00:41:54 Just imagine that times all the countries. + +00:41:57 Wow. + +00:41:58 You know, one of the things that's always fun for me to do is to go to the GitHub repo, right? + +00:42:02 And I've got Django with 85,000 stars and 32,000 forks. + +00:42:07 That's a really interesting number. + +00:42:08 That's way more forks to stars than an average project, by the way. + +00:42:13 But if you go down the side here, you can see they now track how many contributors and used by 1.997 million projects. + +00:42:23 That is a lot. + +00:42:25 Now, that's just on GitHub, right? + +00:42:26 That are public repositories that we can know about and stuff. + +00:42:29 And used by could be a student project, not a government entity. + +00:42:33 But still, that's pretty wild. + +00:42:35 Very impressive. + +00:42:35 Let's talk about some of the more recent transitions. + +00:42:39 I mean, I feel like the speed of major version releases is picking up. + +00:42:44 You know, we had the first three years, 2005 to 2008 for one. + +00:42:47 Then it was like one for a while, and then two, and then three, then four, then five. + +00:42:51 You guys are maybe working on six. + +00:42:52 I don't know. + +00:42:53 And also adopting some of the lot of changes coming in Python, right? + +00:42:57 Like async and await and that kind of stuff. + +00:43:00 How has that been? + +00:43:01 How did it start? + +00:43:01 In some ways, it's really tricky because people really love Django as it is. + +00:43:06 And from that standpoint, you have to make sure that such a big group of people agree on what exactly the future of Python means. + +00:43:15 So, you know, definitely mentioned AsyncAway. It's one of the big ones. + +00:43:18 API, is it in Django Core? Is it not? + +00:43:21 Mobile web, how much we get involved with that? + +00:43:23 These days, it's like the HTMLX that people really want to see Django support well. + +00:43:27 And I guess also AI to some extent. I don't know if we want to get there. + +00:43:31 But I guess for us, I think it's also, if it's not in core, we definitely have so many packages out there that we just try and curate a bit more these days, make sure they work well together. + +00:43:42 So that also helps a lot. + +00:43:43 Yeah. + +00:43:43 I think the package curation is really important. + +00:43:46 I would actually like to see Python in the standard library smaller and then recommend more stuff to keep the flexibility open so they could move faster, focus more. + +00:43:55 So what I find tricky about this one, Django has always been a batteries-included framework, And that's because in 2004, 2005, PyPI and PIPA didn't really exist. + +00:44:05 So if you weren't batteries included, then people had to, it was very high friction for people to glue together the pieces that they needed. + +00:44:12 And of course, today, I feel like Python packaging is incredibly good shape. + +00:44:16 Like you could ship a web framework today that was entirely dependent on other things. + +00:44:21 So then the value from the Django thing comes down more to everything being, that there's a Django promise that the stuff in Django core will continue to work. You're not going to have a situation where Python 3.15 comes out and Django templates stop working, but the ORM still works because everything is maintained in the one place by that single team. And I love that. I love that you just know that all of the pieces of Django are going to continue to work together. And the moment you add external packages, there's that little bit of risk. What if that external package that you're using becomes unmaintained in the future? So I'd quite like to see just a little of a sort of adoption of crucial packages under the Django software foundation, but not as part of Django itself. I want separate packages that maintain that promise to be maintained. Because we have fellows, which means that we can actually do that kind of work. But it's also a very big commitment to take on. It's an interesting pattern that I'd like to see explored a little bit more, + +00:45:21 I think. Yeah, it's easy to forget that it wasn't always there. Sorry, Will, go ahead. + +00:45:26 I was just going to ask Adrian and Simon. I mean, even today, batteries included web frameworks are not that common. It's still the trend, more micro frameworks. I'm curious what your take is on that. I mean, like FastAPI, right? That's a lightweight framework. It seems, I'm always shocked there aren't more batteries included frameworks in PICA language. Maybe it's too difficult to do, but micro frameworks seem more common. There's sort of only one batteries included option for programming language. + +00:45:52 I feel like the sales pitch, like you get to pick the best pieces for the components for your project, always sounds great. + +00:46:00 Obviously, you don't want one thing that dictates what you're going to use for all of the different things. + +00:46:03 You'd much rather pick and choose the things that are most valuable. + +00:46:06 But the trade-off that people don't talk about much is the fact that now every single one of those selections is getting you further away from that sort of community standards. + +00:46:14 There's more risk of future maintenance problems. + +00:46:17 There's more work that you have to do to wire things together. + +00:46:20 But I feel like the biggest innovation in the open source world in the past 15 years has been packaging. + +00:46:25 Like every programming language has a PyPI style packaging solution of some sort. + +00:46:31 You've got cargo and you've got the Ruby gems and all of those kinds of things. + +00:46:37 And that pattern mostly works really, really well. + +00:46:40 But I just keep on coming back to that feeling that with Django, I know it's all going to work. + +00:46:45 And I know it's going to work far into the future. + +00:46:47 And that feels really valuable to me. + +00:46:49 Good, Adrian. + +00:46:50 For my answer, I think I'd have to evaluate it on a case-by-case basis, but the way I would evaluate it is to look at the origin of the project. + +00:47:00 Was it extracted from something from the real world or was it built either as an academic or for fun exercise? + +00:47:07 What was the reason for its existence? + +00:47:10 And generally, I trust stuff that was extracted from real production sites more. + +00:47:15 That's just my own preference. + +00:47:17 It's also sort of to flip it around. I never really cared whether someone used Django or not. I would evaluate you on your product, not based on like the web framework you use. Who cares really what web framework you use? And I think that kind of ethos may be seeped into Django at some point unintentionally. I mean, I never intentionally sort of made that, had that philosophy communicated, but yeah. + +00:47:44 I always felt like one of the most interesting difference between Rails and Django is that Rails was extracted from Basecamp. + +00:47:49 They built Basecamp and they pulled the framework out of it. + +00:47:52 I feel like Django was kind of reverse extracted from Lawrence.com because we had Lawrence.com. + +00:47:57 It was PHP and MySQL. + +00:47:58 And the goal with Django was always to get it to a point where it was good enough that it could replace the PHP in Lawrence.com. + +00:48:04 So everything that we were building was always like, what would Lawrence.com need in this new thing for us to replace it? + +00:48:10 And the end result was a framework that you could build Lawrence.com in. + +00:48:13 Lawrence.com was the entertainment website for the newspaper. + +00:48:15 It's the best local entertainment website for any city that has ever existed. + +00:48:20 I will very confidently state that. + +00:48:22 I've never lived in a city with a website as good for figuring out what the things to do in a town are. + +00:48:29 And that was Adrian's biggest project, Lawrence Journal World, was always Lawrence.com and evolving Django up to the point where it could power that site. + +00:48:37 It's what got me here. + +00:48:38 Like I saw that site living 200 miles away and went to San Francisco and kind of looked at Y Combinator and was like, you know, this is only 200 miles. + +00:48:45 Like they've got something good going here and that site especially. + +00:48:48 I think to a Django's batteries, Django, I think 20 years ago, Django, there was a lot of things that you had to do the Django way. + +00:48:56 And I think there's been so much concerted effort, especially the first 12 to 14 years. + +00:49:01 It's so much as pluggable that like if you don't like the batteries, I don't know what you're missing. + +00:49:05 Like Django is still a really good core. + +00:49:07 Don't use the admin if you don't want it. + +00:49:09 I think most of the problems I see, the foundation and the community, I think there's like a lot of lore that comes with Django. + +00:49:16 And it's like lore of like, maybe it's harder to change certain things on the website than it really needs to be or is. + +00:49:22 And I think this last year, we've definitely pushed back on, like, if you do a search on the Django docs now and you look for REST, you wouldn't know that Django is a REST framework or has the ability to do that by looking at the docs. + +00:49:32 And so these were just technical challenges of like certain steering council members have written articles and stuff, blog posts saying like, you can do rest with Django. + +00:49:40 There's some good third party packages to do it. + +00:49:42 In the past, we've kind of been like apprehensive to want to recommend third party packages through docs. + +00:49:47 I think that's just one of those. + +00:49:49 It sounds weird when I say it out loud. + +00:49:50 At least it does to me. + +00:49:51 And I think once we have these discussions, then we start like realizing that, well, we have a website that everybody comes to. + +00:49:57 There's very specifically doc sections and there's blog, but there's no search that pulls stuff in together. + +00:50:03 And so the steering council and different members have been trying to make search work across everything. + +00:50:07 And then that way you can get hits from rest and you can get hits for third party packages that the steering council recommends just because they're good packages everybody should use. + +00:50:16 And so I think some of the lore and some of the, hey, these things have been done this certain way doesn't mean we want to keep doing them. + +00:50:21 It's just probably somebody had an afternoon or a night and they made search work a particular way. + +00:50:26 And that was really good for the docs. + +00:50:27 but like sometime look up REST and look at PHP. + +00:50:30 There's actually more hits for PHP on the Django website than there are REST + +00:50:33 unless you get into like Sphinx, you know, restructured text. + +00:50:37 So, but those are challenges that until we talk about and we realize, and I think like that's the advent of a FastAPI kind of taking off is I think like Django didn't have a great REST story by looking at the docs and looking at information. + +00:50:50 At PyCon US this year, I had dozens of people ask me, how do I use the Django ORM from FastAPI? + +00:50:54 And it's like, why? + +00:50:55 You don't have to. + +00:50:56 Django is async enough that you don't need to do that. + +00:50:59 That's hard. + +00:51:00 And that kind of sucks to do. + +00:51:01 So why? + +00:51:03 And those are things we're focusing and talking about internally too. + +00:51:06 Yeah. + +00:51:07 Talking about search and search across different things, somewhat directly to you, Jeff, but all of you, and maybe Simon as well, is like, what about just like chat.djangoproject.com and do some rag on it and let people have a conversation with Django? + +00:51:23 You know what? + +00:51:23 We don't even need it because Django is stable enough that every chatbot out there knows Django inside out already. One of the great benefits of having great documentation and lots of examples for 20 years that is stable is, yeah, you can, GPT-5 will spit out a full Django project and just + +00:51:40 get it all right. So yeah, I completely disagree. I think we need a big sponsor who can come in, whether it's Anthropic or OpenAI or somebody else. We need a big sponsor, a title sponsor, even in fact that can help give the money for this and donate the chat. + +00:51:55 Because even though you're right that it isn't ingrained, you still have that delay between because Django kicks out a new version or two a year. + +00:52:01 And so there's always that lag of nine months or a year. + +00:52:04 And most of it recommends the most popular version, not the latest version or something like that. + +00:52:08 Right. + +00:52:09 Exactly. + +00:52:09 Yeah. + +00:52:10 Because I see a lot of demos where they're actually building Django apps using these, these tools. + +00:52:14 And so we're selling AI by virtue of Django and Django's docs being so good. + +00:52:18 Or occasionally one of these things will say that Sound Slice has a feature that it doesn't have, and then Adrian has to build the feature. + +00:52:25 Oh, God. + +00:52:26 Oh, you guys, in the chat, it's coming on here. + +00:52:29 We have Frank Wasson, but our chatbot can have a cool personality. + +00:52:32 And Kenneth loves to insert a random name every once in a while or something, you know, pony style. + +00:52:40 Adrian, what is the story behind that? + +00:52:43 Oh, it has nothing to do with Django. + +00:52:46 So yeah, just the 30 second version is ChatGPT was telling people about my site, my company sounds like having a feature that doesn't actually exist. + +00:52:56 And it was happening like dozens of times per day. + +00:52:58 And so we had this inbound set of users who had a wrong expectation. + +00:53:03 So we ended up just writing the feature to appease the ChatGPT gods, which I think is the first time that at least to my knowledge of uh yeah product decisions being influenced by + +00:53:17 misinformation from llms i just love that it's easier to just build the feature than to get the + +00:53:22 misinformation corrected like there is no he's got to backfill it he's got to backfill it + +00:53:29 incredible all right um that's crazy i let's let's do two more quick topics like somewhat lightning round, we have about six minutes left for this show. So one, there's been, you guys mentioned it a couple of times, and it's also just kind of going back and forth in the audience here, that the whole REST idea. So we've got, you know, Tom Christie doing amazing work with Django REST framework. We've also, I think, you know, people who do talk about FastAPI and how do I use the ORM with it, like Django Ninja, which is really, really close to FastAPI and so on. That's one side, that's a really positive side of things with Django. And the other side I think is, you know, I'm not a super fan of, but it's, it's something a lot of people do is, well, can we just make the backend a dumb crud API thing? And let me just write everything in JavaScript on the front end, which I'm not as much of a fan of. I think a lot of people choose that because they've been told that's how it's supposed to be. I don't think it needs to be that way. + +00:54:25 So what do you all feel about that? And sort of the, how this has an impact for Django, Because if people just see the backend as an API that's just a dumb thing, then a lot of the value that Django brings, I think, starts to vanish. + +00:54:38 You've got things like HTML and so on. + +00:54:40 So, yeah, what are your thoughts on all this? + +00:54:41 I think one of the hardest problems in computer science right now is talking a front-end team out of building everything as a pure JavaScript app, especially if that's developers who onboarded in web development over the past five to ten years. + +00:54:54 It's infuriating because my impression is that it just takes longer and you get more bugs and it's harder to build useful software. + +00:55:02 The one thing I will say is that HTMX has been a huge breath of fresh air on this front. + +00:55:07 HTMX gives you your single page applications. + +00:55:11 It's all stuff that's rendered server-side in templates. + +00:55:14 It fits really well with Django. + +00:55:15 There's a great Django extension for looking into HTMX. + +00:55:19 And I feel like that, the pendulum has just started swinging slightly back away from the everything must be rendered in JavaScript thing, partly because of the work that the HTMX people have been doing. + +00:55:28 So that I felt pretty good about. + +00:55:30 I'm going to point to Adrian now because SoundSlice is a pure JavaScript front-end app, and it should be based on what it does, right? + +00:55:39 Yeah, it's interactive sheet music. + +00:55:41 So it's sheet music that's rendered entirely in your browser, responsive, blah, blah, blah, with an editor, a lot of crazy JavaScript. + +00:55:48 But yeah, I mean, this is a whole other podcast episode about the horribleness of modern web development. + +00:55:55 I feel very lucky that I came up 20 years ago when before it got so insane. + +00:56:02 But yeah, I have sort of a related comment, which is I would like to see Django be a stronger advocate for the web in general. + +00:56:13 It doesn't matter if you're talking to developers or people using tech products. + +00:56:18 the expectation has been shifting away from the web to the phone for many, many reasons. + +00:56:23 And I don't like that. + +00:56:25 I'm pretty sure that most, if not all of the people on this chat don't like that either. + +00:56:29 And I don't really see a lot of effort being pushed in that direction. + +00:56:34 So just to sprinkle a little idea here. + +00:56:39 I personally wish that progressive web apps were more front and center, giving you an option at least like, okay, you have to be on the phone, + +00:56:45 but could we package it up a little more? + +00:56:47 There's a really exciting development there that's finally beginning to be rolled out. + +00:56:51 There's this thing called view transitions in CSS, where you can have a little bit of CSS that says, "And when the user goes from one page to another, keep the navigation bar in the same place and do a quick blur effect for this piece here." It means that... + +00:57:05 And it's been supported in Chrome for a few years. + +00:57:07 It's finally rolling out through the browsers. + +00:57:09 It means you can build something that feels like a slick app with page transitions and so forth, but it's not. + +00:57:14 It's just boring old HTML and CSS. + +00:57:16 No JavaScript required at all. + +00:57:18 Gives you that experience much faster to load. + +00:57:20 And I have been itching to see this start showing up. + +00:57:24 And I think it's finally this year, I feel, is the year where the browser support is good enough that we can start really depending on this. + +00:57:30 And yeah, because it gives you that app-like experience on a boring old website, which is, I feel like, that's the thing that we all want. + +00:57:37 So back to part of Adrian's piece, and sorry, Tebow, real quick. + +00:57:41 My second thing I have to pitch is because of what Adrian was saying. + +00:57:44 I think because systematically we, you know, we are really lucky to be around for 20 years because we've had nothing but volunteer leadership in years. + +00:57:51 One thing we want to do is find an executive director, somebody that can run the foundation. + +00:57:55 The biggest reason goes back to Adrian's point. + +00:57:57 I think that we sometimes don't get a seat at the table when it comes to web standards, when it comes to these conversations, is because it's the cycling of Thibaut being the president now. + +00:58:06 It was Frank before. + +00:58:07 It was Anna from Africa. + +00:58:09 We've had many really good presidents over the years. + +00:58:11 But it's really hard to work with foundations and nonprofits when you don't have a set person. + +00:58:16 Even our fellows seem to rotate every three to four years. + +00:58:19 And so that's part of like Django growing up, which is something I would love to see us do. + +00:58:23 And then Thibault, I love the fact that this started off in the middle of like the day. + +00:58:26 And now it is like perfectly cycled to like dark. + +00:58:28 Yeah, I guess it was high and curve you as president. + +00:58:30 Go ahead. + +00:58:31 To me, there's a developer experience and like keeping devs happy. + +00:58:36 So, you know, when we say our people go to React or whichever like JS heavy, It's also because they think they need it to build whatever complexity that they are after, but also because they think it has good DX, I assume. + +00:58:47 So when we say like the REST framework or the Django Ninja, to me, it's this as well, that we have to be aware of what's the evolving standards for what devs want to do with their web. + +00:58:57 And I guess how we can make Django meet that, whether it's better types, better CLI, better packaging, more Rust, maybe, whichever. + +00:59:06 Will, do you want to get a quick word in on this everything's a front-end thing now before we move on to the last bit? + +00:59:12 Oh, sure. I'm just going to say, and this probably goes back to Adrian and Simon, like all of them, + +00:59:16 it seems very wise to keep the front-end light because there was certainly 10 years ago a lot of pressure to integrate React or Vue or Angular or something. + +00:59:25 But the fact that it's remained just the Django templates means HTMX can rise and we can have more innovation. + +00:59:32 And just shout out, Carson Gross of HTMX is giving the keynote at DjangoCon US in three weeks. + +00:59:37 So even though he's not a Django developer, he's in fact more of a, he's a professor who does Java and obviously JavaScript. + +00:59:43 It still slides into Django in a way that if Django had built out the front end story, we wouldn't be able to. + +00:59:49 I love how that also fits. + +00:59:50 DjangoCon has had a long tradition of having a keynote from somebody from outside the community, which started in 2008 with Cal Henderson at DjangoCon over in Mountain View. + +01:00:00 Was that the why I hate Django? + +01:00:02 Because there was that whole history of, I feel like there should be another one of those, you know? + +01:00:06 Absolutely, yeah. + +01:00:07 But that's great. + +01:00:08 That's a fantastic keynote for DjangoCon, I think. + +01:00:10 Yeah, and coming up in Chicago, right? + +01:00:13 Pretty soon. + +01:00:13 So if you are in or would like to be, go check it out, right? + +01:00:16 That's part of that community thing I talked about. + +01:00:19 And people listening who are not watching, listening, not watching, they don't realize that you did this outside. + +01:00:26 And there's a great comment from Kojo saying, It's really cool that you're using in real life dark mode. + +01:00:32 It's great. + +01:00:34 All right, let's close it out with this, everyone. + +01:00:36 We're pretty much out of time. + +01:00:37 20 years, you know, amazing, amazing stuff. + +01:00:42 We're celebrating Django having its birthday, right? + +01:00:45 But it's going to live on for a long time. + +01:00:46 There's a ton of code. + +01:00:48 There's a ton of things people are doing with Django. + +01:00:51 Even if you guys step away at some point, it's still going to be going. + +01:00:54 So give me your, you know, going around and starting with Will, I guess. + +01:00:57 Give me your forward-looking thoughts on Django. + +01:00:59 over the next five, 10 years? + +01:01:02 Well, I think it's integrating with AI, broadly defined, however that means, because in a way related to Adrian's point, like the boring old web has never been more relevant, even though we're focused on chatbots right now, you still need all the other things. + +01:01:15 And Django can integrate into that. + +01:01:16 And I think it's always kind of a marketing problem for Django because we don't have one person in charge or a company behind it. + +01:01:22 So telling the story of we've been here, there's still double digit PRs. + +01:01:26 It has async in many areas. + +01:01:28 And I worry that the next generation of web developers just skip it because they don't know what it already has. + +01:01:34 And they have this presumption that it's too hard to learn, which it's not. + +01:01:38 Yeah, absolutely. + +01:01:39 Is it me? + +01:01:40 It is you. + +01:01:41 I don't have any. + +01:01:41 I hear you're boring about the web. + +01:01:43 What's going on now? + +01:01:44 Tell us about what do you see for the future? + +01:01:46 For the future. + +01:01:47 Yeah, I'm actually, perhaps ironically, like the worst person to ask because I have worked on the same app for almost 13 straight years full time now. + +01:01:57 I use Django every day and it does everything I need. + +01:02:01 So I'm very happy with the current state. + +01:02:05 And I think, like I said earlier, if it were up to me, the thing I would focus on is more philosophical or trying to promote the concept of the web versus non-web stuff. + +01:02:21 Because if the web goes away, Django goes away. + +01:02:23 I mean, it's inherent. + +01:02:25 A lot of beautiful things go away, yeah. + +01:02:27 A lot of beautiful things go away. + +01:02:28 So I think that would be my hope for the next five years. + +01:02:32 Yeah, I've done combat with the App Store review people, both Apple and Google, and it is not pretty. + +01:02:38 We do not want the web. + +01:02:40 There's definitely AI in there and definitely the web. + +01:02:43 I really like that you mentioned the view transitions. + +01:02:46 I think if web technologies get that much better in the next five years or so, we can match what's possible with a mobile dev, but not only match it, but in terms of accessibility and how many people you can reach with a web app. + +01:03:00 We can go far, far beyond. + +01:03:03 So I guess to me for that, the other side is having a foundation behind Django that's probably a bit more professional, that takes it a bit more seriously, that we're the people looking after the framework for millions of devs at this point. + +01:03:18 And as you said, the people in this room, we come and go, that's very natural. + +01:03:22 So just making sure that we have the pipelines. + +01:03:25 So hopefully five, 10 more years, you know, the Django node space you mentioned earlier, that's the people who will be the presidents and people having this podcast maybe in 10 years time. + +01:03:34 Yeah, absolutely. Jeff? + +01:03:36 I'm really curious to see what Rust's impact is. + +01:03:38 I think Python is just getting so much faster, so much better. + +01:03:41 And I know there's some really smart people working on like WebAssembly and Rust. + +01:03:45 And I dream of a day when a Django view can be the same thing in your browser as it is on the server. + +01:03:51 And I think when we look at like talking to a query set, your browser, you know, doing data exchange, serialization, JSON, something. + +01:03:59 I feel like in the next five years, there's this there's this opportunity that whatever Python is becoming, whatever Django is becoming that like, you know, it's less about how are we going to use React and render? + +01:04:09 I hate React, by the way. + +01:04:10 You've got to put that plug in. + +01:04:11 But I'm just really curious to see like how these things can kind of mold together as, you know, developer experiences have gotten a lot better because of tooling and Rust. + +01:04:18 There's just this whole new world that I see as far as these technologies that start to marry the two together in a way that we just haven't been able to do for a while. + +01:04:28 But we're starting to see glimpses of it. + +01:04:29 For sure. + +01:04:30 Simon, final word for... + +01:04:31 I'm going to double down a little bit on the WebAssembly stuff is so exciting. + +01:04:36 Pyodide, the thing that lets you run Python in your browser, is now at least 10 years old. + +01:04:39 It's very, very good. + +01:04:42 It's about a two megabyte download. + +01:04:43 A few years ago, I'd have said two megabyte downloads is too much. + +01:04:46 These days, 2 megabyte download for a web page is hardly anything at all. + +01:04:49 But I feel like the thing that I love most about Django these days is it is reliable. + +01:04:55 It's so dependable. + +01:04:56 Nobody ever picks Django for a project and regrets it because Django, it's mature, it's stable, it's well documented. + +01:05:03 It moves at a sensible place. + +01:05:04 Long may that continue. + +01:05:06 I love the idea that Django becomes... + +01:05:08 And Django for many years has been a great default technology for you to use as a project. + +01:05:13 And if we double down on that, which I think is what we're doing already, that's fantastic for me. + +01:05:17 I love that idea that you can always pick Django. + +01:05:20 It will always do what you need it to do. + +01:05:22 There will never be any nasty surprises or sort of regrets from working with that. + +01:05:26 And if we can tap into things like free-threaded Python is coming along now, which is very well, it means maybe we don't have to rewrite everything as async things just to handle large concurrency. + +01:05:35 There's all of that kind of stuff, the WebAssembly stuff. + +01:05:38 It feels like the Python ecosystem is in a really good shape. + +01:05:41 And Django sits very close to the center of that ecosystem. + +01:05:45 And I'm really excited about that. + +01:05:46 I love that we can continue to provide the best default option for building great things. + +01:05:52 Yeah, totally agree. + +01:05:53 All right, everyone. + +01:05:54 Thank you so much for being here. + +01:05:55 Happy birthday, Django. + +01:05:57 Happy birthday, Django. + +01:05:59 Thanks, Michael. + +01:06:01 This has been another episode of Talk Python To Me. + +01:06:04 Thank you to our sponsors. + +01:06:06 Be sure to check out what they're offering. + +01:06:07 It really helps support the show. + +01:06:09 This episode is brought to you entirely by Sentry. + +01:06:12 It's a bit of an episode takeover, if you will. + +01:06:15 Sentry has two excellent and exciting services to tell you about. + +01:06:19 Sear, your agentic AI debugging assistant, which takes all the data already gathered by Sentry to help discover the problems and even propose fixes as GitHub PRs. + +01:06:30 And the other is AI agent monitoring, which adds deep observability to your AI agents in your app. + +01:06:37 If you're adding AI and LLM features to your Python apps, you'll want to know about AI agent monitoring. + +01:06:43 I'll tell you more about both of these later in the episode. + +01:06:46 And remember, however you happen to sign up for Sentry, if you do, use our code TALKPYTHON, one word, all caps. + +01:06:53 Want to level up your Python? + +01:06:55 We have one of the largest catalogs of Python video courses over at TALKPYTHON. + +01:06:59 Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:07:04 And best of all, there's not a subscription in sight. + +01:07:07 Check it out for yourself at training.talkpython.fm. + +01:07:10 Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:07:14 We should be right at the top. + +01:07:16 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +01:07:25 We're live streaming most of our recordings these days. + +01:07:28 If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:07:36 This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. + +01:07:40 Now get out there and write some Python code. + +01:07:54 I'm out. + diff --git a/transcripts/518-django-birthday.vtt b/transcripts/518-django-birthday.vtt new file mode 100644 index 0000000..06575a0 --- /dev/null +++ b/transcripts/518-django-birthday.vtt @@ -0,0 +1,2624 @@ +WEBVTT + +00:00:00.120 --> 00:00:08.880 +20 years after a scrappy newsroom team hacked together a framework to ship stories fast, Django remains the Python web framework that ships real apps responsibly. + +00:00:09.460 --> 00:00:25.700 +In this anniversary roundtable with its creators and longtime stewards, Simon Wilson, Adrian Holovaty, Will Vincent, Jeff Triplett, and Tiavod Kolas, we trace the path from the Lawrence Journal world to version 1.0, DjangoCon, and the Django Software Foundation. + +00:00:26.100 --> 00:00:35.100 +Unpack how a BSD license and a culture of docs, tests, and mentorship grew a global community, and revisit lessons from deployments like Instagram. + +00:00:35.700 --> 00:00:48.260 +We talk modern Django too, ASGI and async, HTMX-friendly patterns, building APIs with DRF and Django Ninja, and how Django pairs with React and Serverless without losing its batteries-included soul. + +00:00:48.660 --> 00:00:56.580 +You'll hear about Django Girls, Django Nauts, and the Django Fellowship that keeps momentum going, plus where Django fits in today's AI stacks. + +00:00:57.030 --> 00:01:01.360 +Finally, we look ahead at the next decade of speed, security, and sustainability for Django. + +00:01:02.160 --> 00:01:06.780 +This is Talk Python To Me, episode 518, recorded August 18th, 2025. + +00:01:19.960 --> 00:01:25.480 +It's time to unite. We started in pyramid cruising. Welcome to Talk Python To Me, a weekly podcast + +00:01:25.860 --> 00:02:36.740 +on Python. This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is brought to you entirely by Sentry. It's a bit of an episode takeover, if you will. Sentry has two excellent and exciting services to tell you about. Sear, your agentic AI debugging assistant, which takes all the data already gathered by Sentry to help discover the problems and even propose fixes as GitHub PRs. And the other is AI agent monitoring, which adds deep observability to your AI agents in your app. If you're adding AI and LLM features to your Python apps, you'll want to know about AI agent monitoring. I'll tell you more about both of these later in the episode. And remember, however you happen to sign up for Sentry, if you do, use our code TALKPYTHON, one word, all caps. + +00:02:37.160 --> 00:02:38.100 +Let's get to the interview. + +00:02:39.100 --> 00:02:39.560 +Hello, everyone. + +00:02:40.360 --> 00:02:41.600 +Welcome to Talk Python To Me. + +00:02:41.680 --> 00:02:45.840 +I'm so excited to be celebrating Django's 20th birthday. + +00:02:46.180 --> 00:02:46.720 +What a milestone. + +00:02:47.300 --> 00:02:51.860 +We have Will Vincent, we have Adrian, we have Theobald, Jeff, we have Simon. + +00:02:52.420 --> 00:02:56.780 +Many of the original people have the origin of Django here to talk about it. + +00:02:56.900 --> 00:02:58.240 +So, you know, welcome all of you. + +00:02:58.460 --> 00:02:58.880 +Thanks, Michael. + +00:02:59.280 --> 00:03:00.140 +Yeah, you bet. + +00:03:00.240 --> 00:03:06.360 +It's going to be a lot of fun to go back to some stories and maybe also see where things are going after 20 years. + +00:03:06.480 --> 00:03:08.840 +It's still going strong, to be sure. + +00:03:09.020 --> 00:03:10.020 +So what an accomplishment. + +00:03:10.460 --> 00:03:15.000 +Let's just start with a real quick, who am I, introduction from each of you. + +00:03:15.180 --> 00:03:18.480 +Just give us the elevator pitch on you and your Django life. + +00:03:18.740 --> 00:03:21.440 +And we'll go around the Brady Bunches in order, I suppose, here on the screen. + +00:03:21.740 --> 00:03:22.980 +So Will, welcome. + +00:03:23.140 --> 00:03:23.240 +Okay. + +00:03:23.360 --> 00:03:24.720 +So we'll go from least to most important. + +00:03:26.200 --> 00:03:30.120 +So I'm a developer advocate at JetBrains, some Django books, a podcast. + +00:03:31.260 --> 00:03:31.940 +I think that's enough. + +00:03:31.980 --> 00:03:33.580 +We have more important people to come. + +00:03:33.840 --> 00:03:34.080 +All right. + +00:03:34.480 --> 00:03:35.080 +Adrian, welcome. + +00:03:35.320 --> 00:03:37.160 +First time to have you on the show. Happy to have you. + +00:03:37.720 --> 00:03:44.940 +Hi, I'm Adrian, one of the creators of Django many years ago, precisely 20, a long time web developer. + +00:03:45.340 --> 00:03:46.660 +And yeah, it's basically it. + +00:03:47.420 --> 00:03:50.400 +I'm Simon. I'm Adrian's intern, effectively. + +00:03:51.620 --> 00:03:56.680 +We built the first version of Django during my year long paid internship at the Longest Journal World in Kansas. + +00:03:56.840 --> 00:04:01.260 +It was my university had a year in industry thing, which meant I could get a visa. + +00:04:01.340 --> 00:04:05.160 +And yeah, we worked on what we called the CMS at the time. + +00:04:05.220 --> 00:04:07.040 +We didn't know that it was a web framework. + +00:04:07.200 --> 00:04:09.120 +We thought it was a tool for building local newspaper. + +00:04:09.580 --> 00:04:09.820 +Yeah. + +00:04:10.160 --> 00:04:11.900 +Almost got named TPS, which would be amazing. + +00:04:13.080 --> 00:04:13.920 +Well, I'm a... + +00:04:14.520 --> 00:04:15.000 +Oh, go ahead. + +00:04:15.200 --> 00:04:15.360 +Sorry. + +00:04:15.740 --> 00:04:19.980 +I was going to say, Simon, you're almost more well-known for your AI stuff these days. + +00:04:20.260 --> 00:04:20.720 +You've been... + +00:04:20.840 --> 00:04:21.400 +Last couple of years. + +00:04:22.019 --> 00:04:23.240 +Like, last two years. + +00:04:23.240 --> 00:04:24.480 +You've been in the news so much. + +00:04:24.520 --> 00:04:25.580 +I see you everywhere now. + +00:04:25.720 --> 00:04:28.340 +It's like, congratulations on having two... + +00:04:28.580 --> 00:04:28.840 +Yeah. + +00:04:29.240 --> 00:04:30.099 +I mean, that was... + +00:04:30.120 --> 00:04:44.560 +was. The trick there is I have a blog and not everyone else gave up on blogging, but I kept going. And so when AI started up, I was the only person blogging about AI, which turns out gets you invited to all of the weird Silicon Valley mansion parties and things. So yeah, that's been an + +00:04:44.560 --> 00:04:48.580 +interesting little twist over the past couple of years. Yeah. Jeff Triplett, I'm so sorry to cut + +00:04:48.580 --> 00:05:00.660 +you off. Happy to have you. No, no, no. All good. I thought it was pretty free for him, but I came to Lawrence, Kansas two years after these two had left, maybe three years after these two had left To work at the newspaper, I've stuck around. + +00:05:00.820 --> 00:05:03.760 +Probably one of the few people left that stuck around that worked at the journal world. + +00:05:03.940 --> 00:05:05.660 +Me and Frank Wiles, who's over here someplace. + +00:05:06.360 --> 00:05:08.240 +I'm on the Django Software Foundation Board of Directors. + +00:05:08.600 --> 00:05:11.320 +I've helped with DjangoCon US for, I think, 11 years now. + +00:05:11.720 --> 00:05:14.880 +And do a newsletter with Will and a bunch of random Django projects. + +00:05:15.440 --> 00:05:15.940 +So that's me. + +00:05:15.940 --> 00:05:18.080 +Jeff, were you there for the whole of Media for Media? + +00:05:18.480 --> 00:05:21.660 +The commercial arm of the newspaper doing Django stuff? + +00:05:22.360 --> 00:05:23.560 +About three years of it. + +00:05:23.620 --> 00:05:27.280 +It just had moved across the street to the Little Red building they called Little Red. + +00:05:27.500 --> 00:05:32.840 +And so I think I moved when I moved here was within probably like a month or two of that starting in the building. + +00:05:33.100 --> 00:05:37.000 +Awesome. I'd love to I'd love to hear little stories from that era of Django. + +00:05:37.000 --> 00:05:37.840 +I missed that entirely. + +00:05:38.820 --> 00:05:44.120 +I'm on the board with Jeff. I'm the president of the foundation, Django Software Foundation for 2025. + +00:05:44.960 --> 00:05:49.080 +And I've been involved with Django for only 10 years, only 10 years. + +00:05:49.640 --> 00:05:50.840 +So much less than everyone is here. + +00:05:51.910 --> 00:05:53.440 +Yeah. Outside that, I'm a dev. + +00:05:53.940 --> 00:05:59.120 +I'm on the core team for an open source project called Wagtail, which is a CMS as well built on Django. + +00:05:59.440 --> 00:06:00.500 +Yeah, welcome. + +00:06:00.920 --> 00:06:03.080 +And Wagtail is a really cool project. + +00:06:03.380 --> 00:06:08.300 +I think people who are looking to take the CMS idea and level it up even more. + +00:06:08.780 --> 00:06:13.180 +And you know, I've got a little bit of Wagtail related career history. + +00:06:13.480 --> 00:06:15.400 +Wagtail came out of a company called Torchbox. + +00:06:15.480 --> 00:06:23.720 +I worked for Torchbox for about six months to a year quite early on, working on one of the first Django projects that Torchbox worked on. + +00:06:23.920 --> 00:06:26.080 +And it was a carbon calculator that we were building. + +00:06:26.290 --> 00:06:27.740 +I have nothing to do with Wagtail at all. + +00:06:27.770 --> 00:06:29.520 +That was probably 10 years after I left Torchbox. + +00:06:29.630 --> 00:06:31.560 +But I know the Torchbox team really well. + +00:06:31.820 --> 00:06:36.060 +We should also, I mean, Jeff, amongst other things, you run Django packages.org right now. + +00:06:36.130 --> 00:06:40.520 +And I think Wagtail is number one or two with Django REST framework for most downloaded. + +00:06:40.920 --> 00:06:41.720 +It's something like that. + +00:06:41.770 --> 00:06:42.600 +It's one of the top three. + +00:06:42.930 --> 00:06:44.100 +I feel confident saying that. + +00:06:44.420 --> 00:06:44.800 +Maybe you don't. + +00:06:44.910 --> 00:06:45.120 +I do. + +00:06:46.140 --> 00:06:47.620 +Yeah, it's downloaded a lot. + +00:06:47.730 --> 00:06:49.220 +I don't know if it's top 25. + +00:06:49.620 --> 00:06:50.000 +Tebow with it. + +00:06:50.000 --> 00:06:50.920 +I think it's in the top 100. + +00:06:50.960 --> 00:06:51.760 +We did a post, right? + +00:06:51.840 --> 00:06:52.740 +So, you know, you're close. + +00:06:52.900 --> 00:06:53.380 +Okay, Tebow. + +00:06:53.430 --> 00:06:54.820 +Amongst 20,000 or so plus. + +00:06:56.040 --> 00:06:56.620 +But, yeah. + +00:06:57.000 --> 00:06:57.880 +Yeah, yeah, of Django ones. + +00:06:58.220 --> 00:06:59.660 +Wait, of Django ones? + +00:07:00.080 --> 00:07:00.280 +Oh. + +00:07:00.670 --> 00:07:00.980 +What is that? + +00:07:01.100 --> 00:07:02.460 +There's like 4,000 now, Jeff? + +00:07:02.640 --> 00:07:03.260 +Something like that? + +00:07:03.580 --> 00:07:06.360 +I think there's probably like 16,000, 20,000. + +00:07:06.470 --> 00:07:09.520 +But I think we're tracking like the top 4,000 or 5,000. + +00:07:09.640 --> 00:07:11.320 +It's what people... + +00:07:11.320 --> 00:07:12.180 +I never wanted to add code. + +00:07:12.210 --> 00:07:14.320 +They just pulled everything in just to have a lot of numbers. + +00:07:14.700 --> 00:07:17.600 +So, it's still like user submitted and ran. + +00:07:18.000 --> 00:07:19.180 +Oh, it is the most important thing. + +00:07:19.180 --> 00:07:20.060 +But Tebow, GitHub stars. + +00:07:20.300 --> 00:07:21.180 +That's the most important thing. + +00:07:21.280 --> 00:07:22.320 +How many is it for Wagtail? + +00:07:22.400 --> 00:07:25.120 +But yeah, I think we just like almost at 20,000. + +00:07:25.240 --> 00:07:30.320 +I'm pretty sure if you're on here listening to this show, stars right now will be at 20,000 in like a week. + +00:07:30.640 --> 00:07:32.740 +Yeah, it's really high in downloads as well. + +00:07:32.780 --> 00:07:37.680 +But you know, the Django ecosystem is really big these days and CMS only definitely in a smaller portion of it. + +00:07:37.880 --> 00:07:38.260 +Oh, there you go. + +00:07:38.660 --> 00:07:40.160 +Very, very, very popular. + +00:07:40.420 --> 00:07:40.980 +That's amazing. + +00:07:41.420 --> 00:07:43.600 +So let's start at the beginning, folks. + +00:07:43.900 --> 00:07:50.080 +I have on the screen here a picture of the University of Kansas where I also have a connection to. + +00:07:50.140 --> 00:07:54.460 +I actually went here and got my undergraduate degree in math at this school. + +00:07:54.500 --> 00:07:56.080 +And I love my time in Lawrence. + +00:07:56.200 --> 00:08:02.480 +I think it's a lovely town, but it's also the home of Lawrence Journal World, right? + +00:08:02.860 --> 00:08:05.060 +Tell us who wants to start the story. + +00:08:05.120 --> 00:08:07.220 +Like what's the origin story here? + +00:08:07.480 --> 00:08:09.580 +Probably it makes sense for me to start it. + +00:08:09.820 --> 00:08:10.640 +Yeah, I think so. + +00:08:10.780 --> 00:08:11.200 +Let's do it. + +00:08:11.420 --> 00:08:18.300 +Back in 2002 slash 2003, early 2003, I joined that newspaper as a web developer. + +00:08:18.720 --> 00:08:24.960 +There was the guy who ran it was this guy called Rob Curley, who was seen as this big tech innovator in the news industry. + +00:08:25.090 --> 00:08:35.000 +And he was sort of recruited me and put together a great team of awesome, awesome web developers, believe it or not, in this tiny little newspaper in the middle of Kansas. + +00:08:35.900 --> 00:08:42.900 +We were tasked with building interactive database driven apps for the newspaper, which at that time was unheard of. + +00:08:43.050 --> 00:08:46.780 +Even these days, it's kind of weird for a newspaper to have developers on its staff. + +00:08:48.020 --> 00:09:03.500 +And at the time I was using PHP and yeah, fast forward, maybe a year into the job, we had an opening to hire another developer and I posted a job entry on my blog and Simon read it all the way in the UK. + +00:09:04.260 --> 00:09:08.340 +And we followed each other's blogs back when that was like the social media of the day. + +00:09:08.690 --> 00:09:09.280 +To be fair. + +00:09:09.370 --> 00:09:11.500 +Much more interesting than social media of today. + +00:09:11.800 --> 00:09:16.300 +At the time, I felt like there were about 100 people blogging about web development. + +00:09:16.580 --> 00:09:17.360 +And we were two of those. + +00:09:17.600 --> 00:09:19.700 +Yeah, and we all followed everyone. + +00:09:20.600 --> 00:09:21.920 +Did you guys have a blog role? + +00:09:22.440 --> 00:09:23.460 +Oh, yeah, absolutely. + +00:09:25.079 --> 00:09:27.620 +I think we were both massive fans of Mark Pilgrim. + +00:09:27.800 --> 00:09:33.140 +Mark Pilgrim was the web development of Python blogger that we were paying attention to. + +00:09:33.420 --> 00:09:34.720 +Unfortunately, he's not around anyway. + +00:09:34.940 --> 00:09:37.960 +I mean, he's alive, but he stopped blogging many, many years ago. + +00:09:38.280 --> 00:09:40.640 +Yeah, he wrote an online book called Dive into Python. + +00:09:41.220 --> 00:09:44.220 +And we both read that and we're like, oh, what's this Python thing? + +00:09:44.620 --> 00:09:51.600 +And thanks to the kindness or possibly ignorance of our boss, we just started using Python one day. + +00:09:52.120 --> 00:09:53.880 +We didn't really have to get much buy-in or anything. + +00:09:54.960 --> 00:10:00.520 +And we'd started making little bits and pieces in Python, never really with the intent of making a framework. + +00:10:00.880 --> 00:10:06.780 +But over time, at some point, we just decided, okay, we're switching to Python. + +00:10:06.900 --> 00:10:07.600 +No more PHP. + +00:10:07.820 --> 00:10:12.500 +just going to do everything in this new thing. And I remember the key thing for us is how do + +00:10:12.500 --> 00:10:51.260 +you deploy Python? Well, there was modpython, the Apache module, and we were just a little bit nervous about it because not many other people were using it. And so we're thinking, okay, well, we can try modpython, but what if it doesn't work? So, okay, well, if we have an abstraction layer between our code and modpython, like a request and a response object, then if modpython doesn't work out, we can swap it out for something else. We didn't, modpython worked great, But that abstraction layer, I think that was the origins of Django, that tiny little layer between R code and mod python, just in case. Classic thing you shouldn't do in programming, right? You're not going to need it. You don't need the abstraction layer, but we built one anyway. And fast forward 20 years and maybe that was a good idea after all. + +00:10:51.740 --> 00:10:54.300 +Is mod python still around? It's got to be, right? + +00:10:54.500 --> 00:10:58.320 +I don't know. I mean, modwhiskey, it was replaced by modwhiskey many years ago. + +00:10:59.240 --> 00:11:03.480 +But yeah, I don't know actually. I've not checked in on the ModPython project in a while. + +00:11:04.519 --> 00:11:08.100 +It's been last commit eight months ago to ModPython on GitHub. + +00:11:08.100 --> 00:11:08.700 +Oh, very smart. + +00:11:08.720 --> 00:11:09.140 +Yeah, yeah. + +00:11:09.860 --> 00:11:10.600 +That sounds alive. + +00:11:10.820 --> 00:11:12.020 +Yeah, yeah, it's definitely alive. + +00:11:12.340 --> 00:11:18.000 +And now we have WSGI and ASGI and that's sort of carrying on the same theme in a way. + +00:11:18.300 --> 00:11:37.940 +You know, WSGI kind of came out of the work we were doing in that we were working on Django, at the Journal World. The mailing list popped up, the Python web-sig mailing list. This mailing list popped up. We were very active on that initially, trying to pitch our request and response objects. + +00:11:38.140 --> 00:11:49.160 +We did not win with our response request objects, and Whiskey came out of that instead. But yeah, that was definitely... Whiskey came out of that early era of what we were trying to achieve with with the stuff we were doing. + +00:11:49.380 --> 00:11:55.700 +So we built up this series of libraries slash tools slash, I guess it turned into a framework. + +00:11:56.100 --> 00:11:59.120 +And we started thinking, oh, this is kind of interesting. + +00:11:59.600 --> 00:12:04.380 +Personally, speaking for myself, I was like, oh, I'm probably not going to work here forever, but I would love to keep using this. + +00:12:04.600 --> 00:12:07.160 +So that sort of started planting the seed about open sourcing it. + +00:12:07.600 --> 00:12:15.240 +But I think very few people today know that even 20 years ago, there was this meme that there are way too many Python web frameworks. + +00:12:15.980 --> 00:12:20.200 +So we went to PyCon US and it had to have been 2004. + +00:12:21.460 --> 00:12:29.000 +And there was a keynote where the speaker said, she showed a slide that was like a tag cloud of all the Python web framework names. + +00:12:29.490 --> 00:12:30.700 +And there were like 50 of them. + +00:12:31.140 --> 00:12:33.980 +So Simon and I were sitting there like, + +00:12:34.160 --> 00:12:36.260 +oh shoot, we made yet another one. + +00:12:36.280 --> 00:12:36.820 +We're fools. + +00:12:37.120 --> 00:12:37.740 +We're so stupid. + +00:12:38.020 --> 00:12:38.400 +We're embarrassed. + +00:12:39.000 --> 00:12:44.700 +But I ended up doing a lightning talk at that PyCon and showed what we had. + +00:12:44.860 --> 00:12:46.280 +It didn't have a name yet. + +00:12:46.500 --> 00:12:50.040 +And, you know, people came up to me afterward and said, oh, please open source. + +00:12:50.520 --> 00:12:51.520 +We're very interested in this. + +00:12:51.560 --> 00:12:55.260 +It looks very promising open source, even though it's yet another Python web framework. + +00:12:55.520 --> 00:13:01.360 +So I guess extrapolating a lesson from that, it's never too late to make stuff. + +00:13:01.640 --> 00:13:05.840 +I think another fact there is that Ruby on Rails had come out maybe six months before that PyCon. + +00:13:05.960 --> 00:13:10.920 +Like Ruby on Rails was the absolute buzz of everything in the web development industry at the time. + +00:13:11.500 --> 00:13:15.240 +And there was an open question like, what's Python's answer to Ruby on Rails? + +00:13:15.360 --> 00:13:19.320 +And so it's possible that the Django timing was really good on that front. + +00:13:19.580 --> 00:13:23.140 +I believe that Rails helped influence the newspaper to open source Django. + +00:13:23.300 --> 00:13:24.860 +Was that part of the conversation then? + +00:13:25.460 --> 00:13:34.800 +Well, I remember making a list of 10 to 20 reasons why we should open source it and really thinking hard about it, probably lost some sleep about it. + +00:13:35.120 --> 00:13:39.100 +And we went into the meeting with Rob to make our pitch. + +00:13:39.540 --> 00:13:42.060 +And we only got to like five reasons. + +00:13:42.270 --> 00:13:43.580 +And finally he was like, yeah, let's do it. + +00:13:44.020 --> 00:13:47.700 +So, I mean, endless, infinite credit to him for being cool. + +00:13:48.240 --> 00:13:52.180 +And I mean, the company took advantage of open source a lot. + +00:13:52.290 --> 00:14:01.860 +And I think the higher ups knew it because it also owned a cable company that did a lot of stuff with Linux and all sorts of crazy cable company software that was open source. + +00:14:02.400 --> 00:14:03.640 +Which was made Frank, right? + +00:14:03.790 --> 00:14:07.320 +That was Frank Wiles at the cable company driving all of that stuff. + +00:14:07.900 --> 00:14:24.480 +Yeah, you know, I think it's worth pointing out as well. One, Adrian, you're making me nostalgic just to think about tag clouds. Okay. They were such a thing back in the early 2000s. It's a usability nightmare. I know. Any kind of data or information out of that thing. + +00:14:24.580 --> 00:14:25.860 +It really is. + +00:14:25.980 --> 00:14:31.320 +I think I use, oh my God, Delicious a lot, and you'll get tag clouds out of it and all sorts of stuff. + +00:14:31.620 --> 00:14:33.160 +But that's not why I'm bringing this up. + +00:14:33.280 --> 00:14:39.680 +More in that timeframe, open sourcing something was not the clear choice it is today. + +00:14:39.920 --> 00:14:42.540 +At least it was really more of a push, I think. + +00:14:43.140 --> 00:14:45.360 +Legal liabilities, giving away our IP. + +00:14:45.650 --> 00:14:46.560 +Like it was a different time. + +00:14:46.700 --> 00:14:48.700 +There was open source, but not like today. + +00:14:50.760 --> 00:14:54.060 +This portion of Talk Python To Me is brought to you by Centuries Seer. + +00:14:54.500 --> 00:14:57.540 +I'm excited to share a new tool from Sentry, Seer. + +00:14:58.060 --> 00:15:05.400 +Seer is your AI-driven pair programmer that finds, diagnoses, and fixes code issues in your Python app faster than ever. + +00:15:05.900 --> 00:15:09.600 +If you're already using Sentry, you are already using Sentry, right? + +00:15:10.100 --> 00:15:14.380 +Then using Seer is as simple as enabling a feature on your already existing project. + +00:15:15.180 --> 00:15:18.520 +Seer taps into all the rich context Sentry has about an error. + +00:15:19.020 --> 00:15:23.080 +Stack traces, logs, commit history, performance data, essentially everything. + +00:15:23.680 --> 00:15:27.800 +Then it employs its agentic AI code capabilities to figure out what is wrong. + +00:15:28.420 --> 00:15:32.200 +It's like having a senior developer pair programming with you on bug fixes. + +00:15:32.960 --> 00:15:38.300 +Sear then proposes a solution, generating a patch for your code and even opening a GitHub pull request. + +00:15:38.860 --> 00:15:43.480 +This leaves the developers in charge because it's up to them to actually approve the PR. + +00:15:43.900 --> 00:15:47.620 +But it can reduce the time from error detection to fix dramatically. + +00:15:48.400 --> 00:15:53.960 +Developers who've tried it found it can fix errors in one shot that would have taken them hours to debug. + +00:15:54.660 --> 00:15:59.060 +SEER boasts a 94.5% accuracy in identifying root causes. + +00:15:59.640 --> 00:16:06.320 +SEER also prioritizes actionable issues with an actionability score, so you know what to fix first. + +00:16:06.940 --> 00:16:13.940 +This transforms sentry errors into actionable fixes, turning a pile of error reports into an ordered to-do list. + +00:16:14.600 --> 00:16:23.660 +If you could use an always-on-call AI agent to help track down errors and propose fixes before you even have time to read the notification, check out Sentry's Seer. + +00:16:24.300 --> 00:16:28.280 +Just visit talkpython.fm/seer, S-E-E-R. + +00:16:28.940 --> 00:16:30.780 +The link is in your podcast player's show notes. + +00:16:31.300 --> 00:16:33.600 +Be sure to use our code, TALKPYTHON. + +00:16:33.960 --> 00:16:34.860 +One word, all caps. + +00:16:35.540 --> 00:16:37.600 +Thank you to Sentry for supporting Talk Pythonemy. + +00:16:38.520 --> 00:16:44.900 +Well, also, the publisher of the newspaper at the time still used a mechanical typewriter to write up his things. + +00:16:45.100 --> 00:16:46.180 +He was legendary for it. + +00:16:47.140 --> 00:16:49.000 +But my understanding is that he got into it. + +00:16:49.880 --> 00:16:54.340 +The argument that was convincing to the newspaper was, "We benefit from this community. + +00:16:54.400 --> 00:16:58.780 +We should give back to this community." That evidently was part of the pitch. + +00:17:00.860 --> 00:17:02.160 +It just took off. + +00:17:03.360 --> 00:17:07.380 +I feel like that newspaper did very well from that one decision that they made back then. + +00:17:07.740 --> 00:17:27.459 +Yeah, I mean, here we are this many years later talking about it on a podcast, right? Like already, a lot of people know about it. And a lot of people, all of you on the call, but many, many, many others have contributed back to Django to make so much better. If that was just an internal CMS that you guys left there, how many external contributors would you have? Right? Not that many. + +00:17:27.660 --> 00:17:34.340 +Well, one thing, actually, I'm curious while the two of you are here, I heard a story that there were some efforts to monetize it that didn't pan out. + +00:17:34.800 --> 00:17:35.800 +Jeff, maybe you were there for that. + +00:17:36.080 --> 00:17:36.840 +Is that true? + +00:17:37.000 --> 00:17:42.820 +Were there some efforts to package it up and sell it to newspapers in some capacity as a premium product, I think? + +00:17:43.000 --> 00:17:43.760 +Am I making that up? + +00:17:44.620 --> 00:17:46.460 +That was after my time, I think. + +00:17:46.860 --> 00:17:47.580 +Are you talking about? + +00:17:47.820 --> 00:17:48.800 +Maybe Frank mentioned that. + +00:17:48.820 --> 00:17:50.480 +I thought there was something around. + +00:17:50.800 --> 00:17:51.780 +Yeah, this was Ellington. + +00:17:51.920 --> 00:17:52.720 +I think we should. + +00:17:52.720 --> 00:17:53.200 +Oh, I guess. + +00:17:53.320 --> 00:17:53.400 +Yeah. + +00:17:53.600 --> 00:17:53.680 +Sorry. + +00:17:53.860 --> 00:18:25.680 +to Jeff who um yeah this is kind of my era I guess then was um and I still feel like I'm new to Django because I felt like and I've been around for 18 years but like I just I saw these guys code so much and then I came into that era so to me it felt like next gen Star Trek versus now the originals um and so yeah when I started we had hundreds of newspapers were buying Ellington which was basically like what was it the first or second Django app ever written and they basically created a CMS to run your newspaper and then they also had something called marketplace which was almost like an online Craigslist or business directory. + +00:18:26.090 --> 00:18:31.780 +And so I came in to kind of help manage some of the inventory and do a lot of data imports and stuff for newspapers. + +00:18:32.050 --> 00:18:34.240 +Like we had all of Massachusetts at one point. + +00:18:35.060 --> 00:18:37.660 +There was a lot of kind of that, they went that direction with it. + +00:18:38.060 --> 00:18:43.400 +And I left before that kind of had shuttered and I think they sold it to a big company to buy it. + +00:18:43.410 --> 00:18:45.420 +And I think Ellington still runs to this day. + +00:18:46.340 --> 00:18:50.720 +At DjangoCon every two or three years, somebody will approach and say, we want to open source Ellington. + +00:18:50.910 --> 00:18:52.920 +Would you like to, could we do this at DjangoCon? + +00:18:53.040 --> 00:18:58.140 +We always say, "Yeah, that'd be really cool." Then six years has gone by and it happens every so often. + +00:18:58.620 --> 00:19:04.860 +I hope that code gets to see the light of day because it was hundreds of apps by the time we worked on it for hundreds of newspapers. + +00:19:05.300 --> 00:19:10.380 +I feel like the product was fantastic and the market was great, but it's a terrible market to be selling into. + +00:19:11.060 --> 00:19:18.380 +In the late 2000s, early 2010s, if you're selling to local newspapers, which are blinking out of existence at a rate of knots. + +00:19:21.820 --> 00:19:25.120 +My hunch is that the marketplace challenges were down to that. + +00:19:25.280 --> 00:19:28.720 +It was selling into a shrinking industry, which is a difficult thing to do. + +00:19:29.000 --> 00:19:31.120 +Yeah, I think they were hitting a brick wall for sure. + +00:19:31.840 --> 00:19:33.740 +You mentioned Rails earlier and PHP. + +00:19:34.340 --> 00:19:35.540 +And two things come to mind. + +00:19:35.900 --> 00:19:39.900 +I have an embarrassing email I sent to Simon at one point because I followed your PHP blog. + +00:19:40.240 --> 00:19:43.360 +And also Matt Croydon, who later became my boss at Media for Media. + +00:19:43.700 --> 00:19:44.400 +And I did not. + +00:19:44.420 --> 00:19:45.420 +I saw a couple of things. + +00:19:45.620 --> 00:19:48.440 +I think you were so into Django, you weren't really posting about Django. + +00:19:48.860 --> 00:19:51.880 +And so I think I emailed you once to say, have you seen this thing called Django? + +00:19:52.350 --> 00:19:53.260 +It looks pretty cool. + +00:19:53.450 --> 00:19:55.400 +Now that you're doing Python, I think you may like it. + +00:19:55.610 --> 00:19:58.780 +And so if you ever check your Gmail, it may be there. + +00:19:58.890 --> 00:20:00.360 +We don't have to do anything live on the air. + +00:20:00.560 --> 00:20:04.080 +But yeah, I to this day think, oh man, that was kind of wild. + +00:20:04.820 --> 00:20:10.040 +What brought me to Django though was I did get the Rails book because they very quickly created a Ruby on Rails book. + +00:20:10.050 --> 00:20:14.540 +It was kind of the main way to learn how to use it because their website kind of sucked as far as documentation went. + +00:20:14.880 --> 00:20:19.720 +And as I was going through it, so much was boilerplate just to get anything running in Rails. + +00:20:20.070 --> 00:20:28.060 +And so the Django admin was just so beautifully designed and well done that to me, it was just this win of like, this looks better than anything I could do by hand. + +00:20:28.250 --> 00:20:30.620 +And it's largely stood the test of time for 20 years. + +00:20:31.380 --> 00:20:35.980 +I remember like Wilson Miner I had met a couple of times had done some of that work or most of that work. + +00:20:36.170 --> 00:20:40.200 +He did all of that design work and he designed the original Django website. + +00:20:40.540 --> 00:20:50.400 +And I think that's one of the main reasons that we got a lot of traction was because the design of the site and the documentation were so good, especially considering the context. + +00:20:50.480 --> 00:21:01.780 +I mean, that back in those times, it was par for the course for an open source project to have to be completely on style, like a normal, like browser default H1 Craigslist background. + +00:21:02.280 --> 00:21:04.320 +This design isn't the one that Wilson did. + +00:21:04.560 --> 00:21:07.580 +The one that's archived that or get to go back. + +00:21:07.900 --> 00:21:09.700 +But it was pretty timeless at the time, though. + +00:21:09.700 --> 00:21:10.180 +It was amazing. + +00:21:10.640 --> 00:21:15.080 +And I feel like the admin has evolved a lot, but it still has that sort of... + +00:21:15.080 --> 00:21:18.280 +It feels like the Wilson minor design today, even 20 years later. + +00:21:19.660 --> 00:21:24.980 +Wilson was there the same time I was at the Journal World, and he then went on to early Facebook. + +00:21:25.200 --> 00:21:27.160 +He was, he's bounced around Silicon Valley. + +00:21:27.900 --> 00:21:28.020 +Yeah. + +00:21:28.180 --> 00:21:29.920 +Redesigned Apple.com, I think too. + +00:21:30.260 --> 00:21:32.820 +He worked with me at EveryBlock for a few years. + +00:21:33.480 --> 00:21:34.740 +He's now working on Apple Music. + +00:21:35.060 --> 00:21:35.880 +I didn't know that. + +00:21:35.880 --> 00:21:36.240 +Excellent. + +00:21:37.100 --> 00:21:40.520 +As of last time we talked to him, which was recent-ish. + +00:21:40.940 --> 00:21:41.500 +Yeah, that's quite neat. + +00:21:41.920 --> 00:21:43.160 +Trying to pull it up here in the Wayback Machine. + +00:21:43.360 --> 00:21:44.700 +We'll see what I can get. + +00:21:44.730 --> 00:21:45.860 +I think the other thing I... + +00:21:46.020 --> 00:21:46.600 +How about this, guys? + +00:21:46.740 --> 00:21:47.280 +Is this the OG? + +00:21:47.460 --> 00:21:47.800 +There we go. + +00:21:48.240 --> 00:21:48.500 +Yep, yep. + +00:21:50.220 --> 00:21:50.820 +That's the one. + +00:21:51.120 --> 00:21:53.860 +And that, to be fair, that looks good by today's standard. + +00:21:54.340 --> 00:21:59.980 +You know, I feel like I would not be at all upset to publish a website look like that in 2025. + +00:22:00.740 --> 00:22:01.560 +Yeah, that is pretty timeless. + +00:22:01.860 --> 00:22:02.100 +That's cool. + +00:22:02.400 --> 00:22:03.380 +Wait, I see the... + +00:22:03.650 --> 00:22:04.160 +Oh, go ahead, Simon. + +00:22:04.500 --> 00:22:23.220 +Oh, it's good. The other thing that's worth mentioning with Django, Django's documentation has always been fantastic. And I think that's basically because it was Jacob Kaplan Moss had a, he was a literature major and you have Adrian, who's a journalism major. And it turns out if you put a journalist and somebody who like put that talent on your documentation, you get really + +00:22:23.320 --> 00:22:34.980 +good documentation. There was, that was a double-edged sword because it took a few years for any publisher to be interested in publishing a book about Django because the standard line was always, oh, the docs are good enough. + +00:22:35.220 --> 00:22:36.860 +Everyone just, there's no market for it. + +00:22:38.400 --> 00:22:47.240 +Well, but at the same time, I wanted to ask, I mean, because there was the link there to the Django book, which I have the URL now and Jeff and I run, it lists all the Django books. + +00:22:47.460 --> 00:22:49.260 +But I mean, the two of you wrote that. + +00:22:49.340 --> 00:22:50.640 +I thought that was a great book. + +00:22:50.740 --> 00:22:52.360 +I think you, it was in print, right? + +00:22:52.440 --> 00:22:53.660 +But it was mainly open source, Adrian? + +00:22:54.020 --> 00:22:54.920 +Yeah, it was both. + +00:22:55.040 --> 00:22:58.040 +It was free online and it was published by A-Press. + +00:22:58.480 --> 00:22:59.900 +There were a couple of editions, yeah. + +00:23:00.240 --> 00:23:07.240 +The interesting question for Paul Everett out there in the audience is, In the first years, did your team also have to admin the newspaper site? + +00:23:07.330 --> 00:23:09.660 +Like, did you DevOps before DevOps was a thing? + +00:23:10.300 --> 00:23:11.160 +Or how did you run it? + +00:23:11.350 --> 00:23:12.980 +The cloud stuff wasn't that big back then. + +00:23:13.240 --> 00:23:14.640 +No, we had our own physical servers. + +00:23:15.240 --> 00:23:15.880 +In the basement. + +00:23:16.330 --> 00:23:17.000 +In the basement. + +00:23:17.720 --> 00:23:18.660 +The team was in the basement. + +00:23:18.680 --> 00:23:19.660 +You could probably hear them. + +00:23:19.750 --> 00:23:20.260 +Your programmers. + +00:23:20.780 --> 00:23:23.860 +And yeah, there was a room in the basement with physical servers. + +00:23:24.370 --> 00:23:25.780 +And oh, I've forgotten their names now. + +00:23:25.890 --> 00:23:27.440 +I think Frank helped. + +00:23:27.560 --> 00:23:29.880 +Well, they were named after characters in Beatles songs. + +00:23:30.000 --> 00:23:30.960 +One was Mr. Mustard. + +00:23:31.090 --> 00:23:32.840 +One was Pam from Polytheon Pam. + +00:23:33.260 --> 00:23:46.420 +Yes. Yep. But yeah, and Frank and Nick from Sunflat Cable helped us wire those up. But yeah, it was a fit. There was an Apple server in there as well. That was the media server, + +00:23:46.830 --> 00:23:54.600 +I think. It was doing like imagery sizing. Yeah, it was like a 2U box or something. I forget. It was actually like a physical server that now, because I'd never seen one before in person. + +00:23:55.040 --> 00:24:07.460 +XServe. It was an XServe. Yeah. Also, I remember that room had a wire that was the AP News wire, Like I didn't realize there was a physical wire that the news came in on from the Associated Press. + +00:24:07.910 --> 00:24:10.540 +And when you hear it came in over the wire, that was the wire? + +00:24:10.780 --> 00:24:11.420 +That was the wire. + +00:24:11.520 --> 00:24:13.440 +We had the actual wire in our basement. + +00:24:13.700 --> 00:24:13.760 +Yeah. + +00:24:14.640 --> 00:24:14.980 +Incredible. + +00:24:15.520 --> 00:24:29.280 +Well, Simon and I were lucky because we didn't really have any sysadmin experience, but there were some really experienced guys, Frank and Nick, who worked at the company and actually did some training with us. + +00:24:29.680 --> 00:24:31.680 +I remember going into their dark room. + +00:24:32.500 --> 00:24:33.800 +All the lights were like turned off. + +00:24:34.000 --> 00:24:35.820 +You could only see like these Unix shells. + +00:24:36.080 --> 00:24:38.200 +I was like, whoa, I'm really learning the good stuff. + +00:24:38.720 --> 00:24:39.880 +And this is the same Frank. + +00:24:40.020 --> 00:24:43.940 +This is Frank Wiles, who he's been president of the Django Software Foundation in the past. + +00:24:43.980 --> 00:24:46.660 +And he's still very heavily involved with Django to this day. + +00:24:46.960 --> 00:24:47.120 +Yeah. + +00:24:47.440 --> 00:24:47.600 +All right. + +00:24:47.680 --> 00:24:48.760 +Let's talk about 2008. + +00:24:48.760 --> 00:24:50.420 +A lot of stuff happened around 2008. + +00:24:50.940 --> 00:24:54.800 +That was the timeframe a little bit before then Django's open source. + +00:24:55.280 --> 00:25:00.000 +But maybe you'll have to tell me, maybe a little bit inspired by Ruby on Rails. + +00:25:00.480 --> 00:25:05.640 +There's a lot of, in Ruby on Rails, a lot of convention over configuration, a lot of magic. + +00:25:05.980 --> 00:25:09.160 +And Django used to have a little more implicit magic. + +00:25:09.280 --> 00:25:14.100 +And there was the, when Django went to, around when it went to 1.0, there was the magic removal. + +00:25:14.380 --> 00:25:16.800 +And it was the first DjangoCon, all those things. + +00:25:17.200 --> 00:25:18.740 +Tell us about that, whoever wants to take it. + +00:25:18.940 --> 00:25:21.640 +I can talk to DjangoCon a little bit, the very first DjangoCon. + +00:25:22.539 --> 00:25:29.600 +I was working for a radio station in London that was doing a big Django project. + +00:25:30.220 --> 00:25:35.680 +One of my co-workers there was this chap, Rob Lofthouse, who very foolishly stepped up and said, "Well, okay. + +00:25:36.200 --> 00:25:38.460 +Everyone says there should be a conference and no one's arranging a conference. + +00:25:38.620 --> 00:25:39.500 +I'll arrange a conference. + +00:25:39.660 --> 00:25:43.940 +How hard can it be?" The answer is everyone in the school understands it's very, very, very hard. + +00:25:44.320 --> 00:25:45.160 +He managed to pull it off. + +00:25:46.100 --> 00:25:48.640 +The big thing there was that Google offered to host it. + +00:25:49.260 --> 00:25:53.140 +We got the venue and the catering and everything was in Google's campus in Mountain View. + +00:25:53.540 --> 00:25:55.900 +Was 2008, was that the year of the first DjangoCon? + +00:25:56.170 --> 00:25:57.080 +I think it was, right? + +00:25:57.320 --> 00:25:57.720 +I think it was. + +00:25:58.480 --> 00:26:01.580 +And yeah, we pulled together a conference. + +00:26:03.620 --> 00:26:06.400 +The videos from that conference were all available on YouTube, actually. + +00:26:06.470 --> 00:26:10.500 +I rewatched one of the panels quite recently, just out of interest. + +00:26:11.040 --> 00:26:30.740 +It was really fascinating seeing there are people on those panels, like Russell Keith mcgee was on the database migrations panel and he was just like one of the people who was trying to solve database migrations back then and of course he's been an enormous fixture in the python and django community ever since um but yeah that was that was really fun that was uh like how many of + +00:26:30.740 --> 00:26:46.900 +us were that jeff were you at the first django conference i was i was barely working at the journal world like maybe six months or a year or something so yeah i just just kind of experienced at all as a nobody. So it was great. And I think my biggest contribution to that conference was, + +00:26:47.380 --> 00:27:13.440 +so I was working on a migration system called demigrations and Andrew Godwin was working on, I think he was calling it South back then. the, like his migration and we had Russell Keith and I got, I managed to, I made the pitch for Andrew, who was just a student at university at time to be flown to California to present at this conference. I think it may have been his first ever conference talk. And he has been to and spoken at every Django and Python conference since. + +00:27:13.760 --> 00:27:19.000 +So I like to think I helped just provide that little spark to get his speaking career started. + +00:27:20.180 --> 00:27:24.440 +But yeah, it was a very, very notable moment, that first DjangoCon. + +00:27:24.680 --> 00:28:00.920 +I had a mind-blowing moment there. A guy from, I think it was Japan, came up to me. He didn't speak a word of English and he gave me his book that he had written about Django whoa and and he just like bowed and and went away and that's I still think about that and it still kind of brings a tear to me it's like what it's it's mind-blowing that people all around the world that you've never met who speak languages you don't speak use your code like it I still it's been 20 years I still can't even like 100% come to terms with it it's really cool yeah I thought + +00:28:00.980 --> 00:28:10.000 +you're gonna go a different direction and you're gonna say when Cal Henderson put that sign up of the unicorn like that was going to forever be the moment but i don't know if you want to expand well + +00:28:10.200 --> 00:28:33.780 +that's the downside of that first jango con there was a q a with me and jacob and i think it was me i said are there any pony requests and that was intended to mean are there any feature ideas sort of open suggestions and evidently that was the um the impetus for the idea of the jango pony which I despise. + +00:28:34.190 --> 00:28:38.540 +I think it's so dumb and I think that's directly related to me retiring from the project, frankly. + +00:28:38.860 --> 00:28:39.280 +Oh no! + +00:28:39.620 --> 00:28:43.480 +I know it's funny in a way but it's also infuriating to me. + +00:28:43.480 --> 00:28:48.140 +I think it's so stupid and unprofessional and the type of humor is just dumb. + +00:28:48.560 --> 00:28:49.520 +No wit at all. + +00:28:49.700 --> 00:28:50.640 +I'll stop my rant. + +00:28:50.860 --> 00:28:51.560 +That'll be on my own. + +00:28:51.680 --> 00:28:55.500 +Whenever I start my own podcast I'm going to have the first episode on why the Django pony sucks. + +00:28:56.260 --> 00:28:57.260 +Wait, what should it be though? + +00:28:57.530 --> 00:28:58.440 +What would you like it to be? + +00:28:58.610 --> 00:28:59.280 +Do you have thoughts? + +00:28:59.620 --> 00:29:00.840 +I think you underestimated. + +00:29:02.240 --> 00:29:06.180 +I would have guessed it would have went like a it should at least be playing a guitar people love + +00:29:07.740 --> 00:29:09.920 +celebrating this for that event and occasion + +00:29:10.780 --> 00:29:15.140 +qualified people Timo I have one of your stickers on my fridge + +00:29:15.320 --> 00:29:17.500 +that's one of the things I did for the 20th birthday + +00:29:17.700 --> 00:29:19.100 +I designed a new pony + +00:29:20.060 --> 00:29:27.300 +to distribute to people around the world you might have seen one at the Amsterdam event people really like the mascots + +00:29:27.780 --> 00:29:31.000 +a bag of the stickers was passed to me and I passed it along. + +00:29:31.570 --> 00:29:33.220 +To the trash can. No, just kidding. + +00:29:35.180 --> 00:29:39.220 +This portion of Talk Python To Me is brought to you by Sentry's AI agent monitoring. + +00:29:39.990 --> 00:29:43.200 +Are you building AI capabilities into your Python applications? + +00:29:43.950 --> 00:29:52.060 +Whether you're using open AI, local LLMs, or something else, visibility into your AI agent's behavior, performance, and cost is critical. + +00:29:52.860 --> 00:29:57.280 +You will definitely want to give Sentry's brand new AI agent monitoring a look. + +00:29:58.140 --> 00:30:07.740 +AI agent monitoring gives you transparent observability into every step of your AI features so you can debug, optimize, and control the cost with confidence. + +00:30:08.540 --> 00:30:11.900 +You'll get full observability into every step of your AI agents. + +00:30:12.040 --> 00:30:16.900 +That is model calls, prompts, external tool usage, and custom logic steps. + +00:30:17.740 --> 00:30:24.240 +AI agent monitoring captures every step of an AI agent's workflow from the user's input to the final response. + +00:30:24.940 --> 00:30:31.480 +And your app will have a dedicated AI agent's dashboard showing traces and timelines for each agent run. + +00:30:32.080 --> 00:30:40.560 +You'll get alerts on model errors, latency spikes, token usage surges, and API failures protecting both performance and cost. + +00:30:41.020 --> 00:30:43.480 +It's plug-and-play Python SDK integration. + +00:30:43.980 --> 00:30:49.800 +Open AI for now for Django, Flask, and FastAPI apps with more AI platforms coming soon. + +00:30:50.400 --> 00:30:58.960 +In summary, AI agent monitoring turns the often black box behavior of AI in your app into transparent, debuggable processes. + +00:30:59.700 --> 00:31:04.680 +If you're adding AI capabilities to your Python app, give Sentry's AI agent monitoring the look. + +00:31:05.200 --> 00:31:13.680 +Just visit talkpython.fm/sentryagents to get started and be sure to use our code TALKPYTHON, one word, all caps. + +00:31:14.120 --> 00:31:16.060 +The link is in your podcast player's show notes. + +00:31:16.520 --> 00:31:19.100 +Thank you to Sentry for supporting Talk Python and me. + +00:31:20.720 --> 00:32:00.040 +another notable thing that happened in 2008 other than the housing crisis crash that wrecked a bunch of things was the foundation right the jingo software foundation and i think more broadly i think one of the things that makes you know i talked to a lot of different people about different web frameworks and the thing that really makes jingo stick and stand out i think is one the admin thing but the other is the community around jingo and we're talking about the jingo conference the foundation it's just unlike other things there's there's a whole community that you get into when you go into django not just some apis you use you know what i mean so uh who wants to take the + +00:32:00.570 --> 00:32:35.000 +tell us about how the foundation got started so that was i was there uh so the there was the question of who owns the code um it was i think technically owned by the lawrence journal world correct me if I'm wrong. And so that wasn't going to last forever. So we needed to find for legal, you know, ownership reasons, an entity to own it. We didn't want to go the route of Ruby on Rails, copyright David Hennemeyer Hanson. I think that's just really cringe. So it's good to have an organization as part of it. So yeah, we set up 501c3 if that's the right one that we use. I forget. + +00:32:36.080 --> 00:32:41.160 +And yeah, that's basically the origin story. From my perspective, I'm sure there's a lot more + +00:32:41.180 --> 00:32:46.780 +subtlety to it oh gosh do you want to weigh in yeah i was only barely started that time so + +00:32:47.050 --> 00:33:16.340 +like i definitely relate to you know the community around django you were all telling why you got involved and when and how and for me it was it was whacked as a day job but then what got me to stick around is actually django girls and organizing some of those local workshops and you know having organizations like this like alongside the django project not even the main thing is definitely only possible because we've had such a like long running open source foundation. There's lots of aspects of the foundation we could discuss, but definitely, you know, that kind of longevity + +00:33:16.940 --> 00:33:33.160 +for community run open source is rare and to be cherished. Yeah. Maybe you could tell us a little bit like what was it when it started and what is it now? Like how has it changed over the time, over the years? I'm going to put in a plug for the Django Fellows program. I feel like the Django + +00:33:33.200 --> 00:33:38.520 +Fellows program has been an absolutely astonishingly effective part of the mission of the foundation. + +00:33:38.820 --> 00:34:05.900 +For people who aren't familiar with that, the Django Software Foundation employs fellows who are paid contributors who work on the bits of open source maintenance that aren't necessarily so glamorous. They do bug issue, they do code review and handle security incidents and bug triage and all of these different things. And thanks to them, Django has a release schedule that operates like clockwork. Like Django, there's a new release that comes out when they said it was going to come out. + +00:34:06.060 --> 00:34:45.040 +Code gets reviewed, bugs get trashed. It's a wonderful, wonderful thing. It's unique because most open source projects that are run like that have a company behind them. There are people who have a full-time job and there's a company that's commercially supporting that. Django does not have a commercial sponsor in that respect. It has this nonprofit that people fund and that employs these these fellows and the fellows keep everything ticking long. I keep on saying to other open source projects, if you can figure out a pattern to use that's similar to the Django fellows, that's absolutely worth doing. The Python Software Foundation has a developer in residence program now, or several developers in residence, which were inspired by the Django fellows program. + +00:34:45.320 --> 00:34:59.380 +I think it's been really influential. And it's notable because there are very few major open source projects that don't have a company behind them. The other one that comes up a lot is Postgres, Postgres is very much a community-organized program. + +00:34:59.800 --> 00:35:03.800 +If you look at the way Postgres works, a lot of the key maintainers have their own consultancies. + +00:35:04.180 --> 00:35:08.700 +So they've got a good way of financially sustaining the work that they're doing. + +00:35:09.240 --> 00:35:12.640 +I think the Django Fellows Program, which was introduced... + +00:35:13.140 --> 00:35:13.760 +When was that? + +00:35:13.820 --> 00:35:16.500 +That was at least 10 years ago the Fellows Program started. + +00:35:16.500 --> 00:35:17.960 +I think it's been an enormous success. + +00:35:18.520 --> 00:35:18.880 +Yeah, absolutely. + +00:35:19.320 --> 00:35:27.200 +Another thing that I think is really interesting, it's a little bit newer, but is the Django Knots program, right? + +00:35:27.620 --> 00:35:29.980 +And Django Girls as well, shout out to them. + +00:35:30.500 --> 00:35:37.160 +You know, so many people I've talked to are like, I got into Python because I wanted to do Django, which for you all is probably a little bit mind-blowing as well. + +00:35:37.320 --> 00:35:39.300 +Like, they didn't use Django because they were Python, + +00:35:39.540 --> 00:35:40.300 +but the other way around. + +00:35:40.360 --> 00:35:43.380 +The first time I met somebody who said, oh yeah, I got into Python because of Django. + +00:35:43.700 --> 00:35:45.080 +And I was absolutely bowled away. + +00:35:45.260 --> 00:35:50.320 +I always assumed you'd have to be a proficient Python programmer before you could start picking up something like Django. + +00:35:50.380 --> 00:35:51.360 +It turns out it was the other way around. + +00:35:51.580 --> 00:35:56.240 +People would get in from, Because partly because the Django documentation was so good. + +00:35:56.550 --> 00:36:02.540 +But yeah, that's so many people got their start in the Python community as web developers who dabbled with Django and then got into Python from there. + +00:36:02.720 --> 00:36:02.940 +Yeah. + +00:36:03.200 --> 00:36:05.640 +Kenneth, love it on the audience because I learned Python through Django. + +00:36:06.020 --> 00:36:06.480 +Very cool. + +00:36:06.720 --> 00:36:09.640 +When I chaired DjangoCon US, I'd always ask the audience. + +00:36:09.960 --> 00:36:11.080 +For one, like I was always a nervous speaker. + +00:36:11.370 --> 00:36:14.480 +And so anytime you ask the audience to do something, it like takes my nerves away. + +00:36:14.900 --> 00:36:17.820 +But I'd always ask who learned Python by learning Django. + +00:36:18.110 --> 00:36:20.320 +And over half the room every year would always raise their hands up. + +00:36:20.480 --> 00:36:22.920 +So it was a heck of a gateway to learn Python. + +00:36:23.040 --> 00:36:44.000 +on. Also, we just hired our third fellow Jacob Walls last week, who I think starts this next Monday. So Django's up to three fellows now. And it's just been, the load is just incredible. So trying to keep up. I think our existing fellows have over a thousand tickets that they're trying to sort through. And if people are watching and want to help fund, contact Tebow and I, + +00:36:44.010 --> 00:36:47.440 +and we will definitely help you in the USF to support more fellows. + +00:36:47.880 --> 00:36:51.380 +We'll give you our annual reports, which details everything the foundation does, + +00:36:51.680 --> 00:36:52.500 +And more on the other side. + +00:36:53.880 --> 00:37:04.740 +Well, I just want to make one point on that because, I mean, T-Bone and Jeff, you know now, but the finances for a long time, like when I was treasurer in 2020 for three years, the total budget was under $200,000. + +00:37:04.940 --> 00:37:06.540 +And that was almost all fellows. + +00:37:07.000 --> 00:37:11.180 +So like the total budget for everything, including grants, legal stuff, hosting. + +00:37:11.680 --> 00:37:17.140 +I think now maybe it's $250,000, $300,000, but it's, you know, to your point, Simon, it's not funded by a corporation. + +00:37:17.620 --> 00:37:18.320 +It's tiny. + +00:37:19.100 --> 00:37:22.600 +And people don't realize that aside from the fellows, everything else is volunteers, right? + +00:37:22.760 --> 00:37:23.420 +Even the conferences. + +00:37:23.760 --> 00:37:28.380 +It's not, you know, PyCon, US, EuroPython have a lot of volunteers, but they also have paid employees. + +00:37:28.620 --> 00:37:30.800 +Like Django Cons are all volunteers. + +00:37:31.260 --> 00:37:32.200 +Yeah, they're all volunteer based. + +00:37:32.460 --> 00:37:35.560 +The PSF runs a staff of about, I used to be on the PSF board. + +00:37:35.680 --> 00:37:36.320 +Simon's on the board. + +00:37:36.560 --> 00:37:36.980 +I'm on the board. + +00:37:37.440 --> 00:37:37.580 +Yeah. + +00:37:38.120 --> 00:37:43.120 +Last time I saw, they probably have about, what, 2 million in salaries a year because of so many people. + +00:37:43.480 --> 00:37:49.880 +You've got like five or six developer in residence, at least probably what, like 12 to 13 people, maybe 14 people on staff. + +00:37:50.140 --> 00:37:50.680 +That's about right. + +00:37:50.820 --> 00:37:50.940 +Yeah. + +00:37:52.160 --> 00:38:05.700 +It's interesting how much the PSF is an accounting organization because it turns out accounting, especially if you're a nonprofit with like with a whole bunch of organizations who use the PSF as their fiscal sponsor to help them with their nonprofit accounting. + +00:38:06.020 --> 00:38:11.300 +There's a lot of there's a lot of accounting and logistical works that goes into an organization like the PSF. + +00:38:11.520 --> 00:38:16.880 +And PyCon US is run by the PSF as well, which is a 2,500 person conference these days. + +00:38:17.100 --> 00:38:18.580 +That gets very complicated. + +00:38:19.000 --> 00:38:22.820 +Yeah, so the DSF runs on shoestring size budget compared to the PSF. + +00:38:23.120 --> 00:38:25.600 +But Django is a pretty good amount of the community, though. + +00:38:25.710 --> 00:38:28.780 +So that's kind of one of the things we're doing is by getting that third fellow. + +00:38:29.700 --> 00:38:36.540 +And part of it, too, is we've ran, not to nerd out on expenses for a while, but the DSF for a long time has ran with over $200,000 in the bank. + +00:38:36.750 --> 00:38:38.720 +And to me, that's just not doing much with our money. + +00:38:38.980 --> 00:38:41.600 +So I know that we're also doing investments and stuff as well. + +00:38:42.000 --> 00:38:46.520 +But I think getting that third fellow was something that was just long overdue for the number of people that we have. + +00:38:46.760 --> 00:38:54.880 +If one fellow goes on vacation for a while and leave, it's just too much for one person to keep the foundation, keep security releases, releases going. + +00:38:55.200 --> 00:38:58.360 +So it was kind of long overdue, but I'm happy we're able to pull it off this year. + +00:38:58.680 --> 00:38:59.260 +Yeah, that's great. + +00:38:59.660 --> 00:39:06.060 +My favorite talk at JaggerCon last year was Jacob Kaplan Mosk gave a keynote about what the DSF would do with a million dollars. + +00:39:06.420 --> 00:39:09.940 +Like if the DSF could go up to a million dollars a year, what would they spend that money on? + +00:39:10.140 --> 00:39:13.320 +As also a pitch for why you should be sponsoring the DSF. + +00:39:13.600 --> 00:39:18.900 +I recommend people watch that if you want to get a feel for how that organization was shaped at least sort of 10 months ago. + +00:39:19.240 --> 00:39:30.500 +I think there's a huge impact that just having one or two people whose full-time job is to go into the corner, the dark corners of some open source project that's just not that interesting. + +00:39:30.720 --> 00:39:39.040 +Nobody really wants to go mess with, but just start hacking away on those and, you know, really putting the polish on stuff that is not as exciting as other pieces. + +00:39:39.200 --> 00:39:55.420 +And so there's not, you know, you said it's not that big relative to the PSF, but put the PSF aside, any other open source Python project, it's a really huge pool that you're working with relative to how many other projects even have fellows. + +00:39:56.000 --> 00:39:56.700 +Not many, if any. + +00:39:57.140 --> 00:39:57.200 +Yeah. + +00:39:57.480 --> 00:39:58.080 +Super, super cool. + +00:39:58.540 --> 00:40:02.040 +So let's talk about some of the more recent things. + +00:40:02.370 --> 00:40:11.080 +I know that at least personally, when the Instagram keynote came out a couple of years ago at PyCon, or maybe more than a couple, it was 2017, I think. + +00:40:11.420 --> 00:40:19.400 +But around that timeframe, just realizing how big some of the deployments of Django were, how much traffic people were serving through them. + +00:40:20.000 --> 00:40:23.120 +When it first came out, you probably didn't think on that scale. + +00:40:23.980 --> 00:40:26.120 +No, Instagram, Pinterest, NASA. + +00:40:26.860 --> 00:40:40.080 +Yeah, no, that was... The goal was to build newspaper websites, right? We wanted to be able to build... I loved Django's original tagline was "Web development on journalism deadlines". That's always been my favorite description of the project. + +00:40:41.060 --> 00:40:45.340 +Nice. Who else wants to chime in on this idea? Like this is just... It blew my mind. I'm like, wow. + +00:40:45.440 --> 00:40:59.000 +One thing I will say is a thread, Instagram's Twitter clone came out what last year or the year before. That's a Django app. I met one of the engineers of mine. It's just a classic Django app that Facebook launched just a few years ago. + +00:40:59.260 --> 00:41:04.320 +I believe technically they have their own fork. They didn't just download Django 4.0. + +00:41:04.920 --> 00:41:08.700 +I feel like at that scale, if you haven't got your own fork, I'd be very surprised. + +00:41:08.770 --> 00:41:13.420 +But yeah. For me, it runs on center. Sorry. I think I'm a brief + +00:41:13.540 --> 00:41:14.500 +I mean, one interesting thing. + +00:41:14.600 --> 00:41:16.000 +500 milliseconds behind all of you. + +00:41:16.600 --> 00:41:19.960 +For me, as far as scale, there's also the number of projects out there. + +00:41:20.300 --> 00:41:23.540 +I've been doing the counting to figure out our impacts. + +00:41:23.800 --> 00:41:30.240 +And I think we're at around the 500,000 live Django sites and apps out there from the numbers I could find. + +00:41:30.520 --> 00:41:33.980 +And also specific types of outreach. + +00:41:34.220 --> 00:41:41.260 +So governments, for example, seeing how many central governments of all the countries around the world basically use it. + +00:41:41.460 --> 00:41:43.260 +And when I say use it, it's not just like the one project. + +00:41:43.540 --> 00:41:46.960 +They have teams whose role is just to manage 10, 20, 50. + +00:41:47.400 --> 00:41:54.480 +I think right now, Brazil, of all countries, I don't have a connection there, had like 84 sites I could detect in a single environment using Django. + +00:41:54.740 --> 00:41:56.800 +Just imagine that times all the countries. + +00:41:57.260 --> 00:41:57.480 +Wow. + +00:41:58.020 --> 00:42:02.620 +You know, one of the things that's always fun for me to do is to go to the GitHub repo, right? + +00:42:02.880 --> 00:42:07.080 +And I've got Django with 85,000 stars and 32,000 forks. + +00:42:07.380 --> 00:42:08.300 +That's a really interesting number. + +00:42:08.380 --> 00:42:12.700 +That's way more forks to stars than an average project, by the way. + +00:42:13.040 --> 00:42:23.280 +But if you go down the side here, you can see they now track how many contributors and used by 1.997 million projects. + +00:42:23.840 --> 00:42:24.560 +That is a lot. + +00:42:25.000 --> 00:42:26.060 +Now, that's just on GitHub, right? + +00:42:26.140 --> 00:42:29.280 +That are public repositories that we can know about and stuff. + +00:42:29.560 --> 00:42:33.140 +And used by could be a student project, not a government entity. + +00:42:33.720 --> 00:42:34.960 +But still, that's pretty wild. + +00:42:35.260 --> 00:42:35.740 +Very impressive. + +00:42:35.820 --> 00:42:38.680 +Let's talk about some of the more recent transitions. + +00:42:39.060 --> 00:42:43.800 +I mean, I feel like the speed of major version releases is picking up. + +00:42:44.000 --> 00:42:47.220 +You know, we had the first three years, 2005 to 2008 for one. + +00:42:47.440 --> 00:42:51.280 +Then it was like one for a while, and then two, and then three, then four, then five. + +00:42:51.800 --> 00:42:52.880 +You guys are maybe working on six. + +00:42:52.920 --> 00:42:53.280 +I don't know. + +00:42:53.880 --> 00:42:57.880 +And also adopting some of the lot of changes coming in Python, right? + +00:42:57.980 --> 00:43:00.220 +Like async and await and that kind of stuff. + +00:43:00.840 --> 00:43:01.240 +How has that been? + +00:43:01.240 --> 00:43:01.580 +How did it start? + +00:43:01.740 --> 00:43:06.280 +In some ways, it's really tricky because people really love Django as it is. + +00:43:06.760 --> 00:43:14.680 +And from that standpoint, you have to make sure that such a big group of people agree on what exactly the future of Python means. + +00:43:15.000 --> 00:43:17.660 +So, you know, definitely mentioned AsyncAway. It's one of the big ones. + +00:43:18.140 --> 00:43:20.560 +API, is it in Django Core? Is it not? + +00:43:21.130 --> 00:43:22.720 +Mobile web, how much we get involved with that? + +00:43:23.040 --> 00:43:27.580 +These days, it's like the HTMLX that people really want to see Django support well. + +00:43:27.860 --> 00:43:31.060 +And I guess also AI to some extent. I don't know if we want to get there. + +00:43:31.140 --> 00:43:41.560 +But I guess for us, I think it's also, if it's not in core, we definitely have so many packages out there that we just try and curate a bit more these days, make sure they work well together. + +00:43:42.020 --> 00:43:43.020 +So that also helps a lot. + +00:43:43.240 --> 00:43:43.340 +Yeah. + +00:43:43.600 --> 00:43:45.840 +I think the package curation is really important. + +00:43:46.320 --> 00:43:54.860 +I would actually like to see Python in the standard library smaller and then recommend more stuff to keep the flexibility open so they could move faster, focus more. + +00:43:55.200 --> 00:44:05.120 +So what I find tricky about this one, Django has always been a batteries-included framework, And that's because in 2004, 2005, PyPI and PIPA didn't really exist. + +00:44:05.440 --> 00:44:12.060 +So if you weren't batteries included, then people had to, it was very high friction for people to glue together the pieces that they needed. + +00:44:12.500 --> 00:44:16.200 +And of course, today, I feel like Python packaging is incredibly good shape. + +00:44:16.400 --> 00:44:20.720 +Like you could ship a web framework today that was entirely dependent on other things. + +00:44:21.020 --> 00:45:21.920 +So then the value from the Django thing comes down more to everything being, that there's a Django promise that the stuff in Django core will continue to work. You're not going to have a situation where Python 3.15 comes out and Django templates stop working, but the ORM still works because everything is maintained in the one place by that single team. And I love that. I love that you just know that all of the pieces of Django are going to continue to work together. And the moment you add external packages, there's that little bit of risk. What if that external package that you're using becomes unmaintained in the future? So I'd quite like to see just a little of a sort of adoption of crucial packages under the Django software foundation, but not as part of Django itself. I want separate packages that maintain that promise to be maintained. Because we have fellows, which means that we can actually do that kind of work. But it's also a very big commitment to take on. It's an interesting pattern that I'd like to see explored a little bit more, + +00:45:21.920 --> 00:45:26.380 +I think. Yeah, it's easy to forget that it wasn't always there. Sorry, Will, go ahead. + +00:45:26.540 --> 00:45:52.500 +I was just going to ask Adrian and Simon. I mean, even today, batteries included web frameworks are not that common. It's still the trend, more micro frameworks. I'm curious what your take is on that. I mean, like FastAPI, right? That's a lightweight framework. It seems, I'm always shocked there aren't more batteries included frameworks in PICA language. Maybe it's too difficult to do, but micro frameworks seem more common. There's sort of only one batteries included option for programming language. + +00:45:52.900 --> 00:45:59.580 +I feel like the sales pitch, like you get to pick the best pieces for the components for your project, always sounds great. + +00:46:00.260 --> 00:46:03.740 +Obviously, you don't want one thing that dictates what you're going to use for all of the different things. + +00:46:03.900 --> 00:46:06.300 +You'd much rather pick and choose the things that are most valuable. + +00:46:06.740 --> 00:46:14.500 +But the trade-off that people don't talk about much is the fact that now every single one of those selections is getting you further away from that sort of community standards. + +00:46:14.820 --> 00:46:17.560 +There's more risk of future maintenance problems. + +00:46:17.760 --> 00:46:20.000 +There's more work that you have to do to wire things together. + +00:46:20.240 --> 00:46:25.760 +But I feel like the biggest innovation in the open source world in the past 15 years has been packaging. + +00:46:25.960 --> 00:46:31.400 +Like every programming language has a PyPI style packaging solution of some sort. + +00:46:31.960 --> 00:46:37.180 +You've got cargo and you've got the Ruby gems and all of those kinds of things. + +00:46:37.520 --> 00:46:39.860 +And that pattern mostly works really, really well. + +00:46:40.220 --> 00:46:45.780 +But I just keep on coming back to that feeling that with Django, I know it's all going to work. + +00:46:45.960 --> 00:46:47.740 +And I know it's going to work far into the future. + +00:46:47.820 --> 00:46:49.480 +And that feels really valuable to me. + +00:46:49.740 --> 00:46:50.140 +Good, Adrian. + +00:46:50.220 --> 00:46:59.620 +For my answer, I think I'd have to evaluate it on a case-by-case basis, but the way I would evaluate it is to look at the origin of the project. + +00:47:00.090 --> 00:47:07.080 +Was it extracted from something from the real world or was it built either as an academic or for fun exercise? + +00:47:07.520 --> 00:47:10.320 +What was the reason for its existence? + +00:47:10.980 --> 00:47:15.260 +And generally, I trust stuff that was extracted from real production sites more. + +00:47:15.680 --> 00:47:17.080 +That's just my own preference. + +00:47:17.660 --> 00:47:44.000 +It's also sort of to flip it around. I never really cared whether someone used Django or not. I would evaluate you on your product, not based on like the web framework you use. Who cares really what web framework you use? And I think that kind of ethos may be seeped into Django at some point unintentionally. I mean, I never intentionally sort of made that, had that philosophy communicated, but yeah. + +00:47:44.080 --> 00:47:49.720 +I always felt like one of the most interesting difference between Rails and Django is that Rails was extracted from Basecamp. + +00:47:49.830 --> 00:47:51.760 +They built Basecamp and they pulled the framework out of it. + +00:47:52.080 --> 00:47:57.320 +I feel like Django was kind of reverse extracted from Lawrence.com because we had Lawrence.com. + +00:47:57.350 --> 00:47:58.360 +It was PHP and MySQL. + +00:47:58.590 --> 00:48:03.960 +And the goal with Django was always to get it to a point where it was good enough that it could replace the PHP in Lawrence.com. + +00:48:04.200 --> 00:48:09.760 +So everything that we were building was always like, what would Lawrence.com need in this new thing for us to replace it? + +00:48:10.180 --> 00:48:13.160 +And the end result was a framework that you could build Lawrence.com in. + +00:48:13.300 --> 00:48:15.320 +Lawrence.com was the entertainment website for the newspaper. + +00:48:15.860 --> 00:48:19.860 +It's the best local entertainment website for any city that has ever existed. + +00:48:20.400 --> 00:48:22.400 +I will very confidently state that. + +00:48:22.560 --> 00:48:28.900 +I've never lived in a city with a website as good for figuring out what the things to do in a town are. + +00:48:29.050 --> 00:48:36.720 +And that was Adrian's biggest project, Lawrence Journal World, was always Lawrence.com and evolving Django up to the point where it could power that site. + +00:48:37.140 --> 00:48:38.100 +It's what got me here. + +00:48:38.400 --> 00:48:45.180 +Like I saw that site living 200 miles away and went to San Francisco and kind of looked at Y Combinator and was like, you know, this is only 200 miles. + +00:48:45.360 --> 00:48:47.740 +Like they've got something good going here and that site especially. + +00:48:48.960 --> 00:48:56.420 +I think to a Django's batteries, Django, I think 20 years ago, Django, there was a lot of things that you had to do the Django way. + +00:48:56.800 --> 00:49:00.800 +And I think there's been so much concerted effort, especially the first 12 to 14 years. + +00:49:01.220 --> 00:49:05.480 +It's so much as pluggable that like if you don't like the batteries, I don't know what you're missing. + +00:49:05.660 --> 00:49:07.240 +Like Django is still a really good core. + +00:49:07.680 --> 00:49:09.200 +Don't use the admin if you don't want it. + +00:49:09.680 --> 00:49:16.120 +I think most of the problems I see, the foundation and the community, I think there's like a lot of lore that comes with Django. + +00:49:16.570 --> 00:49:21.940 +And it's like lore of like, maybe it's harder to change certain things on the website than it really needs to be or is. + +00:49:22.240 --> 00:49:32.100 +And I think this last year, we've definitely pushed back on, like, if you do a search on the Django docs now and you look for REST, you wouldn't know that Django is a REST framework or has the ability to do that by looking at the docs. + +00:49:32.560 --> 00:49:40.240 +And so these were just technical challenges of like certain steering council members have written articles and stuff, blog posts saying like, you can do rest with Django. + +00:49:40.640 --> 00:49:42.500 +There's some good third party packages to do it. + +00:49:42.770 --> 00:49:47.300 +In the past, we've kind of been like apprehensive to want to recommend third party packages through docs. + +00:49:47.840 --> 00:49:49.040 +I think that's just one of those. + +00:49:49.130 --> 00:49:50.620 +It sounds weird when I say it out loud. + +00:49:50.890 --> 00:49:51.560 +At least it does to me. + +00:49:51.940 --> 00:49:57.420 +And I think once we have these discussions, then we start like realizing that, well, we have a website that everybody comes to. + +00:49:57.820 --> 00:50:02.720 +There's very specifically doc sections and there's blog, but there's no search that pulls stuff in together. + +00:50:03.050 --> 00:50:06.780 +And so the steering council and different members have been trying to make search work across everything. + +00:50:07.150 --> 00:50:15.680 +And then that way you can get hits from rest and you can get hits for third party packages that the steering council recommends just because they're good packages everybody should use. + +00:50:16.020 --> 00:50:21.380 +And so I think some of the lore and some of the, hey, these things have been done this certain way doesn't mean we want to keep doing them. + +00:50:21.680 --> 00:50:25.900 +It's just probably somebody had an afternoon or a night and they made search work a particular way. + +00:50:26.170 --> 00:50:27.400 +And that was really good for the docs. + +00:50:27.800 --> 00:50:30.060 +but like sometime look up REST and look at PHP. + +00:50:30.480 --> 00:50:33.580 +There's actually more hits for PHP on the Django website than there are REST + +00:50:33.900 --> 00:50:36.680 +unless you get into like Sphinx, you know, restructured text. + +00:50:37.020 --> 00:50:49.540 +So, but those are challenges that until we talk about and we realize, and I think like that's the advent of a FastAPI kind of taking off is I think like Django didn't have a great REST story by looking at the docs and looking at information. + +00:50:50.100 --> 00:50:54.740 +At PyCon US this year, I had dozens of people ask me, how do I use the Django ORM from FastAPI? + +00:50:54.900 --> 00:50:55.780 +And it's like, why? + +00:50:55.940 --> 00:50:56.560 +You don't have to. + +00:50:56.780 --> 00:50:59.700 +Django is async enough that you don't need to do that. + +00:50:59.840 --> 00:51:00.300 +That's hard. + +00:51:00.430 --> 00:51:01.600 +And that kind of sucks to do. + +00:51:01.780 --> 00:51:02.660 +So why? + +00:51:03.420 --> 00:51:06.320 +And those are things we're focusing and talking about internally too. + +00:51:06.620 --> 00:51:06.800 +Yeah. + +00:51:07.460 --> 00:51:22.860 +Talking about search and search across different things, somewhat directly to you, Jeff, but all of you, and maybe Simon as well, is like, what about just like chat.djangoproject.com and do some rag on it and let people have a conversation with Django? + +00:51:23.130 --> 00:51:23.560 +You know what? + +00:51:23.840 --> 00:51:40.120 +We don't even need it because Django is stable enough that every chatbot out there knows Django inside out already. One of the great benefits of having great documentation and lots of examples for 20 years that is stable is, yeah, you can, GPT-5 will spit out a full Django project and just + +00:51:40.130 --> 00:51:55.000 +get it all right. So yeah, I completely disagree. I think we need a big sponsor who can come in, whether it's Anthropic or OpenAI or somebody else. We need a big sponsor, a title sponsor, even in fact that can help give the money for this and donate the chat. + +00:51:55.440 --> 00:52:01.560 +Because even though you're right that it isn't ingrained, you still have that delay between because Django kicks out a new version or two a year. + +00:52:01.960 --> 00:52:04.320 +And so there's always that lag of nine months or a year. + +00:52:04.520 --> 00:52:08.860 +And most of it recommends the most popular version, not the latest version or something like that. + +00:52:08.940 --> 00:52:09.000 +Right. + +00:52:09.520 --> 00:52:09.820 +Exactly. + +00:52:09.900 --> 00:52:10.040 +Yeah. + +00:52:10.280 --> 00:52:14.440 +Because I see a lot of demos where they're actually building Django apps using these, these tools. + +00:52:14.640 --> 00:52:18.480 +And so we're selling AI by virtue of Django and Django's docs being so good. + +00:52:18.840 --> 00:52:24.880 +Or occasionally one of these things will say that Sound Slice has a feature that it doesn't have, and then Adrian has to build the feature. + +00:52:25.340 --> 00:52:25.720 +Oh, God. + +00:52:26.920 --> 00:52:29.380 +Oh, you guys, in the chat, it's coming on here. + +00:52:29.400 --> 00:52:32.140 +We have Frank Wasson, but our chatbot can have a cool personality. + +00:52:32.340 --> 00:52:39.360 +And Kenneth loves to insert a random name every once in a while or something, you know, pony style. + +00:52:40.640 --> 00:52:43.240 +Adrian, what is the story behind that? + +00:52:43.500 --> 00:52:44.800 +Oh, it has nothing to do with Django. + +00:52:46.200 --> 00:52:56.120 +So yeah, just the 30 second version is ChatGPT was telling people about my site, my company sounds like having a feature that doesn't actually exist. + +00:52:56.420 --> 00:52:58.360 +And it was happening like dozens of times per day. + +00:52:58.900 --> 00:53:03.340 +And so we had this inbound set of users who had a wrong expectation. + +00:53:03.880 --> 00:53:17.140 +So we ended up just writing the feature to appease the ChatGPT gods, which I think is the first time that at least to my knowledge of uh yeah product decisions being influenced by + +00:53:17.500 --> 00:53:22.440 +misinformation from llms i just love that it's easier to just build the feature than to get the + +00:53:22.640 --> 00:53:26.800 +misinformation corrected like there is no he's got to backfill it he's got to backfill it + +00:53:29.200 --> 00:54:24.840 +incredible all right um that's crazy i let's let's do two more quick topics like somewhat lightning round, we have about six minutes left for this show. So one, there's been, you guys mentioned it a couple of times, and it's also just kind of going back and forth in the audience here, that the whole REST idea. So we've got, you know, Tom Christie doing amazing work with Django REST framework. We've also, I think, you know, people who do talk about FastAPI and how do I use the ORM with it, like Django Ninja, which is really, really close to FastAPI and so on. That's one side, that's a really positive side of things with Django. And the other side I think is, you know, I'm not a super fan of, but it's, it's something a lot of people do is, well, can we just make the backend a dumb crud API thing? And let me just write everything in JavaScript on the front end, which I'm not as much of a fan of. I think a lot of people choose that because they've been told that's how it's supposed to be. I don't think it needs to be that way. + +00:54:25.260 --> 00:54:37.540 +So what do you all feel about that? And sort of the, how this has an impact for Django, Because if people just see the backend as an API that's just a dumb thing, then a lot of the value that Django brings, I think, starts to vanish. + +00:54:38.480 --> 00:54:39.940 +You've got things like HTML and so on. + +00:54:40.020 --> 00:54:41.140 +So, yeah, what are your thoughts on all this? + +00:54:41.340 --> 00:54:53.140 +I think one of the hardest problems in computer science right now is talking a front-end team out of building everything as a pure JavaScript app, especially if that's developers who onboarded in web development over the past five to ten years. + +00:54:54.320 --> 00:55:01.860 +It's infuriating because my impression is that it just takes longer and you get more bugs and it's harder to build useful software. + +00:55:02.140 --> 00:55:06.720 +The one thing I will say is that HTMX has been a huge breath of fresh air on this front. + +00:55:07.880 --> 00:55:11.020 +HTMX gives you your single page applications. + +00:55:11.520 --> 00:55:14.160 +It's all stuff that's rendered server-side in templates. + +00:55:14.370 --> 00:55:15.700 +It fits really well with Django. + +00:55:15.880 --> 00:55:19.020 +There's a great Django extension for looking into HTMX. + +00:55:19.380 --> 00:55:28.520 +And I feel like that, the pendulum has just started swinging slightly back away from the everything must be rendered in JavaScript thing, partly because of the work that the HTMX people have been doing. + +00:55:28.680 --> 00:55:30.480 +So that I felt pretty good about. + +00:55:30.980 --> 00:55:39.500 +I'm going to point to Adrian now because SoundSlice is a pure JavaScript front-end app, and it should be based on what it does, right? + +00:55:39.780 --> 00:55:40.940 +Yeah, it's interactive sheet music. + +00:55:41.200 --> 00:55:47.800 +So it's sheet music that's rendered entirely in your browser, responsive, blah, blah, blah, with an editor, a lot of crazy JavaScript. + +00:55:48.320 --> 00:55:54.820 +But yeah, I mean, this is a whole other podcast episode about the horribleness of modern web development. + +00:55:55.160 --> 00:56:00.660 +I feel very lucky that I came up 20 years ago when before it got so insane. + +00:56:02.020 --> 00:56:12.680 +But yeah, I have sort of a related comment, which is I would like to see Django be a stronger advocate for the web in general. + +00:56:13.120 --> 00:56:17.760 +It doesn't matter if you're talking to developers or people using tech products. + +00:56:18.280 --> 00:56:22.600 +the expectation has been shifting away from the web to the phone for many, many reasons. + +00:56:23.640 --> 00:56:25.320 +And I don't like that. + +00:56:25.500 --> 00:56:29.320 +I'm pretty sure that most, if not all of the people on this chat don't like that either. + +00:56:29.700 --> 00:56:34.060 +And I don't really see a lot of effort being pushed in that direction. + +00:56:34.560 --> 00:56:38.720 +So just to sprinkle a little idea here. + +00:56:39.120 --> 00:56:45.380 +I personally wish that progressive web apps were more front and center, giving you an option at least like, okay, you have to be on the phone, + +00:56:45.600 --> 00:56:47.600 +but could we package it up a little more? + +00:56:47.880 --> 00:56:51.740 +There's a really exciting development there that's finally beginning to be rolled out. + +00:56:51.860 --> 00:57:05.700 +There's this thing called view transitions in CSS, where you can have a little bit of CSS that says, "And when the user goes from one page to another, keep the navigation bar in the same place and do a quick blur effect for this piece here." It means that... + +00:57:05.810 --> 00:57:07.480 +And it's been supported in Chrome for a few years. + +00:57:07.640 --> 00:57:08.880 +It's finally rolling out through the browsers. + +00:57:09.190 --> 00:57:14.440 +It means you can build something that feels like a slick app with page transitions and so forth, but it's not. + +00:57:14.720 --> 00:57:16.300 +It's just boring old HTML and CSS. + +00:57:16.780 --> 00:57:18.200 +No JavaScript required at all. + +00:57:18.580 --> 00:57:20.480 +Gives you that experience much faster to load. + +00:57:20.790 --> 00:57:24.080 +And I have been itching to see this start showing up. + +00:57:24.090 --> 00:57:29.880 +And I think it's finally this year, I feel, is the year where the browser support is good enough that we can start really depending on this. + +00:57:30.240 --> 00:57:36.920 +And yeah, because it gives you that app-like experience on a boring old website, which is, I feel like, that's the thing that we all want. + +00:57:37.160 --> 00:57:41.060 +So back to part of Adrian's piece, and sorry, Tebow, real quick. + +00:57:41.480 --> 00:57:43.900 +My second thing I have to pitch is because of what Adrian was saying. + +00:57:44.600 --> 00:57:51.340 +I think because systematically we, you know, we are really lucky to be around for 20 years because we've had nothing but volunteer leadership in years. + +00:57:51.960 --> 00:57:55.320 +One thing we want to do is find an executive director, somebody that can run the foundation. + +00:57:55.670 --> 00:57:57.780 +The biggest reason goes back to Adrian's point. + +00:57:57.970 --> 00:58:06.880 +I think that we sometimes don't get a seat at the table when it comes to web standards, when it comes to these conversations, is because it's the cycling of Thibaut being the president now. + +00:58:06.890 --> 00:58:07.800 +It was Frank before. + +00:58:07.850 --> 00:58:08.980 +It was Anna from Africa. + +00:58:09.220 --> 00:58:11.720 +We've had many really good presidents over the years. + +00:58:11.980 --> 00:58:16.340 +But it's really hard to work with foundations and nonprofits when you don't have a set person. + +00:58:16.700 --> 00:58:19.120 +Even our fellows seem to rotate every three to four years. + +00:58:19.400 --> 00:58:22.980 +And so that's part of like Django growing up, which is something I would love to see us do. + +00:58:23.340 --> 00:58:26.120 +And then Thibault, I love the fact that this started off in the middle of like the day. + +00:58:26.320 --> 00:58:28.600 +And now it is like perfectly cycled to like dark. + +00:58:28.600 --> 00:58:30.700 +Yeah, I guess it was high and curve you as president. + +00:58:30.840 --> 00:58:31.220 +Go ahead. + +00:58:31.960 --> 00:58:35.680 +To me, there's a developer experience and like keeping devs happy. + +00:58:36.040 --> 00:58:46.680 +So, you know, when we say our people go to React or whichever like JS heavy, It's also because they think they need it to build whatever complexity that they are after, but also because they think it has good DX, I assume. + +00:58:47.340 --> 00:58:57.100 +So when we say like the REST framework or the Django Ninja, to me, it's this as well, that we have to be aware of what's the evolving standards for what devs want to do with their web. + +00:58:57.340 --> 00:59:05.900 +And I guess how we can make Django meet that, whether it's better types, better CLI, better packaging, more Rust, maybe, whichever. + +00:59:06.500 --> 00:59:12.140 +Will, do you want to get a quick word in on this everything's a front-end thing now before we move on to the last bit? + +00:59:12.600 --> 00:59:16.580 +Oh, sure. I'm just going to say, and this probably goes back to Adrian and Simon, like all of them, + +00:59:16.960 --> 00:59:25.340 +it seems very wise to keep the front-end light because there was certainly 10 years ago a lot of pressure to integrate React or Vue or Angular or something. + +00:59:25.500 --> 00:59:31.440 +But the fact that it's remained just the Django templates means HTMX can rise and we can have more innovation. + +00:59:32.040 --> 00:59:37.240 +And just shout out, Carson Gross of HTMX is giving the keynote at DjangoCon US in three weeks. + +00:59:37.650 --> 00:59:43.240 +So even though he's not a Django developer, he's in fact more of a, he's a professor who does Java and obviously JavaScript. + +00:59:43.700 --> 00:59:48.880 +It still slides into Django in a way that if Django had built out the front end story, we wouldn't be able to. + +00:59:49.120 --> 00:59:50.580 +I love how that also fits. + +00:59:50.840 --> 00:59:59.720 +DjangoCon has had a long tradition of having a keynote from somebody from outside the community, which started in 2008 with Cal Henderson at DjangoCon over in Mountain View. + +01:00:00.840 --> 01:00:01.920 +Was that the why I hate Django? + +01:00:02.130 --> 01:00:05.640 +Because there was that whole history of, I feel like there should be another one of those, you know? + +01:00:06.059 --> 01:00:06.780 +Absolutely, yeah. + +01:00:07.620 --> 01:00:08.240 +But that's great. + +01:00:08.360 --> 01:00:10.720 +That's a fantastic keynote for DjangoCon, I think. + +01:00:10.960 --> 01:00:12.920 +Yeah, and coming up in Chicago, right? + +01:00:13.190 --> 01:00:13.560 +Pretty soon. + +01:00:13.760 --> 01:00:16.760 +So if you are in or would like to be, go check it out, right? + +01:00:16.860 --> 01:00:18.600 +That's part of that community thing I talked about. + +01:00:19.020 --> 01:00:26.020 +And people listening who are not watching, listening, not watching, they don't realize that you did this outside. + +01:00:26.760 --> 01:00:32.440 +And there's a great comment from Kojo saying, It's really cool that you're using in real life dark mode. + +01:00:32.680 --> 01:00:32.960 +It's great. + +01:00:34.120 --> 01:00:35.980 +All right, let's close it out with this, everyone. + +01:00:36.440 --> 01:00:37.320 +We're pretty much out of time. + +01:00:37.820 --> 01:00:40.580 +20 years, you know, amazing, amazing stuff. + +01:00:42.359 --> 01:00:45.120 +We're celebrating Django having its birthday, right? + +01:00:45.360 --> 01:00:46.760 +But it's going to live on for a long time. + +01:00:46.860 --> 01:00:48.160 +There's a ton of code. + +01:00:48.280 --> 01:00:50.400 +There's a ton of things people are doing with Django. + +01:00:51.500 --> 01:00:53.560 +Even if you guys step away at some point, it's still going to be going. + +01:00:54.030 --> 01:00:57.460 +So give me your, you know, going around and starting with Will, I guess. + +01:00:57.690 --> 01:00:59.680 +Give me your forward-looking thoughts on Django. + +01:00:59.700 --> 01:01:01.120 +over the next five, 10 years? + +01:01:02.140 --> 01:01:14.600 +Well, I think it's integrating with AI, broadly defined, however that means, because in a way related to Adrian's point, like the boring old web has never been more relevant, even though we're focused on chatbots right now, you still need all the other things. + +01:01:15.010 --> 01:01:16.520 +And Django can integrate into that. + +01:01:16.530 --> 01:01:22.620 +And I think it's always kind of a marketing problem for Django because we don't have one person in charge or a company behind it. + +01:01:22.740 --> 01:01:26.620 +So telling the story of we've been here, there's still double digit PRs. + +01:01:26.840 --> 01:01:28.460 +It has async in many areas. + +01:01:28.980 --> 01:01:34.180 +And I worry that the next generation of web developers just skip it because they don't know what it already has. + +01:01:34.570 --> 01:01:37.940 +And they have this presumption that it's too hard to learn, which it's not. + +01:01:38.240 --> 01:01:38.660 +Yeah, absolutely. + +01:01:39.220 --> 01:01:39.520 +Is it me? + +01:01:40.140 --> 01:01:40.720 +It is you. + +01:01:41.100 --> 01:01:41.640 +I don't have any. + +01:01:41.900 --> 01:01:43.200 +I hear you're boring about the web. + +01:01:43.380 --> 01:01:44.040 +What's going on now? + +01:01:44.480 --> 01:01:46.120 +Tell us about what do you see for the future? + +01:01:46.560 --> 01:01:46.980 +For the future. + +01:01:47.460 --> 01:01:55.880 +Yeah, I'm actually, perhaps ironically, like the worst person to ask because I have worked on the same app for almost 13 straight years full time now. + +01:01:57.040 --> 01:02:00.780 +I use Django every day and it does everything I need. + +01:02:01.160 --> 01:02:04.200 +So I'm very happy with the current state. + +01:02:05.020 --> 01:02:21.080 +And I think, like I said earlier, if it were up to me, the thing I would focus on is more philosophical or trying to promote the concept of the web versus non-web stuff. + +01:02:21.400 --> 01:02:23.800 +Because if the web goes away, Django goes away. + +01:02:23.920 --> 01:02:25.640 +I mean, it's inherent. + +01:02:25.720 --> 01:02:27.080 +A lot of beautiful things go away, yeah. + +01:02:27.260 --> 01:02:28.420 +A lot of beautiful things go away. + +01:02:28.840 --> 01:02:32.540 +So I think that would be my hope for the next five years. + +01:02:32.980 --> 01:02:38.700 +Yeah, I've done combat with the App Store review people, both Apple and Google, and it is not pretty. + +01:02:38.880 --> 01:02:39.780 +We do not want the web. + +01:02:40.420 --> 01:02:43.120 +There's definitely AI in there and definitely the web. + +01:02:43.580 --> 01:02:45.620 +I really like that you mentioned the view transitions. + +01:02:46.520 --> 01:03:00.520 +I think if web technologies get that much better in the next five years or so, we can match what's possible with a mobile dev, but not only match it, but in terms of accessibility and how many people you can reach with a web app. + +01:03:00.920 --> 01:03:02.540 +We can go far, far beyond. + +01:03:03.260 --> 01:03:17.960 +So I guess to me for that, the other side is having a foundation behind Django that's probably a bit more professional, that takes it a bit more seriously, that we're the people looking after the framework for millions of devs at this point. + +01:03:18.460 --> 01:03:22.380 +And as you said, the people in this room, we come and go, that's very natural. + +01:03:22.680 --> 01:03:24.480 +So just making sure that we have the pipelines. + +01:03:25.000 --> 01:03:33.920 +So hopefully five, 10 more years, you know, the Django node space you mentioned earlier, that's the people who will be the presidents and people having this podcast maybe in 10 years time. + +01:03:34.240 --> 01:03:36.220 +Yeah, absolutely. Jeff? + +01:03:36.540 --> 01:03:38.760 +I'm really curious to see what Rust's impact is. + +01:03:38.790 --> 01:03:41.460 +I think Python is just getting so much faster, so much better. + +01:03:41.800 --> 01:03:45.180 +And I know there's some really smart people working on like WebAssembly and Rust. + +01:03:45.580 --> 01:03:51.240 +And I dream of a day when a Django view can be the same thing in your browser as it is on the server. + +01:03:51.600 --> 01:03:58.760 +And I think when we look at like talking to a query set, your browser, you know, doing data exchange, serialization, JSON, something. + +01:03:59.130 --> 01:04:08.620 +I feel like in the next five years, there's this there's this opportunity that whatever Python is becoming, whatever Django is becoming that like, you know, it's less about how are we going to use React and render? + +01:04:09.000 --> 01:04:10.040 +I hate React, by the way. + +01:04:10.100 --> 01:04:11.040 +You've got to put that plug in. + +01:04:11.300 --> 01:04:18.440 +But I'm just really curious to see like how these things can kind of mold together as, you know, developer experiences have gotten a lot better because of tooling and Rust. + +01:04:18.840 --> 01:04:27.800 +There's just this whole new world that I see as far as these technologies that start to marry the two together in a way that we just haven't been able to do for a while. + +01:04:28.000 --> 01:04:29.660 +But we're starting to see glimpses of it. + +01:04:29.820 --> 01:04:30.140 +For sure. + +01:04:30.420 --> 01:04:31.680 +Simon, final word for... + +01:04:31.860 --> 01:04:35.440 +I'm going to double down a little bit on the WebAssembly stuff is so exciting. + +01:04:36.059 --> 01:04:39.800 +Pyodide, the thing that lets you run Python in your browser, is now at least 10 years old. + +01:04:39.920 --> 01:04:41.600 +It's very, very good. + +01:04:42.020 --> 01:04:43.400 +It's about a two megabyte download. + +01:04:43.610 --> 01:04:46.020 +A few years ago, I'd have said two megabyte downloads is too much. + +01:04:46.080 --> 01:04:49.220 +These days, 2 megabyte download for a web page is hardly anything at all. + +01:04:49.860 --> 01:04:54.560 +But I feel like the thing that I love most about Django these days is it is reliable. + +01:04:55.060 --> 01:04:56.340 +It's so dependable. + +01:04:56.740 --> 01:05:03.340 +Nobody ever picks Django for a project and regrets it because Django, it's mature, it's stable, it's well documented. + +01:05:03.380 --> 01:05:04.340 +It moves at a sensible place. + +01:05:04.880 --> 01:05:05.940 +Long may that continue. + +01:05:06.100 --> 01:05:08.520 +I love the idea that Django becomes... + +01:05:08.520 --> 01:05:13.140 +And Django for many years has been a great default technology for you to use as a project. + +01:05:13.640 --> 01:05:17.780 +And if we double down on that, which I think is what we're doing already, that's fantastic for me. + +01:05:17.840 --> 01:05:20.360 +I love that idea that you can always pick Django. + +01:05:20.500 --> 01:05:22.200 +It will always do what you need it to do. + +01:05:22.240 --> 01:05:26.400 +There will never be any nasty surprises or sort of regrets from working with that. + +01:05:26.660 --> 01:05:35.740 +And if we can tap into things like free-threaded Python is coming along now, which is very well, it means maybe we don't have to rewrite everything as async things just to handle large concurrency. + +01:05:35.880 --> 01:05:38.120 +There's all of that kind of stuff, the WebAssembly stuff. + +01:05:38.380 --> 01:05:41.440 +It feels like the Python ecosystem is in a really good shape. + +01:05:41.900 --> 01:05:44.800 +And Django sits very close to the center of that ecosystem. + +01:05:45.020 --> 01:05:46.660 +And I'm really excited about that. + +01:05:46.660 --> 01:05:51.760 +I love that we can continue to provide the best default option for building great things. + +01:05:52.200 --> 01:05:53.060 +Yeah, totally agree. + +01:05:53.380 --> 01:05:54.020 +All right, everyone. + +01:05:54.510 --> 01:05:55.560 +Thank you so much for being here. + +01:05:55.940 --> 01:05:56.780 +Happy birthday, Django. + +01:05:57.960 --> 01:05:58.860 +Happy birthday, Django. + +01:05:59.660 --> 01:06:00.240 +Thanks, Michael. + +01:06:01.580 --> 01:06:04.040 +This has been another episode of Talk Python To Me. + +01:06:04.860 --> 01:06:05.820 +Thank you to our sponsors. + +01:06:06.300 --> 01:06:07.500 +Be sure to check out what they're offering. + +01:06:07.640 --> 01:06:08.940 +It really helps support the show. + +01:06:09.720 --> 01:06:12.500 +This episode is brought to you entirely by Sentry. + +01:06:12.840 --> 01:06:14.980 +It's a bit of an episode takeover, if you will. + +01:06:15.540 --> 01:06:19.360 +Sentry has two excellent and exciting services to tell you about. + +01:06:19.840 --> 01:06:29.960 +Sear, your agentic AI debugging assistant, which takes all the data already gathered by Sentry to help discover the problems and even propose fixes as GitHub PRs. + +01:06:30.560 --> 01:06:37.060 +And the other is AI agent monitoring, which adds deep observability to your AI agents in your app. + +01:06:37.280 --> 01:06:42.780 +If you're adding AI and LLM features to your Python apps, you'll want to know about AI agent monitoring. + +01:06:43.420 --> 01:06:45.600 +I'll tell you more about both of these later in the episode. + +01:06:46.060 --> 01:06:53.260 +And remember, however you happen to sign up for Sentry, if you do, use our code TALKPYTHON, one word, all caps. + +01:06:53.920 --> 01:06:54.780 +Want to level up your Python? + +01:06:55.240 --> 01:06:59.020 +We have one of the largest catalogs of Python video courses over at TALKPYTHON. + +01:06:59.360 --> 01:07:04.060 +Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:07:04.400 --> 01:07:06.680 +And best of all, there's not a subscription in sight. + +01:07:07.160 --> 01:07:09.600 +Check it out for yourself at training.talkpython.fm. + +01:07:10.300 --> 01:07:14.480 +Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:07:14.900 --> 01:07:15.800 +We should be right at the top. + +01:07:16.300 --> 01:07:25.180 +You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +01:07:25.820 --> 01:07:28.060 +We're live streaming most of our recordings these days. + +01:07:28.480 --> 01:07:35.900 +If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:07:36.700 --> 01:07:40.460 +This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. + +01:07:40.820 --> 01:07:42.400 +Now get out there and write some Python code. + +01:07:54.420 --> 01:07:57.220 +I'm out. + diff --git a/youtube_transcripts/356-ml-startups.vtt b/youtube_transcripts/356-ml-startups.vtt index 3ce4f5d..3a84a44 100644 --- a/youtube_transcripts/356-ml-startups.vtt +++ b/youtube_transcripts/356-ml-startups.vtt @@ -777,7 +777,7 @@ in Washington, DC, where I went to school. 00:09:30.000 --> 00:09:35.440 -And that's how I met Matt McKay, who we had a mutual connection. +And that's how I met Matt Makai, who we had a mutual connection. 00:09:35.440 --> 00:09:40.200 diff --git a/youtube_transcripts/459-ai-coding.vtt b/youtube_transcripts/459-ai-coding.vtt index 211b86f..4577cfc 100644 --- a/youtube_transcripts/459-ai-coding.vtt +++ b/youtube_transcripts/459-ai-coding.vtt @@ -2620,7 +2620,7 @@ there's more stuff to write to achieve what you want to achieve. 00:42:34.740 --> 00:42:37.660 -But the thing is that with chatGPT, +But the thing is that with ChatGPT, 00:42:37.660 --> 00:42:42.660 or today with a GitHub copilot embedded in the IDE, @@ -3433,7 +3433,7 @@ fit together and work. - No, I started working with the GPTs from OpenAI, 00:56:11.880 --> 00:56:13.760 -from chatGPT. +from ChatGPT. 00:56:13.760 --> 00:56:15.080 I created a few of them. @@ -3463,7 +3463,7 @@ And he said, "Oh, you should read that "about the module." 00:56:42.040 --> 00:56:43.960 -And I just threw it into chatGPT, +And I just threw it into ChatGPT, 00:56:43.960 --> 00:56:47.980 uploaded the PDF, and I created a custom GPT. @@ -3526,7 +3526,7 @@ but one of the websites where you can get the transcript of a video. 00:57:39.140 --> 00:57:42.020 -So I go get the transcript, throw it into chatGPT +So I go get the transcript, throw it into ChatGPT 00:57:42.020 --> 00:57:45.820 and then I ask, what are the five important things diff --git a/youtube_transcripts/507-agentic-ai-workflows-with-langgraph.vtt b/youtube_transcripts/507-agentic-ai-workflows-with-langgraph.vtt index 74baf17..4494334 100644 --- a/youtube_transcripts/507-agentic-ai-workflows-with-langgraph.vtt +++ b/youtube_transcripts/507-agentic-ai-workflows-with-langgraph.vtt @@ -553,7 +553,7 @@ and right okay maybe as useful for you i can imagine for the case where you're like running something on a schedule the end result might be like okay you've analyze sentiment, do I need to like send an alert to my user or things like that? 00:16:10.140 --> 00:16:27.400 -Yeah, for sure. All right. Well, let's talk a bit about, I guess, Langchain first. So there's a lot of things going on here, right? There's Langchain, the company. There's Langchain, the open source project. There's Langgraph. Tell us about all these things. +Yeah, for sure. All right. Well, let's talk a bit about, I guess, LangChain first. So there's a lot of things going on here, right? There's LangChain, the company. There's LangChain, the open source project. There's Langgraph. Tell us about all these things. 00:16:28.300 --> 00:16:48.580 Yeah, yeah. So happy to give kind of an overview. So LangChain, the open source tool, is basically a tool for developers to use LLMs and really increase the value of LLMs or, you know, kind of augment the power of LLMs with your data. @@ -574,7 +574,7 @@ So if I want to chain a bunch of model responses together, it's kind of hard to There are kind of different response formats and features, etc. 00:17:39.940 --> 00:17:46.440 -And so Langchain makes it really easy for developers to use all of those different model providers in a single +And so LangChain makes it really easy for developers to use all of those different model providers in a single 00:17:46.440 --> 00:17:46.980 application. @@ -604,7 +604,7 @@ And then if you want to use that LLM and then start to use some other more comme You can combine those. 00:18:16.120 --> 00:18:21.100 -And so the name Langchain comes together when we think about chains of systems. +And so the name LangChain comes together when we think about chains of systems. 00:18:21.360 --> 00:18:25.000 One common example that we use is RAG, right? diff --git a/youtube_transcripts/517-agentic-ai-youtube.vtt b/youtube_transcripts/517-agentic-ai-youtube.vtt index d713ebd..050ea6b 100644 --- a/youtube_transcripts/517-agentic-ai-youtube.vtt +++ b/youtube_transcripts/517-agentic-ai-youtube.vtt @@ -517,7 +517,7 @@ And that is kind of almost think about it as like the core. There's inputs into the system and non-deterministic outputs that come out the other side. 00:13:19.080 --> 00:13:25.160 -So you'll say something like, write me a bunch of unit tests and in Cloud Code or in Cursor or in WindSurf. +So you'll say something like, write me a bunch of unit tests and in Claude Code or in Cursor or in WindSurf. 00:13:25.740 --> 00:13:31.800 And then it will interpret those inputs and then produce code or some sort of output out the other side. @@ -604,7 +604,7 @@ It just messes up more than it provides. They're using the cheapest free models that they can find, right? 00:15:34.900 --> 00:15:48.820 -If you use Cloud's Opus model versus some free model, like a 3 billion parameter local model, they're not even in the same category. +If you use Claude's Opus model versus some free model, like a 3 billion parameter local model, they're not even in the same category. 00:15:49.050 --> 00:15:54.120 The type of accuracy and insight and the context. @@ -643,7 +643,7 @@ So even to your example of refactor, here's the ORM, that sort of thing. I started very... 00:18:36.460 --> 00:18:38.760 -When I have a model, and I use a lot of Cloud Code. +When I have a model, and I use a lot of Claude Code. 00:18:38.800 --> 00:18:41.560 I use a lot of Opus and so on now, but I'm sure this will evolve. @@ -784,10 +784,10 @@ For me, it was not the light bulb moment. It was where I started initially using it was, I would, if I was like, oh, I need to like use this new API, or I need to like, kind of like develop a script or whatever it is, it was kind of like my, because I don't normally work in VS Code, I'm like a Vim, Tmux, like for a long time, like that's kind of my natural environment. And I was never going to like kind of adapt my own workflow. And I think a lot of people are like that, right? 00:26:49.960 --> 00:27:30.780 -Like you're in Jeff Reigns, or you're in Xcode or whatever it is, you don't, you, what there's a breakthrough for you is like to build it in a way to your work your development workflow that just is natural and i think that's kind of uh not the canonical definition but to me is kind of most representative of kind of like agentic programming is like it it's just completely a part of your workflow and you don't have to adapt so um so again like that that cursor mode is kind of like okay i'd use it for like one-off stuff for me the breakthrough was using was using And I'll actually, instead of talking about Cloud Code, I'll say, here was the bet that I made. +Like you're in Jeff Reigns, or you're in Xcode or whatever it is, you don't, you, what there's a breakthrough for you is like to build it in a way to your work your development workflow that just is natural and i think that's kind of uh not the canonical definition but to me is kind of most representative of kind of like agentic programming is like it it's just completely a part of your workflow and you don't have to adapt so um so again like that that cursor mode is kind of like okay i'd use it for like one-off stuff for me the breakthrough was using was using And I'll actually, instead of talking about Claude Code, I'll say, here was the bet that I made. 00:27:31.880 --> 00:27:41.400 -My feeling was there was enough value from Cloud Code based off of what I've been, the videos that I've been watching, what I've been reading, and a lot of folks have been using it. +My feeling was there was enough value from Claude Code based off of what I've been, the videos that I've been watching, what I've been reading, and a lot of folks have been using it. 00:27:41.400 --> 00:27:48.620 But I was like, I'm a little worried about getting too reliant upon this tool, which is an API. @@ -1339,7 +1339,7 @@ I don't have like an insane number of extensions or, you know, or I guess plugin Like I actually try to keep things relatively standard and then just focus on the patterns that are most important or the configurations that are most important. 00:47:25.860 --> 00:47:30.240 -And I'm still kind of like that with using Cloud Code. +And I'm still kind of like that with using Claude Code. 00:47:30.580 --> 00:47:35.500 I do have, you know, cloud MD file and stuff, but I also found that it's not 100% accurate. @@ -1417,7 +1417,7 @@ Yeah, exactly. So I think maybe just some quick shout outs to some of the tools, right? 00:49:30.100 --> 00:49:32.940 -So Cloud Code, obviously, is a really interesting one. +So Claude Code, obviously, is a really interesting one. 00:49:33.220 --> 00:49:34.260 We've been talking about that a lot. @@ -1837,7 +1837,7 @@ Because if you fix something or you're asking the agent, be introspective, how c What are the design patterns? 01:03:41.620 --> 01:03:44.680 -And particularly, at least in Cloud Code, there's a plan mode. +And particularly, at least in Claude Code, there's a plan mode. 01:03:44.920 --> 01:03:47.080 I use plan mode. It's like shift tab.