From b0f84bebbf1996c323fb2ab5ec6a1288d73facc4 Mon Sep 17 00:00:00 2001 From: Michael Kennedy Date: Mon, 31 Mar 2025 15:17:29 -0700 Subject: [PATCH 1/2] transcripts --- ...ware-and-the-state-of-python-on-mobile.txt | 2134 ++++++++ ...ware-and-the-state-of-python-on-mobile.vtt | 4442 +++++++++++++++++ 2 files changed, 6576 insertions(+) create mode 100644 transcripts/499-beeware-and-the-state-of-python-on-mobile.txt create mode 100644 transcripts/499-beeware-and-the-state-of-python-on-mobile.vtt diff --git a/transcripts/499-beeware-and-the-state-of-python-on-mobile.txt b/transcripts/499-beeware-and-the-state-of-python-on-mobile.txt new file mode 100644 index 0000000..45e235c --- /dev/null +++ b/transcripts/499-beeware-and-the-state-of-python-on-mobile.txt @@ -0,0 +1,2134 @@ +00:00:00 This episode is all about Beware, the project that is working towards true native apps built on Python, especially for iOS and Android. + +00:00:08 Russell's been at this for more than a decade, and the progress is now hitting critical mass. + +00:00:13 We'll cover the Toga GUI toolkit, building and shipping your apps with Briefcase, + +00:00:18 and the newly official support for iOS and Android in CPython, and much more. + +00:00:24 I can't wait to explore how Beware opens up the entire mobile ecosystem for Python developers, + +00:00:29 So let's jump right in. + +00:00:31 This is Talk Python to Me, episode 499, recorded February 18th, 2025. + +00:00:37 Are you ready for your host, please? + +00:00:40 You're listening to Michael Kennedy on Talk Python to Me. + +00:00:43 Live from Portland, Oregon, and this segment was made with Python. + +00:00:49 Welcome to Talk Python to Me, a weekly podcast on Python. + +00:00:53 This is your host, Michael Kennedy. + +00:00:55 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, both accounts over at Fostadon.org, + +00:01:03 and keep up with the show and listen to over nine years of episodes at talkpython.fm. + +00:01:08 If you want to be part of our live episodes, you can find the live streams over on YouTube. + +00:01:13 Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. + +00:01:19 This episode is sponsored by Posit and Posit Workbench. + +00:01:23 Posit Workbench allows data scientists to code in Python within their preferred environment + +00:01:28 without any additional strain on IT. + +00:01:31 It gives data scientists access to all the development environments they love, + +00:01:34 including Jupyter Notebooks, JupyterLab, Positron, and VS Code, and helps ensure reproducibility and consistency. + +00:01:41 If you work on a data science team where consistency matters, check out Posit Workbench. + +00:01:46 Visit talkpython.fm/workbench for details. + +00:01:50 Russell, welcome to Talk Python to Me. + +00:01:52 Great to have you back. + +00:01:53 Hey, thank you for having me again. + +00:01:55 Yeah, it's good to have you back. + +00:01:56 You've been diligently working away on Beware, and I am very excited to see what you all are up to these days + +00:02:04 because I have very high hopes for the project. + +00:02:06 I'm multiple angles. + +00:02:08 I'm a fan. + +00:02:09 Thank you very much. + +00:02:10 Yeah, I have been very busy, particularly busy over the last couple of years, + +00:02:14 but, yeah, we've made some good progress, and, yeah, we started to see the fruits of what is now at this point almost a 10-year – + +00:02:21 well, it is over a 10-year project. + +00:02:23 So you've been at it for a long time and it's been quite a while since I had you on the show + +00:02:26 and Talk Python has been around for almost 10 years. And so I know you've been at it for + +00:02:30 quite a while. That's awesome. And it seems like you're starting to get some real traction, + +00:02:35 which it couldn't come soon enough. + +00:02:36 Yes. And the good, the sort of the reason why that has happened is basically about three years ago, I was hired by Anaconda to work on their open source + +00:02:46 team um so i am employed full-time just to work on beware uh with another guy malcolm smith who + +00:02:53 works with me based in based in london um and yeah just like the short version is it is amazing how + +00:02:59 much progress you can make when you have two people working full-time on an open source project and + +00:03:02 like just a lot of stuff all the yeah yeah i've always thought that about things it's like there's + +00:03:08 the main product or the main project of the main library and then there's all these little + +00:03:13 not fun, annoying details that somebody just has to grind through. And when it's two hours in on a + +00:03:18 weekend where you're excited about it, it's not the final push over the finish line, grindy details + +00:03:23 that people want to do. And so when it's, I wake up today and we're pushing it forward every day, + +00:03:28 it's awesome. Yeah. And a lot of it is also just like CPython is an awesome project, but it's a + +00:03:34 big project and getting anything to happen. Like it's a, you've got to move a big project in the + +00:03:39 direction of something, that takes a lot of political effort. It takes a lot of time. It + +00:03:43 takes a lot of hanging around our mailing lists. And at some point, if you're just trying to do it + +00:03:46 on like two hours on a weekend, that just doesn't happen. But if you can actually + +00:03:50 say, okay, every morning I'm going to spend an hour responding to all the email that I've received about this + +00:03:55 project, well, then you can do it. And then you can actually have a meaningful progress. + +00:03:59 And then there's also things like, I don't use, I have no disrespect whatsoever for anybody who + +00:04:05 uses Windows. I don't use Windows on a day-to-day basis. It is, as a weekend project, very hard to + +00:04:10 get me excited about working on Windows app signing as a personal side project. And I am going to see + +00:04:17 no benefit out of that whatsoever. But when I've got time for it, a lot of other stuff + +00:04:22 becomes more plausible and more palatable as a thing to look at. + +00:04:26 Yeah. I want to let you do an introduction for yourself real quick before we dive into it. But I + +00:04:31 want to tie this back to PyScript a little bit as well. I think there might be some times that + +00:04:35 align there. + +00:04:36 There absolutely are. Yeah, I can imagine. So for folks who haven't listened + +00:04:41 over all of the years and have a really great memory, tell us, who are you? What are you up + +00:04:46 to these days? Give us a bit of a hint already. + +00:04:48 So hi, I'm Russell. I am in days of yore, I was possibly better known for my contributions to the Django project. I was a very early contributor + +00:04:57 or a member of the Django core team. + +00:04:59 I don't do anywhere, well, I basically don't do any Django work at this point. + +00:05:03 My interests have moved away from web stuff generally. + +00:05:06 For about the last 10 years, I have been working on the Beware project, which is a collection of tools and libraries + +00:05:14 to support building graphical user interfaces, native graphical user interfaces + +00:05:19 in Python and targeting that at desktop platforms, so Windows, Linux, macOS, + +00:05:25 but also mobile platforms, iOS and Android, the web as a GUI user interface, + +00:05:32 and more recently also TextUID, the console-based applications. + +00:05:35 Working on the premise that every platform has a concept of a native interface. + +00:05:41 Now, there is a concept of a button on all of those platforms. + +00:05:45 How you access that button is different on every platform. + +00:05:48 And Beware's goal is to, one, put a clean interface around the outside of that + +00:05:54 so you can have a common API for, I just want a button on the screen so when I push the button, the light goes Bing. + +00:06:00 One, provide a common cross-platform API for that. + +00:06:02 And then two, provide the mechanisms to say, okay, I have written my machine + +00:06:07 that goes Bing with a button. + +00:06:08 I want to give it to someone and I want to give it to them using the platform's app infrastructure. + +00:06:16 I just want to have it on this person's iPhone or give it to a Windows user + +00:06:21 and have them just run it without having to go through, okay, well, first you've got to set up + +00:06:26 a Python virtual environment, and then you've got to pip install it. + +00:06:29 Are you familiar with the terminal? + +00:06:32 Why don't you become familiar with the terminal? + +00:06:33 We'll start there, yeah. + +00:06:35 From my perspective, there are absolutely, as a developer, I need to know that. + +00:06:40 As a Python user, I need to know how that stuff works. + +00:06:43 If I have written a program and I want to give it to someone, if they need to know what Python is, I have failed as a developer + +00:06:50 because as an end user, they don't care. + +00:06:53 They just want to spool the app the same way as everything. + +00:06:56 They want to double click on an icon and make the thing work. + +00:06:59 And that's the other part of doing it. + +00:07:01 That's awesome. + +00:07:01 And I would say if it's not possible to have a little colorful, native-looking icon, either in your dock, in your taskbar, + +00:07:10 in your app launcher on your phone that you touch, and the thing comes to life, + +00:07:14 that's not a real app delivery platform. + +00:07:16 Yeah, it is a very powerful language with which you can do all sorts of incredible things. + +00:07:21 But at the end of the day, you've got to give that thing to someone to do it + +00:07:24 without giving them a three-year degree in computer science before they can actually do the damn thing. + +00:07:29 Sure. + +00:07:29 And even if they can, they might not want to, right? + +00:07:32 Yeah, exactly. + +00:07:32 I don't want to do that. + +00:07:33 It's a barrier. + +00:07:34 Yeah, yeah. + +00:07:35 I love my web browser. + +00:07:36 I don't want to start it that way. + +00:07:37 Yeah. + +00:07:39 Yeah, so the extension of that then is that particularly for the web, for iOS and for Android, + +00:07:47 there's kind of the foundational question then of, well, okay, but how do I run Python on that device at all? You + +00:07:53 know, how do I get Python on my iPhone? + +00:07:57 And I don't just want to have like a Python interpreter on my iPhone because + +00:08:01 you don't like the, an iPhone, everything is inside an app. + +00:08:04 So I need to have it inside my app. How do I get it inside the app? + +00:08:06 And then when it's inside the app, how do I invoke the code that's there and so on and so on. + +00:08:12 So a lot of the work that's kind of been leading up to this point is like + +00:08:16 literally how to make cpython work on ios how do i make cpython work on android how do i make cpython + +00:08:22 work in the browser um right so yeah that's awesome a very very interesting and powerful goal i think + +00:08:29 let's not talk about it just yet but i think it's gonna it's it's really it leaves python open to + +00:08:35 being not the default choice even though it's a really great programming default choice language + +00:08:40 somebody says i want to i want to get started programming awesome you should check out python + +00:08:44 yeah i want to build an app you shouldn't check out python right like you don't want that to be the + +00:08:48 the answer right and this path that you're you're making you're hacking away + +00:08:53 in the forest forest yeah exactly one of the one of the stimulus for me working on this there was a couple + +00:09:00 but one of them is that my son who is now first year university but at the time when i was sort + +00:09:04 of starting it was you know late primary school um was starting to look at how to program and the + +00:09:11 device that he had at school was an iPad. And he says, okay, well, I'm learning to program. Great. + +00:09:17 How do I build an iPad app? Well, you don't. Like, it's complicated. Like, maybe. + +00:09:23 Have you heard the story for it? It's kind of complicated. + +00:09:26 It's learning. Then you say, okay, well, but I'm learning Python because Python is a good + +00:09:32 language for teaching people how to program. Okay, well, now I want to write that iPad app. + +00:09:39 Yeah, but you can't. + +00:09:40 So maybe you've got to learn a different language for that. + +00:09:42 But I don't want to learn a different language. + +00:09:44 I've only barely got my hands around Python. + +00:09:45 Why do I have to learn another language just to be able to have an app there? + +00:09:49 So there's a running sort of semi-joke that goes around that Python is the second best language for everything. + +00:09:55 And part of it being the second best language for everything is that it needs to be able to do all these things, + +00:10:01 maybe not as well as Swift on an iPhone or Coughlin on an Android machine or whatever, + +00:10:08 but well enough that you can get something going. + +00:10:11 Yeah, I would be well happy to say it takes an extra 5% of my battery compared to a regular app. + +00:10:16 They use a Python app. + +00:10:17 I don't care. + +00:10:18 It's fine. + +00:10:19 But it's not that. + +00:10:20 It's possible, impossible. + +00:10:22 Right? + +00:10:23 That's not a great divide. + +00:10:24 All right, before we dive into this though, you mentioned that you work on the open source team at Anaconda. + +00:10:30 So I want to give you a moment to talk a little bit about what that is. + +00:10:33 I had Peter Wang on the show not too long ago and he's always super fun to talk to. + +00:10:38 And I know that they're putting a lot of support behind PyScript. + +00:10:43 And it sounds like the timing of releasing PyScript and all that is a little bit aligned with when you joined. + +00:10:49 Yeah, so maybe talk to us a bit about all this. + +00:10:51 Yeah, so Anaconda took a round of funding a little over three, almost four years ago, + +00:10:57 I think at this point. + +00:10:58 And part of that was to sort of really start doubling down on the contribution of open source. + +00:11:05 Anaconda is a company who is built upon tools that are open source, most notably Conda, but that's not the only piece of it. + +00:11:15 Conda is an open source project. + +00:11:17 Someone needs to maintain it. + +00:11:19 Anaconda is able to make money selling this open source product, this whole Python ecosystem, into Fortune 500 companies who need all sorts of legal compliance and standards compliance and security reporting and all that kind of stuff. + +00:11:33 that your student doing a high school computer programming degree doesn't care about, but + +00:11:38 the big end of town does. + +00:11:40 Anaconda is able to make money out of that, but recognizes that the only reason they're + +00:11:45 able to make money doing that is that there is this open source foundation, that Python + +00:11:49 is an open source ecosystem. + +00:11:51 And unless people are maintaining that stuff, Anaconda is not going to have a company in + +00:11:57 a couple of years because everyone's going to move away from Python or do other things + +00:11:59 or find other ways of solving problems. + +00:12:02 And, you know, that ultimately is bad news for Anaconda. + +00:12:05 The other side of it, and sort of the reason why PyScript is interesting, why Anaconda + +00:12:10 is supporting iOS and Android work, is that there is an opportunity here to just literally + +00:12:15 make the pie bigger. + +00:12:16 That if Anaconda, you know, if someone can come in and say, oh yeah, I want to write + +00:12:20 my, you're at a Fortune 500 company, I want to write my app, I want to write it in Python. + +00:12:24 Okay, well now I can do it at iOS and Android. + +00:12:27 There's now more users for Anaconda's products as a result of this investment that's been + +00:12:32 in iOS and Android to make it possible at all. + +00:12:35 It is very much a long-term project. + +00:12:37 But like this is iOS and Android stuff. + +00:12:39 Conda as a project does not have particularly good support for iOS and Android. + +00:12:43 Eventually, hopefully not that far off, it will. + +00:12:47 But you don't get to Conda supporting it until Python supports it. + +00:12:51 And you don't get Python supports it unless someone sits down and actually does the hard work + +00:12:55 to make that happen, which is essentially what I was hired to, myself and Malcolm, who had to do. + +00:13:00 And PyScript fits in a very similar boat. + +00:13:02 The web is undeniably a platform that is everywhere that you can't run Python in, or you couldn't run Python, + +00:13:08 or it was very hard to run Python in. + +00:13:10 And PyScript as an effort is essentially saying, okay, let's make it as straightforward as possible + +00:13:16 for me to just drop some Python into a browser and do web user interface stuff + +00:13:22 where instead of having to learn JavaScript and learn a whole new ecosystem + +00:13:25 and learn a whole bunch of new set of tools, I can just use Python and open up. + +00:13:30 that whole other ecosystem of places where you can run Python code as another place where you + +00:13:36 might want to do things. + +00:13:37 Yeah. And to give people a sense of what's possible with PyScript, it's very, very interesting. On iOS and also on Android, I'm sure I just haven't tried it, + +00:13:47 you can create progressive web apps and then say, save this to my home screen. And when it launches, + +00:13:52 it looks like a native app, but you can't really tell that it's not a native app. + +00:13:56 I put together a sample when PyScript first came out that had an offline or offline capable PyScript, all Python written app. + +00:14:04 And it was talking to some APIs out on the Internet. + +00:14:07 And even just those early stages, it was like, wow, this really makes things possible. + +00:14:12 The challenge is, I mean, this is the big challenge, is people don't discover apps on their mobile devices through progressive web apps, right? + +00:14:21 Hence your project, right? + +00:14:23 They go to the app store and they click get or buy, and then it shows up on their device, right? + +00:14:29 It's kind of like back to like, well, if you could create a virtual environment, + +00:14:32 you could run this thing. + +00:14:33 If you can create a progressive web app, I'm a huge fan of them, but my fandom doesn't change people's + +00:14:39 behavior, right? + +00:14:40 Sure. + +00:14:41 But there is also like to defend my own project here, like I'm a big fan of PyScript. + +00:14:46 I love what they're doing. + +00:14:47 There are limits to what you can do in a progressive web app. + +00:14:49 Those limits are getting extended further and further out, But there are things that + +00:14:53 if you want to have access to the entire capabilities of your device directly at the hardware layer, you can't do that in a progressive web app. + +00:15:02 You can only do it if you're actually writing to the native APIs and you've got direct hardware access. + +00:15:09 The web browser sandbox does place some limits on what you can do. + +00:15:15 Do those limits matter? + +00:15:16 That's an open question. + +00:15:17 there are a lot of things that probably should be progressive web apps that are, you know, + +00:15:21 or that could be progressive web apps. + +00:15:24 But that doesn't mean that there's no market for native apps or that PyScript is like, + +00:15:29 PyScript is eventually going to make, you know, Beware or native apps completely irrelevant. + +00:15:33 Because there is... + +00:15:33 No, it absolutely won't. + +00:15:34 Yeah. + +00:15:35 And the other thing is that they actually do complement each other really well. + +00:15:38 Like, Beware does have a PyScript web backend or PyScript-based web backend specifically + +00:15:44 so you can say, okay, I want to build an app. + +00:15:47 it has a button and when I press it, the dialog button, my dialog box pops up. + +00:15:51 Nothing about what I just said dictated whether it was a web browser or an iOS app or a desktop app. + +00:15:57 I can render that on all of them and then say, okay, well, it starts as a progressive web app because that's how I want to do it + +00:16:02 and then I'm going to spend a little bit more effort and actually make it into a native app and get it into the iOS app store. + +00:16:07 You can progress along that chain without cutting off any options. + +00:16:13 Yeah. + +00:16:13 Oh, I think your project, the native app side is critical. + +00:16:17 I just think there's also a bunch of people out there that say, well, I'm going to build a front end, interactive front end and have a bunch of Django REST framework or something on the back end. + +00:16:25 So I'll use React or I'll use Vue. + +00:16:27 And why isn't the answer Python, right? + +00:16:29 It really, really could be Python. + +00:16:31 That's kind of, I think, where PyScript has a lot of possibilities. + +00:16:35 This portion of Talk Python to Me is brought to you by the folks at Posit. + +00:16:39 Posit has made a huge investment in the Python community lately. + +00:16:42 known originally for RStudio, they've been building down a suite of tools and services + +00:16:46 for Team Python. Have you ever thought of all the things that go into a Python data science project? + +00:16:53 You need your notebook or IDE, sure. Also a server or cloud environment to run it, + +00:16:58 a version of Python, packages, access to your databases, and internal APIs. That's a lot of + +00:17:03 setup. And if you change any of these things, when you return to your projects a month down the road, + +00:17:08 you might get different results. Wouldn't it be nice to have all of this set up for you in one + +00:17:13 easy to access place whenever you want to get work done? That's the goal of Posit Workbench. + +00:17:19 Posit Workbench allows data scientists to code in Python within their preferred environment + +00:17:24 without an additional strain on IT. It gives data scientists access to all the development + +00:17:29 environments they love, including Jupyter Notebooks, JupyterLab, Positron, and VS Code. + +00:17:34 and yet it helps ensure reproducibility. + +00:17:37 Here's how it works. + +00:17:38 You or your IT team set up Posit Workbench on a powerful, dedicated server within your organization + +00:17:44 or on the same cloud service that is hosting your most important data sources, + +00:17:48 such as AWS, SageMaker, Azure, GCP, Kubernetes, or pretty much anywhere. + +00:17:54 There, you create dedicated, pre-configured environments to run your code and notebooks. + +00:17:59 And importantly, you also configure access to proprietary databases and internal APIs. + +00:18:05 When it's time to onboard a new data scientist or start a new project, you just fire it up in Workbench and it's fully configured and ready to go, including on the infrastructure side of things. + +00:18:14 All of this is securely administered by your organization. + +00:18:18 If you work on a data science team where consistency matters, you owe it to you and your org to check out Posit Workbench. + +00:18:24 Visit talkpython.fm/workbench today and get a three-month free trial to see if it's a good fit. + +00:18:29 That's talkpython.fm/workbench. The link is in your podcast player's show notes. + +00:18:34 Thank you to Posit for supporting Talk Python To Me. + +00:18:38 I think that brings us a little bit to one of your keynotes from 2019, where you talked + +00:18:42 about Black Swan events, right? + +00:18:44 Yes. + +00:18:46 It's a little bit of the same story, but maybe give people a sense of this and we can link + +00:18:49 to this keynote if they want to explore that side of things more. + +00:18:52 Sure. + +00:18:52 Yeah. + +00:18:53 So the idea behind a Black Swan event was Nicholas... + +00:18:58 Nassim Talib, I believe it was. + +00:19:00 Nassim Talib, that's the one. + +00:19:01 Yep. + +00:19:01 I forget his name. + +00:19:02 but he identified this idea of what's called a black swan event. + +00:19:07 The reason I use this is I'm from Perth and black swans come from Perth. + +00:19:10 So it was an interesting little tie-in for an international keynote. + +00:19:13 You guys actually have black swans all over? + +00:19:15 We have them. + +00:19:16 The state emblem of Western Australia is a black swan. + +00:19:19 Like that is literally on our flag. + +00:19:23 So the theory was that the Dutch bumped into Western Australia accidentally looking for the Spice Islands in the 1600s. + +00:19:31 And at the time they did that, they knew with absolute certainty that all swans were white. + +00:19:37 And then they bumped into Western Australia and they discovered that all the swans here are black. + +00:19:42 The idea behind a black swan event is a thing you don't see coming that completely changes your view of the world. + +00:19:50 And black swan events are the sorts of things that completely change markets. + +00:19:53 We don't know what, if you're not looking, if you're not ready for a cataclysmic change in your ecosystem, then the change happens and you are left behind. + +00:20:05 I would say COVID is a good example of what hit people. + +00:20:10 Another one's AI. + +00:20:11 Yeah, AI is still, how is that going to impact? + +00:20:14 It's just a little bit less clear. + +00:20:16 And COVID, we're kind of seeing the impacts of that, whether that's going to have long-term impacts. + +00:20:21 like it's some of those things seem to be kind of fading away. + +00:20:24 But the one that I, yeah, well, yeah, mostly. + +00:20:28 I mean, people, I mean, people still get to say, but like the, the world has shut down and + +00:20:33 will we, will economies just disappear? + +00:20:36 Level of, Oh my gosh. + +00:20:37 It's kind of, that's the history I was thinking of. + +00:20:39 Yeah. + +00:20:39 I, the one that I was highlighting for, for, for Python is that Python is a programming language, + +00:20:46 but it's a programming language that has historically assumed, well, you're running on a laptop, + +00:20:49 right? + +00:20:50 Like that's where you run computer code. + +00:20:51 you always run computer code in a laptop. And then iPhones turned up. And now we've got computing in + +00:20:56 our pocket that is as powerful, if not more powerful than the laptops that we had on our desk + +00:21:01 10 years ago. And Python can't run there, or at least at the time, couldn't really run there. + +00:21:07 So what does that mean for Python? Why am I going to learn Python? If the computing ecosystem moves + +00:21:13 to a world where everybody's using phones and tablets, and I can't run Python on a phone or a + +00:21:18 tablet, why am I going to learn Python? Python as an ecosystem has an existential risk there + +00:21:23 because of this black swan of a change in the way people do computing. + +00:21:29 And at the time I gave that keynote, what I was essentially saying is, okay, as an ecosystem, we need to take a long, hard look + +00:21:35 at ourselves. It's impossible to know what the black swan events are going to be, but we can look + +00:21:40 at where the ecosystem is now. We can look at the trends of things that are happening and we can work + +00:21:44 out institutionally what are our risks, like what would need to change to make Python a + +00:21:52 dead ecosystem. + +00:21:54 And the obvious one, the one that I was really pushing at that time, one of which is platforms + +00:22:00 that we need to support. + +00:22:00 And the second is how do we support this software going forward? + +00:22:04 How do we actually guarantee that this community-driven project continues to be maintained? + +00:22:10 Because if we don't have maintainers, then the code doesn't get written. + +00:22:14 So, yeah. + +00:22:15 Yeah, excellent. + +00:22:15 I recommend people check it out. + +00:22:17 It's on YouTube. + +00:22:17 I will link to it. + +00:22:19 It's scary the last five years ago. + +00:22:21 I know. + +00:22:22 I feel like that was pretty recent, but no. + +00:22:25 No, that was the last one before COVID, by the way, bringing it full circle. + +00:22:29 Tommy out there asked in the audience, is in the iOS Pythonistas app, I had some really good simple use cases in the past. + +00:22:35 The GUI layer, a bit of a hassle, but like the library modules, also could not call some not yet exposed APIs. + +00:22:42 I guess as we get into this, maybe think, just people probably know about the Pythonista app. + +00:22:47 I don't know. + +00:22:49 Yeah, it was around. + +00:22:51 And it's not the same idea. + +00:22:52 It's not what you guys are trying to do. + +00:22:54 Yes and no, because I actually worked on Pythonista. + +00:22:57 The reason that Pythonista supports Python 3, I was contracted to do the Python 3 port. + +00:23:02 So the core of that library is essentially the thing that Beware was doing. + +00:23:05 And the reason it was able to move to Python 3 was because of the work that I was doing. + +00:23:10 It is, yeah, it is very much sort of the Pythonista as an idea is kind of what if idle, but on a phone? + +00:23:18 Like here is a development environment where you can write Python code. + +00:23:22 You have some access to some libraries and some GUI libraries. + +00:23:27 So you can put a button up, but you're not writing, you're not using Pythonista to write an app. + +00:23:33 You're using it to put an interface into a running Pythonista sheet, you know, working + +00:23:40 environment. + +00:23:41 It is absolutely a way to write Python. + +00:23:44 It is a way to have Python running on your phone. + +00:23:48 But it is not something that's going to push something to the iOS app store so that random + +00:23:52 other person can buy your app. + +00:23:55 You know, to me, it feels a little bit like shortcuts on iOS in that you could kind of + +00:23:59 put together some behaviors, but you would never try to make an app. + +00:24:02 yeah that's it's not that's not too far from it and it except that it's all inside the sound + +00:24:07 yeah and you could do real python which is way better than shortcut but the kind of the type + +00:24:12 of thing you accomplish it feels a little bit like i can go find this thing and run it and it'll do + +00:24:16 something for me which is cool but yeah it's not going to be the new airbnb implementation no + +00:24:22 exactly and like even like i guess beware as a project isn't it the intention is not that it would be + +00:24:29 the next Airbnb as a final product. + +00:24:33 It might, however, be the mock-up of the demo app that you take to your initial seed funding round + +00:24:40 to prove, here is the idea we've got. + +00:24:43 We've proved it out with some people, just like a small little sample trial, whatever. + +00:24:47 Now give us the money to go and actually hire a team of iOS developers and a team of Android developers + +00:24:51 to build an absolutely kick-ass, optimized native application of every platform. + +00:24:57 Yeah, maybe a little bit like Playgrounds Well, perhaps in that regard. + +00:25:02 Yeah, a little bit, I guess, like Playgrounds, yeah. + +00:25:04 Yeah, also a different focus. + +00:25:05 But another comment maybe as we dive into this, Joe out there asks us, hey, great project. + +00:25:10 How would you sell this to us? + +00:25:12 And that there's a few other frameworks like WXPython, Kivy. + +00:25:15 WXPython also renders data of GUIs. + +00:25:18 How would I sell it? + +00:25:20 Okay, says he, pulling out the sales sheet. + +00:25:23 Let me get my pitch deck. + +00:25:24 Hold on. + +00:25:25 So there's essentially two questions there. + +00:25:27 One is WX Python and one is KIVI. + +00:25:31 The KIVI is a sort of the two different reasons. + +00:25:35 KIVI does work on iOS and Android and also on the desktop platforms. + +00:25:39 KIVI is taking a very, very distinctly different approach to user interface design. + +00:25:44 So KIVI can run on all those platforms, but the user interface that it puts up is a KIVI + +00:25:49 user interface. + +00:25:50 It always looks like KIVI. + +00:25:51 And it's like if you build a macOS app, it will look the same on Windows. + +00:25:56 It'll look the same on GDK, and it will look the same, you know, size of window notwithstanding. + +00:26:00 It will look the same on iOS and Android. + +00:26:02 That is a completely viable way to build apps. + +00:26:06 It is not a native user interface. + +00:26:10 And one of the major reasons that I dislike that as an approach, I'm not in any way saying that Kibbe is a bad project. + +00:26:17 It is a different way to build user interfaces, and I prefer a different approach, which is to say an actual native button. + +00:26:23 So on macOS, it looks like a macOS app. + +00:26:26 On iOS, it looks like an iOS app. + +00:26:28 On Windows, it looks like a Windows app. + +00:26:32 The downside is that if you need to take screenshots, you've got to have an iOS screenshot + +00:26:36 and a macOS screenshot and a Windows screenshot. + +00:26:39 The benefit is consistency with the rest of the platform. + +00:26:43 And that might seem like a totally cosmetic thing, but one, consistency between applications + +00:26:50 is a mode of user interface discovery because if people know this is how a button works on MacOS, + +00:26:56 they know how your platform does it as well, how your app does it as well, + +00:26:59 rather than having to discover how you have decided to implement menus or buttons or anything else. + +00:27:05 Yeah, one of the really things that stands out, I think, super stark a lot of times is the file open save dialogs. + +00:27:10 Yes, yes. + +00:27:11 It's like, whoa, this is clearly just alien versus... + +00:27:15 Yeah, and a lot of cases, they are missing the one thing on my platform that makes file dialogues actually useful + +00:27:22 because I can do this thing that I can't do in your dialogue because you've had to build it from scratch. + +00:27:27 And speaking of building from scratch, the other big benefit of a native GUI + +00:27:31 is that you get all of the platform's native affordances. + +00:27:34 One of the most surprising things, it wasn't necessarily surprising because I understood why I was doing it, + +00:27:39 but things that might be surprising to people who haven't followed along GUI development, + +00:27:43 we have received an inordinate number of very, very enthusiastic comments + +00:27:48 from blind people writing iOS apps. + +00:27:52 Because an iOS app built with Beware fits in directly with screen readers + +00:27:58 because it is using the platform's native accessibility. + +00:28:02 Kivy has not built explicitly native screen reader support into their widgets. + +00:28:06 To the best of my knowledge, or at least all the reports that I'm getting + +00:28:09 are saying that it's not there because you have to build it from scratch + +00:28:13 on every platform and they haven't done that. + +00:28:16 It's a lot of work. + +00:28:17 I'm not underselling how big of a project it is, but Beware got it for free because we are using native widgets, + +00:28:24 and that's not an insignificant consideration. + +00:28:28 That's Kivy. + +00:28:28 Oh, sorry. + +00:28:29 No, no, please go ahead. + +00:28:30 No, I was going to say, that's the Kivy. + +00:28:32 So that's the not native user interface but looks the same everywhere versus Beware's using native widgets everywhere. + +00:28:38 The other argument is, okay, but why don't you use WX Windows? + +00:28:41 The reason we don't use WX Windows is WX Windows doesn't run on iOS or Android and trying to make it run on iOS and Android. + +00:28:47 Like I did look into it and it was a little bit crunchy. + +00:28:50 Qt apparently works on iOS now. + +00:28:53 It didn't when I started the project, but they very much don't want you to write Python. + +00:28:58 And one of the things that Beware is trying to push and Toga as a GUI platform is trying to push + +00:29:01 is you're writing Python code. + +00:29:03 It looks like Python. + +00:29:04 It tastes like Python. + +00:29:06 If you don't want to write Python, that's fine. + +00:29:08 There are plenty of GUI toolkits out there that don't require you to write Python. + +00:29:11 But Toga is a Python first GUI toolkit. + +00:29:14 And it looks like, tastes like Python, we support, we use, actively use in APIs, iterators and generators and asyncio syntax and context managers. + +00:29:25 Things that WX Windows as a light wrapper around a C++ framework doesn't really expose because they are limited by what C++ looks like when you program it. + +00:29:37 And you also get that to a certain extent in QT as well or PyQT. + +00:29:42 Yeah, they're both wrappers. + +00:29:43 Interesting. + +00:29:44 So let's talk about some of the peps that make this possible, right? + +00:29:48 Yeah. + +00:29:49 There's the foundational, basically adding platforms as a intentionally supported platform + +00:29:56 to the CPython whole process, right? + +00:29:59 The core developers working on it from the builds and the deployments and all those kinds + +00:30:03 of things. + +00:30:03 And so you've got a couple of peps where you're the author of them, which is awesome. + +00:30:08 I'm strictly the author of one. + +00:30:10 Malcolm Smith was the author of the Android one. + +00:30:11 Oh, okay. + +00:30:12 Got it. + +00:30:13 One Android with Malcolm and one iOS with you. + +00:30:16 Okay. + +00:30:16 I see which side fence you come down on. + +00:30:18 I got it. + +00:30:18 Yep. + +00:30:19 No, no, no, no. + +00:30:20 That's all good. + +00:30:20 I'm with you. + +00:30:21 I'm with you. + +00:30:22 All right. + +00:30:22 Tell us about these peps. + +00:30:23 What role do they have? + +00:30:25 And their status is final. + +00:30:27 That's good. + +00:30:29 So, yeah. + +00:30:29 So, the PEP was essentially formalizing the things that Malcolm and I have been maintaining + +00:30:36 independently, both of us, at the time for like about eight, nine years. + +00:30:41 We had patches. + +00:30:42 like I was maintaining a set of patches to make CPython run on iOS, and I was manually maintaining all those patches. + +00:30:48 And every time a new Python release would come out, I'd have to go and sit down and spend, you know, + +00:30:53 a couple of months trying to work out, okay, what have they changed this time? + +00:30:56 And, you know, getting all of those, updating all of the PR, all of the code to make it work on a new version of Python. + +00:31:04 In the 3.13 timeframe, so it's, what, September of 2023, I went along to the CPython core team summit in the US. + +00:31:18 Sorry, it was in Bono. + +00:31:22 And basically with the intention of, I want to upstream all these patches. + +00:31:26 We've got these patches. + +00:31:27 They're not huge. + +00:31:27 They're big, but they're not huge. + +00:31:29 I'd like to formalize that. + +00:31:30 And so the outcome of that was what we need is a PEP. + +00:31:32 We need a formal description of what it is you're going to do, why it's going to change, + +00:31:37 what impact that has on the rest of the ecosystem for you to do that, and to formalize a couple of the little details + +00:31:45 that need to be formally specified in order for the rest of the ecosystem + +00:31:49 to play nice with this. + +00:31:50 Little details like, what does sys.platform return on an iOS machine? + +00:31:54 Like it's a really minor detail, but it's a question you've got to answer. + +00:31:58 And it's like, it's an answer that even Apple Darwin is probably wrong. + +00:32:01 Well, yeah, that's the thing. Like it's Darwin on macOS for some historical reasons, but like even + +00:32:05 on iOS, should it be iOS? Should it be iOS capitalized? Should it be iPhone? + +00:32:10 Should it be iPhone OS? + +00:32:12 Should it be iPhone OS or iPhone simulator? + +00:32:14 All of these are answers that Apple itself uses at various points in their API. + +00:32:18 And what about iPad? + +00:32:20 Well, an iPad is iPad iOS. + +00:32:22 Is that a different answer? + +00:32:23 And so on. + +00:32:24 And then the one that really matters for that is nailing down what is the platform tag that will be used for packaging purposes? + +00:32:32 So when a wheel, a binary wheel, eventually lands on PyPI, what's the tag? + +00:32:38 Like what is it? + +00:32:40 For macOS, it's a macOS X underscore 11 underscore ARM64. + +00:32:44 Okay. + +00:32:45 What is it for iPhone? + +00:32:46 Yeah, maybe we could elaborate. + +00:32:47 Maybe you could elaborate just a little bit on like what that even means, right? + +00:32:50 Because before Wheels, things were different. + +00:32:52 You would download it. + +00:32:53 You would build it. + +00:32:54 You'd have to have Fortran or some random thing. + +00:32:56 Wheels solved that, but that then created a combinatorial explosion of binary assets, + +00:33:00 right? + +00:33:01 Why does that matter? + +00:33:02 Why does that matter? + +00:33:02 So the major change with wheels is that it moved from a world where installing a Python package required you to execute code at time of installation. + +00:33:15 And that's not a good idea from a security perspective because you then need to audit all of the code that's going to execute when you install the package. + +00:33:26 And you can't find anything out about the package without running that code or at least auditing the code that's going to run. + +00:33:34 Wheels provides you to say, okay, this is a wheel is basically just a zip file with a bunch of known metadata that describes exactly what is in here. + +00:33:44 But part of that specification is also a binary specification that says, what version of CPython is it compatible with? + +00:33:50 What version of the operating system is it compatible with? + +00:33:52 What CPU architecture is it compatible with? + +00:33:55 So that if I look at a wheel that is labeled, for example, macOS 11.0 ARM64, I know that this will run on macOS 11 with an ARM64 CPU. + +00:34:07 There are any number of these tags available. + +00:34:09 And there's differences like the Universal 2, which will support both x86 and ARM. + +00:34:15 And on Linux, there's a thing called a mini Linux wheel, which is a specification for specifically what API guarantees does this Linux platform guarantee in terms of what is available in the CPython, in terms of the C library, the GNU C library, et cetera, et cetera. + +00:34:34 You have to be able to specify that so that I know when I get this wheel, I know where it will work and how it will work. + +00:34:39 And for the point of view of PIP, pip knowing which version should I download? + +00:34:44 I've got an app here that I'm running on my macOS machine. + +00:34:47 Which wheel should I download? + +00:34:49 Go and grab one and resolve it and find out and solve that question. + +00:34:52 Yeah. + +00:34:52 So without knowing that answer, you don't know how to install wheels, basically. + +00:34:57 Correct. + +00:34:57 Part of the wheel specification is which – if the wheel contains all the pre-compiled components, in order for it to have pre-compiled components, there's an API or an ABI specification there of exactly what symbols am I going to be able to link against? + +00:35:12 What can I assume exists? + +00:35:14 Will this run on my machine? + +00:35:16 Yeah. + +00:35:17 What's going to happen, right? + +00:35:19 Is it even compiled from architecture? + +00:35:21 And we have a similar thing with Python standalone builds that is now part of Astral, which is really great. + +00:35:26 we can just grab the Python we need because we know the platform tags and all that kind of stuff. + +00:35:30 Any difference about the Android one? + +00:35:33 I mean, you obviously need one for each platform, I suppose. + +00:35:36 We do, yeah, basically. + +00:35:38 And there will be at some point in the hopefully not too distant future, + +00:35:40 there will be one for Inscripten as well, just formalizing Inscripten support, + +00:35:45 which is the Pyodide slash PyScript in the browser version. + +00:35:50 Yeah, you need to have one for every platform. + +00:35:53 I guess the one notable difference between iOS and Android as a PEP is that in theory, + +00:36:00 CPython 12 could compile on Android. + +00:36:04 There were some patches which had been applied without going through the formal PEP process + +00:36:08 on the basis of, well, it doesn't cost us anything to add this code in. + +00:36:13 We're not guaranteeing it's going to work, but it's not breaking anything. + +00:36:16 So we might as well have it there. + +00:36:17 And so there's one, like the process of formalizing it did change one notable thing, + +00:36:25 which is that previously, if you were able to get Sys.platform to compile on Android, + +00:36:29 and it was possible and realistically it needed some patches, but it could be done, + +00:36:33 Sys.platform would return Linux. + +00:36:37 Because technically it is, but not in a way that's even remotely helpful to anyone. + +00:36:43 And so one of the things that the PEP specified, like, yes, it is a Linux kernel, + +00:36:48 but Linux doesn't help you there because nothing about an Android machine + +00:36:52 actually behaves like Linux once you're past like the kernel level. + +00:36:57 The rest of the thing is what you actually care about. + +00:36:59 And the rest of the libraries you can guarantee is what you care about. + +00:37:02 And so one of the things that the PEP had to specify was, okay, we are changing what sys.platform means on Android + +00:37:10 because you need to be able to differentiate I am on an Android machine, not just a desktop Linux box. + +00:37:18 And so, yeah, so that was one notable change. + +00:37:20 It was in that PEP that wasn't in the iOS one. + +00:37:23 Interesting. + +00:37:23 You mentioned mScripten. + +00:37:25 What an interesting project. + +00:37:27 Yeah, it almost looks like it's not AI because it's LLVM, but it has nothing to do with it. + +00:37:33 Yeah, yes. + +00:37:33 It's a different set of LLs altogether. + +00:37:37 Yeah. + +00:37:37 So inscription is kind of the tail end, the current manifestation of a long series of + +00:37:47 developments that started like 15 years ago. + +00:37:50 Someone worked out that modern web browsers have spent so much time optimizing the JavaScript + +00:37:59 compilers that there are certain JavaScript constructs that map almost one-to-one to machine + +00:38:06 code. + +00:38:06 Like you can say, okay, add these two integers in JavaScript or to these two numbers in JavaScript that if you know the numbers are integers, will map through the JavaScript jits that are running. + +00:38:21 It will map to a literal, you know, add instruction on your CPU. + +00:38:25 Like two registers coming together. + +00:38:27 Yeah, two registers coming together. + +00:38:28 And so if you restrict yourself to handwriting just that subset of JavaScript, you can essentially write assembler in JavaScript. + +00:38:40 Now, in practice, nobody can actually do that because it's just way too constraining to know that you actually have to put the brackets around the outside of here or else the jit won't optimize it in just the right way. + +00:38:52 But that's what compilers are for. + +00:38:53 Compilers are really good at taking a set of arbitrary input and following a set of arbitrary rules to produce a consistent set of output. + +00:38:59 And so this thing called ASM.js was born, which is essentially an input format that could be compiled to a JIT-able version of JavaScript that would run as a native platform. + +00:39:10 And then from that, you can then get to WASM, which is WebAssembly, which is a formal text specification of the sort of things that ASM.js was working on. + +00:39:18 And then Inscripten is a compiler tool chain that plugs into LLVM. + +00:39:23 So if you've used a compiler like Clang, it is essentially a wrapper around a set of compiler building tools that produces output. + +00:39:34 So you are used to calling GCC or Clang on a C program. + +00:39:39 Clang calls into your C program, compiles it into an LLVM format, and LLVM converts it into actual runnable executables. + +00:39:49 And so it's an LLVM plugin that instead of spitting out macOS-compatible binary executable assembly, it spits out WASM. + +00:39:59 It spits out JavaScript-optimized code that can run in the browser. + +00:40:03 That's all very much high level. + +00:40:05 There's a lot of very, very complicated details that go on in the internals. + +00:40:08 But the end point is that Enscripten lets you take any arbitrary C code and compile it to + +00:40:14 JavaScript that will run in a browser at nearish native speed. + +00:40:19 Good news is that CPython is written in C. + +00:40:22 And so you can compile CPython into a JavaScript version or into this JavaScript subset, and + +00:40:27 it will run in the browser. + +00:40:29 There's a bunch of other stuff you need to do as well. + +00:40:31 And Enscripten is providing a bunch of other things like guaranteeing a bunch of Unix system + +00:40:35 calls are available. + +00:40:36 But broadly, that's the idea here. + +00:40:38 So you can take any C code and have it run in the browser. + +00:40:41 And at that point, Python is just C code. + +00:40:44 So all of a sudden, yeah, but the best news is because the browser is everywhere, because + +00:40:49 JavaScript interpreters are everywhere, you've now got cross-platform binaries. + +00:40:53 You've got a format that you can compile to once, and it will run pretty much everywhere + +00:40:59 because existentially, every platform that matters has a web browser now, which means + +00:41:03 it's got the ability to run JavaScript. + +00:41:05 And so JavaScript becomes this cross-platform format without ever actually being JavaScript the language. + +00:41:12 It's just kind of exploiting some weird quirks of the way JavaScript works as a language. + +00:41:16 It's like saying you never write assembly, but kind of still do machine instructions. + +00:41:20 But you're always writing assembly. + +00:41:21 Yeah, exactly. + +00:41:22 There's a really, if you go back, oh gosh, 12, 13 years, Gary Bernhardt gave a talk at PyCon + +00:41:29 called The Birth and Death of JavaScript. + +00:41:32 So good. + +00:41:33 It was so good. + +00:41:34 It is hilarious. + +00:41:36 At the time, it was just straight up, oh, this is a funny gag, except that he has accurately predicted the next 15 years + +00:41:43 of software development in terms of the way everything has gone. + +00:41:46 And I've never spoken to Gary about it, so I don't know how much of this was just for the giggles + +00:41:53 and how much of it was accurately like this is where things are going. + +00:41:57 But as it turns out It was both really, really entertaining and really technically well-presented + +00:42:04 and predicted, like you said. + +00:42:05 Yes. + +00:42:06 It's something. + +00:42:07 And for people who are not familiar with this stuff, like one of the examples he showed, + +00:42:10 I don't know if he came up with it or he was just quoting and showing it off, + +00:42:13 but somebody took Mozilla Firefox, I think, and compiled that into Scriptum, + +00:42:18 was running inside of Chrome, but then inside the embedded, Firefox was running like Doom compiled, + +00:42:23 Doom Scriptum or some crazy 3D. + +00:42:26 I mean, it was wild. + +00:42:27 Yeah. + +00:42:27 The list of things you can do once you've got the ability to compile C is truly phenomenal, + +00:42:35 purely because the C compiler toolchain for all of the flaws of C as a language, C is + +00:42:43 lingua franca. + +00:42:44 C is where everything is done. + +00:42:46 And we're now kind of removing, like the whole thing with LRVM is essentially we're removing + +00:42:50 C from the front end, letting you put any language you want on the front. + +00:42:54 But now we can use the same toolchain to target any platform we want. + +00:42:57 And so you divorce the specific language from the tool chain that is producing platform-specific binaries. + +00:43:05 Nice. + +00:43:05 All right, let's talk about some of the projects and get some updates on them. + +00:43:08 I know it's been a while. + +00:43:09 And, you know, you've got a lot of support from Anaconda, which, you know, thank you again. + +00:43:15 Yes, absolutely. + +00:43:16 That's awesome. + +00:43:17 So we'll maybe touch on some of the high-level or most important ones. + +00:43:20 Toga? + +00:43:21 So Toga, the cross-platform GUI toolkit. + +00:43:24 So this is the analog of a Kibbe or a QT or a WX Windows. + +00:43:30 It's the API that lets you say, I want a window that has a button. + +00:43:33 And when I press the button, the light goes off, whatever you want it to do. + +00:43:37 Put this text in the label or whatever. + +00:43:39 Exactly, exactly. + +00:43:41 So it is cross-platform, pure Python, runs cross-platform. + +00:43:47 And by cross-platform, I mean all the desktop platforms, so Windows, macOS, Linux with GTK. + +00:43:53 It runs as a web. + +00:43:55 There's a web backend and a textual backend. + +00:43:56 Both of those are very early prototypes, but I can't, like there are demos I can do + +00:44:00 that I demonstrate that it works. + +00:44:02 And also iOS and Android. + +00:44:04 So essentially anywhere that you might want plausibly to have a user interface, + +00:44:08 I can give you a user interface purely written in Python. + +00:44:11 We also cover a bunch of hardware services as well. + +00:44:15 So for example, you can, using a pure Python interface, get a GPS read. + +00:44:19 You can work out where you are at a GPS and we have a native WAP widget. + +00:44:22 So you can say, get a GPS read and put it by center of the map of where I am currently standing. + +00:44:26 And that works on iOS, on Android, on macOS. + +00:44:30 We've got it working on GTK and we've got a PR in flight to make it work on Windows as well, I think. + +00:44:35 That, you know, it just, same code just runs everywhere and just, where am I? + +00:44:39 Give me an answer. + +00:44:40 Give me a ping every time I move a significant distance. + +00:44:43 And also for cameras, similar, similar thing for cameras. + +00:44:46 There is a question out of the audience, which I don't know the answer, but I'll throw that for you. + +00:44:50 Is there any more love coming for the Beware WebView widget? + +00:44:55 Having a case, not what they're referring to with Ceph, but is there more love coming? + +00:45:03 I mean, sure, as soon as somebody wants to pay attention to it. + +00:45:05 Am I actively intending in the very near future to do anything with it? + +00:45:09 Probably not. + +00:45:10 The most likely change that's on the horizon is to close the loop on callbacks. + +00:45:15 So at the moment, you can, from Python, inject arbitrary JavaScript into a web view, + +00:45:21 but you can't have something happen in the JavaScript to trigger an action outside in the Python, + +00:45:26 which is essentially kind of the last piece of the puzzle you need to have a full kind of electron clone. + +00:45:33 Yeah, I was going to say, that's very electron. + +00:45:35 Yeah, we actually have a bunch of tooling in place. + +00:45:39 We call it positron because it's like electron, but positive because it's Python. + +00:45:42 I love it. + +00:45:43 I love it. + +00:45:43 Yeah, so we can take a website and turn it into a native app. + +00:45:48 The only thing you can't do is then close the loop back out and call native hardware type stuff back into the Python API. + +00:45:56 Not impossible to do. + +00:45:57 I know exactly how to do it on iOS and macOS. + +00:45:59 I think I know how to do it on GTK. + +00:46:01 I don't know how to do it on Windows. + +00:46:03 So that might happen at some point in the not too distant future. + +00:46:08 Beyond that, oh, Seth, are they referring to the Chromium framework? + +00:46:12 No intention to, but if someone wants to write that up, there's no reason you couldn't be a standalone widget. + +00:46:17 This is very much from the Django days of if there is no, just because something exists doesn't mean it needs to be in the core. + +00:46:27 So if someone wants to write a third-party widget that wraps the Chromium embedded framework + +00:46:33 and turn it into a Toga widget, Toga will support that. + +00:46:36 And to the extent that if there's something that makes it a little bit crusty, + +00:46:39 there's probably a couple of areas where it could be a little bit smoother. + +00:46:41 If there's a way we can make it smoother, I'm totally on board with that. + +00:46:44 Doesn't mean it needs to be in Toga core itself. + +00:46:47 Django has a huge ecosystem of Django compatible plugins. + +00:46:52 Toga very much can be in that same space. + +00:46:54 Everything doesn't have to be in Toga for it to be useful in Toga. + +00:46:58 That's awesome. + +00:46:58 That's good to know. + +00:46:59 So I guess maybe the big updates here are the textual and the web targets for Toga. + +00:47:04 Is that right? + +00:47:05 They've been the biggest changes in the last eight years, I guess, is that those two now + +00:47:10 exist. + +00:47:11 And obviously you still need a lot of work, but like proof of concept, you can do it. + +00:47:15 And it's really just kind of turn the handle and make more widgets come out. + +00:47:19 The biggest sort of impediment to that at the moment is actually testing. + +00:47:22 One of the changes that has come on since I've been working at Anaconda full time is we've + +00:47:28 gone and got test coverage. + +00:47:29 We've got 100% test coverage of all of the core and all of the desktop and mobile platforms. + +00:47:34 So we have a very, very solid regression test suite to know that things aren't going to + +00:47:39 break as a result of this weird little tweak over here that we make. + +00:47:42 We've actually got validation of all that. + +00:47:44 We don't have that yet on web and textual, mostly because of quirks of how those platforms actually run. + +00:47:50 There's a kind of a foundational question we need to answer on the web one + +00:47:53 in particular. + +00:47:55 Yeah, I can imagine how the textual one works. + +00:47:57 What's the runtime look for part of the web? + +00:48:00 Is that PyScript or is that PyOxid? + +00:48:03 It is PyScript, yeah. + +00:48:05 So it's a briefcase, which I guess we'll get to in a moment, But Briefcase has the ability to target a web deployment. + +00:48:10 It generates a HTML page that injects a load this wheel. + +00:48:17 And this wheel is your app. + +00:48:19 But it's a Python wheel that just contains your code for your running app. + +00:48:25 And, yeah, but it's PyScript that's making that work. + +00:48:27 Yeah, PyScript is neat. + +00:48:28 Definitely neat. + +00:48:29 Just the fact that it's opening up these types of things. + +00:48:31 So Briefcase, yeah, let's dive a little more into that one. + +00:48:34 Yeah, so Briefcase is then sort of the other part of the story. + +00:48:38 you know, I've written this out. + +00:48:39 It gives you the runtime, but you're still in the terminal to launch it with virtual environments and all that kind of business, right? + +00:48:44 Yeah, okay. + +00:48:44 Yeah, so briefcase is, the scary thing is at the end of the day, briefcase is really just kind of three templates + +00:48:50 in a trench coat. + +00:48:51 There's not a lot of, like, there's not a lot that it's doing beyond that + +00:48:55 and also encoding all of the annoying details that are embedded 15 pages deep in Microsoft's + +00:49:02 how to write an app, you know, documentation around code signing and packaging and building an installer + +00:49:10 and all of that kind of stuff. + +00:49:12 So on the theory that 99% of Python users have code that runs as Python minus M, my app, make that be a macOS app. + +00:49:21 And that's what Briefcase does. + +00:49:22 It says, okay, how do I turn this into an icon on my desktop that I can double click and the app will just start. + +00:49:28 And it will include an embedded version of Python. + +00:49:32 So I don't need to tell my end user how to install Python. + +00:49:35 it will be signed and notarized in accordance with Apple's latest specifications it might come + +00:49:41 inside a DMG because I need to be able to distribute it along with a readme or it might + +00:49:44 come as a.pkg file because I actually need like in the case of non-GUI apps you actually like you + +00:49:49 want to be able to put something on your path and you have to have like a post install script and + +00:49:53 pkgs how you do that on macOS similarly on Windows it produces an MSI and it drops a Python + +00:49:59 embedded and all that kind of stuff make sure your dependencies are there on macOS it makes sure that + +00:50:04 of you install. If I'm building it on ARM machine, the app will still run on x86 because it'll install + +00:50:09 both versions of both the x86 and the ARM binaries and merge them where they're necessary and do all + +00:50:13 that kind of stuff. So it's all the little fiddly details. It is essentially just running a bunch + +00:50:18 of command line, like everything Briefcase does is just a command line tool or rolling out a template. + +00:50:25 If you turn up the verbosity, it will tell you in the prompt, like this is what I'm running. I'm + +00:50:34 notary tool and so on and so on. + +00:50:36 But you don't have to worry about that. + +00:50:37 Like it's just going to, this is what will encode. + +00:50:39 It'll do it all for you and just spit out either a DMG, a.zip, an MSI, IPA files on iOS, APK files or AAB files on Android, + +00:50:51 whatever the native platform is. + +00:50:53 And then on Linux, one of the things we've added more recently, we can do flat packs, we can do Debian, RPM, PKG zip, + +00:51:03 PKG files for Arch and Manjuro, like all of the common packaging formats we try to cover. + +00:51:08 The only one that I can think of that's notable that we don't cover at the moment. + +00:51:12 We don't do NCS installers on Windows and we don't do Snap packages on Linux. + +00:51:19 Not because we can't, just because I've only got two arms and there's a limit to how much I can practically do. + +00:51:25 If someone wants those, I can point you at the code and show you what you're going to do. + +00:51:29 And it's a moderately... + +00:51:31 If you know the commands you need to run to make a thing happen, I can show you how to + +00:51:35 plug that into Briefcase. + +00:51:36 And like Briefcase's infrastructure is designed to be pluggable. + +00:51:39 So it doesn't even need to be in Briefcase. + +00:51:41 You can, again, do it as an external plugin and have it plug into Briefcase. + +00:51:45 There are two things I want to ask you about here. + +00:51:46 One, notarization and signing. + +00:51:48 I remember how long ago I built C++ desktop apps and.NET desktop apps. + +00:51:55 And you could just compile them and give them to somebody and let them run it. + +00:51:59 And those sweet days are long gone. + +00:52:01 You know, it'll get flagged with big warnings. + +00:52:03 Like this looks like a virus because it didn't go through our digital signature system. + +00:52:08 Yep. + +00:52:09 What a frustrating thing. + +00:52:10 So this has support for that. + +00:52:12 It does. + +00:52:12 Yeah. + +00:52:14 Yeah. + +00:52:14 It turns out there are bad people on the internet and they're willing to do bad + +00:52:18 things. + +00:52:19 Who knew? + +00:52:20 So yeah, macOS has a bunch of processes for signing all of the binaries in an application + +00:52:26 and then stamping what's called a notarization certificate to the installer to say, + +00:52:30 This is a package in a format that hasn't been modified that I am willing to verify as me as a person. + +00:52:37 This is what the version of the package looks like when it's correctly installed. + +00:52:41 And then Sorry, probably in practice, if you want to distribute stuff to Mac and Windows, you probably + +00:52:47 need to create a developer account and sign these things, unless your users are willing to run with the big scary, this looks + +00:52:53 unsigned and untrusted. We've moved it to the trash, or you can go in there and run it. You know + +00:52:57 what I mean? Yeah, like on macOS, It like literally has to be at this point. + +00:53:01 You actually can't get away with not getting a developer certificate because the secure stuff on the M1 ship + +00:53:08 literally will not run. + +00:53:09 You can do cheap and nasty ad hoc signing, which means it will run on my machine. + +00:53:14 But if you want to give it to anybody else, it is now at the point where you've got to go into the settings, + +00:53:19 explicitly say, I'm willing to trust content that comes from an unknown provider. + +00:53:23 Start the app by pressing option and double click, selecting the option of I'm willing to accept this. + +00:53:29 Like it can be done, but it's like nobody is actually ever going to do that. + +00:53:33 So you basically, yeah, you've got to sign up for a $99 a year developer account + +00:53:38 and then go through the signing process. + +00:53:40 And signing is a non-trivial activity. + +00:53:42 It is difficult. + +00:53:43 There are like you've got to do things in the right order. + +00:53:46 And there is like even you have to submit the app in a zip format. + +00:53:50 But you can't just use PK zip or your standard command line zip. + +00:53:54 You've got to use Ditto, which is Mac's version of zip, Because otherwise, file system properties can end up being corrupted because the UTF-18 coding won't go quite reversible in the way. + +00:54:04 Like, it is a nightmare. + +00:54:07 Briefcase solves all those problems for you. + +00:54:09 It should be as straightforward as just package this thing and sign and it works. + +00:54:15 Similarly, it's not quite as bad on Windows. + +00:54:18 Like, Windows will let you install an unsigned binary. + +00:54:21 It just comes up with sort of the, this comes from an unknown developer. + +00:54:24 Are you sure you want to do this? + +00:54:26 And you say, okay. + +00:54:27 But realistically, like people's willingness to click the this looks like it might be insecure is low and decreasing. + +00:54:35 And so, yeah, you've just got to go get a certificate, sign the app, validate that it's you. + +00:54:39 It's kind of it's the price of doing app business at this point. + +00:54:42 Yeah, I guess it's sort of the equivalent of you have to have SSL to run a website. + +00:54:46 However, exactly. + +00:54:47 It is exactly that. + +00:54:48 Yes, but we're missing Let's Encrypt. + +00:54:50 We're missing Let's Encrypt. + +00:54:51 Yes. + +00:54:51 That's that part. + +00:54:52 Yeah. + +00:54:52 From the Windows point of view, there is absolutely a window of opportunity to do a Let's Encrypt because Windows doesn't specify who you've got to get your certificate from. + +00:55:04 It just has to be signed by a root certificate that they honor. + +00:55:07 So, like, essentially the instructions in briefcase kind of go get a certificate. + +00:55:10 Where do you get it from? + +00:55:12 You know, here are some options. + +00:55:14 Go have fun. + +00:55:15 So there is absolutely an opportunity for someone to do a Let's Encrypt here for app signing. + +00:55:19 Unfortunately for Apple, that doesn't apply because Apple does insist that the certificate needs to be issued by them, signed by them, so that they can revoke it. + +00:55:28 And maybe the EU changes around app distribution will relax that somewhat, but at least for the moment, you've got to go through Apple. + +00:55:35 Yeah. + +00:55:35 As salty as I am at the EU about the cookie warnings. + +00:55:39 Yeah. + +00:55:40 Yeah. + +00:55:40 But I do have hope that they can bring some reason to some of the other App Store type behaviors. + +00:55:45 It would certainly be. + +00:55:46 I would not. + +00:55:47 I welcome that development. + +00:55:49 I am waiting to see what happens. + +00:55:51 I had a lot of hope that the App Store stuff, I had a lot of hope opening the App Store thing + +00:55:55 would provide a way to make it easier for Beware to distribute apps. + +00:56:00 But in practice, it doesn't look like that's actually going to happen. + +00:56:03 So yeah. + +00:56:04 It's, yeah. + +00:56:05 Nice try, but they squeeze through another track, basically. + +00:56:11 It is what it is. + +00:56:12 It is what it is. + +00:56:14 The gatekeeper stuff drives me crazy. + +00:56:15 I've had so much trouble getting my mobile apps in the app store, not for reasons technical, + +00:56:20 just for reasons of... + +00:56:21 Yeah, and I guess that's the thing. + +00:56:24 It's like, I will, to a limited extent, defend the, not so much the gatekeeping aspect of it, + +00:56:33 but the fact that there is a process involved to protect against completely arbitrary content. + +00:56:40 That side of it is useful. + +00:56:42 But the fact that there's, like, it is that locked down and only from Apple is the part + +00:56:46 where it becomes a little bit onerous and that there's no competition to make Apple work better, + +00:56:50 I guess is the real thing. + +00:56:51 Yeah, yeah. + +00:56:52 You can't go into a competition. + +00:56:54 Yeah, and the rent is a, yeah. + +00:56:55 Yeah, yeah, yeah. + +00:56:56 From my app there. + +00:56:57 You don't have in-app purchasing. + +00:56:59 Well, I don't want to sell anything in my app. + +00:57:00 I just want to let people view the stuff that we already got online. + +00:57:03 Well, you're going to have to add it. + +00:57:04 So weeks and weeks of paying developers and stuff to build it just so they can have their few bucks, + +00:57:11 you know what I mean? + +00:57:12 And it's just like, you have no idea how much pain this is. + +00:57:15 Yeah, and like, I guess, I could almost, I don't like it, but I could almost grant them that, you know, the golden rule of + +00:57:22 has the gold makes the rules, but it's their store. They get to decide what's in there, + +00:57:26 but I don't have the option of another store. + +00:57:28 And in particular, the review process is so inscrutable. Like Google is no better in this regard. If something + +00:57:37 goes wrong with your Google application, good luck with that. For all the money they're taking, they're not providing good + +00:57:46 customer service. They're providing no as a customer service. + +00:57:50 Exactly. Yeah. They're both bad. I'm not saying Google's good. They're both bad. Okay. But for a briefcase here, we have, + +00:57:57 we have the targets that I would expect Linux, Mac, Windows. Yes. But iPad, iPhone, Android, + +00:58:03 tell us about that. Even watch. + +00:58:06 So we, yeah, so we don't, we don't actually have it working for watchOS. + +00:58:09 I have a proof of, yeah, I have a proof of concept that it can work on Apple TV. I have + +00:58:14 written a Python application for Apple TV. + +00:58:17 Web deployment, actually, that's that website's out of date. + +00:58:19 I should need to update that. + +00:58:20 We do have web deployments. + +00:58:21 Put that in the proper bullet list, right? + +00:58:23 Yes. + +00:58:24 Yes, it should be on the proper bullet list. + +00:58:26 So, yeah, but submitting to an app store, there are iOS apps, there are Android apps in the Apple and Google app stores + +00:58:34 that are written with BWare. + +00:58:37 So you are producing a binary. + +00:58:39 It is signed and validated and then sent to Apple, and Apple says, yes, that's fine, and Google says, yes, that's fine, + +00:58:44 and it's available for purchase. + +00:58:45 So, yeah. + +00:58:46 Awesome. + +00:58:46 So what do you get, like an APK for Android? + +00:58:49 And what is the AP? + +00:58:50 It's an IPA. + +00:58:51 Yeah, it's an IPA, but you don't ever see it. + +00:58:54 Like you basically have to, for all practical purposes, you have to submit it through Xcode. + +00:58:58 That might be something we can resolve, but I haven't. + +00:59:00 They don't make it easy to find out how to not use Xcode to do it. + +00:59:04 No, I think you're probably committed. + +00:59:06 Even for my Flutter-based app, I've got to go do a build and an app store submission through Xcode. + +00:59:12 Yeah, yeah. + +00:59:12 The tools are there, but yeah. + +00:59:15 But yeah, you get an AAB for Android. + +00:59:17 You get effectively an IPA, but an Xcode project you can submit for iPhone. + +00:59:23 For web, you get a tarball that is basically upload to your web provider of choice, + +00:59:29 and it would work, and it would end up being similar for Apple TV or WatchOS or WearOS eventually. + +00:59:36 Awesome. + +00:59:37 Well, that's really great that you've been making so much progress on this, + +00:59:40 And is it Steven, the other guy you're working with? + +00:59:44 Malcolm. + +00:59:45 I'm sorry, Malcolm. + +00:59:47 You and Malcolm have been making so much progress. + +00:59:48 That's really, really great. + +00:59:50 And I think only good things are going to come from having the backing of Anaconda. + +00:59:54 So we've got just a couple of minutes left. + +00:59:55 What do you want to tell people about this project, these projects, how they should adopt them, where + +01:00:00 they're ready for use, and so on, things like that? + +01:00:02 I guess the quick thing I'd say is that we're not done yet. + +01:00:05 There's still more to come. + +01:00:07 Just a week and change ago, PIP. + +01:00:10 sorry, PyPI officially started accepting iOS and Android wheels. + +01:00:15 So you can upload an iOS and Android wheel. + +01:00:18 There aren't any on there yet that I'm aware of, but I'm actively working on that. + +01:00:22 So just so I can prove, hey, look, yes, there's one over there. + +01:00:24 Look at it. + +01:00:25 But part of the reason there's not one there yet is that the tooling to build iOS and Android wheels is not quite there. + +01:00:31 And that's what I'm actively working on right now. + +01:00:33 I would be deeply surprised if by the end of this year, there aren't multiple high profile projects that have. + +01:00:39 I have the patches for Pillow to literally produce iOS wheels as part of their normal release process. + +01:00:46 They haven't even seen the PRs yet, but I'm certainly hoping to work with them. + +01:00:51 Yeah, I mean, they know it's coming. + +01:00:52 Hugo knows it's coming. + +01:00:53 So there is a lot of work still to do in terms of getting that patch accepted, but at least I know it can be done. + +01:01:00 And more importantly, things like CI build wheels, like the infrastructure for putting a build all these wheels as part of CI is something where I have the patch and I'm like this close to being able to submit a pull request to CI build wheel to say, here's the back end to let you generate iOS wheels. + +01:01:16 Similarly, we've got to do all that work for Android as well. + +01:01:18 So there is still a lot of work to come. + +01:01:20 My hope is that sort of by the end of this year, says he, being very hopeful, we'll be in a position where like anything you can do on a desktop platform, you can basically do on iOS and Android as well. + +01:01:33 With the caveat that there might be, if there's C code involved, you might need to actually like work out how to compile this thing for iOS and Android. + +01:01:41 For a lot of simple projects, literally there's nothing you've got to do. + +01:01:43 You've just cranked the handle and make the compiler do its thing. + +01:01:46 Yeah, especially if you have Python. + +01:01:48 Yeah, exactly. + +01:01:49 The other part that I'm sort of playing around with this year as a sort of a background project is doing for Enscripten + +01:01:55 what I did last year or I'm up and I did last year for iOS and Android, which is get Enscripten up to tier three support. + +01:02:01 So there will be a PEP and official supported platform. + +01:02:05 Like Enscripten will be an officially supported platform in CPython in the same way that iOS and Android are. + +01:02:13 I'm mostly shepherding that as sort of a member of the CPython core team + +01:02:17 and Hood Chatham is actually doing most of the heavy lifting on that. + +01:02:21 He's a PyDyde maintainer and is involved with PyScript as well to a lesser extent. + +01:02:27 And, yeah, so he is working on the patches for that and I'm kind of shepherding those patches into core. + +01:02:32 We are about to stand up and build bot to actually make the kind of the – + +01:02:35 make sure every pull request is validated to make sure it still works on Inscripten essentially, which is one of the big sort of milestones + +01:02:42 for official platform support. + +01:02:43 Yeah, that's super neat. + +01:02:45 What about CI? + +01:02:46 for people. + +01:02:47 Yeah, what about CI for people who want to build wheels for iOS and Android? + +01:02:52 That's essentially where CI build rule comes in. + +01:02:54 So you can do it right now. + +01:02:55 So every time we make a change to Toga, we run a full test suite on iOS, on Android, + +01:03:01 on macOS and all the other desktop platforms. + +01:03:04 You can use like GitHub Action CI will let you start an iPhone simulator + +01:03:08 and run the tests on that iPhone simulator. + +01:03:10 You can start an Android simulator and it will run a simulator on those platforms. + +01:03:14 It's like it takes three minutes to start the simulator, but that's just the price of doing business for iPhone, basically. + +01:03:21 Yeah, I guess they probably already have CI for iOS and Android because there's probably + +01:03:25 a ton of mobile projects on there. + +01:03:27 It's kind of unusual to complain it with Python, right? + +01:03:30 Yes, absolutely. + +01:03:31 And so you can do it. + +01:03:33 You can run those tests. + +01:03:35 There are probably some things that could be done at an institutional level at GitHub + +01:03:38 to make those tests run faster. + +01:03:41 But that's a, you know, now I've just got to convince Microsoft to implement an entirely new backend for GitHub Actions. + +01:03:48 And they don't support like Windows on ARM yet. + +01:03:51 So I'm not holding their breath thinking I support iOS. + +01:03:53 Yeah, yeah, I hear that. + +01:03:55 One final forward-looking question, I suppose. + +01:03:57 What do you see support for MScript in giving us? + +01:04:01 You know, for example, like having the JavaScript stuff, you'd see like, okay, well, great, I can run it in a browser. + +01:04:06 But well, then Node comes along and now I can run servers on it or I can distribute it or there's tool. + +01:04:11 Like there's these sort of ways in which these environments grow unexpectedly, at least for most people, I think. + +01:04:17 And how do you think InScript in my... + +01:04:19 So my hope is that what we'll end up seeing is the analog of React and Vue, but for Python. + +01:04:29 So you can right now, or you have been able to write a Django server-side website, + +01:04:34 pick any other GUI, any other web framework you want. + +01:04:36 You can write your server-side code, excuse me, you write your server-side code completely in Python. + +01:04:40 It's great. + +01:04:41 but then you've got to push your code to the front end and you have to write JavaScript on the front end. + +01:04:48 And so that's kind of essentially where Node came from is realizing that we've got to write this code, + +01:04:51 like we've got validation code, you've got to run it around the client and run it around the server. + +01:04:54 So I can't put Python in the client, so I might as well bring JavaScript back to the server. + +01:04:59 And so having that same language everywhere is a real advantage. + +01:05:03 InScripten lets us get Python into the browser, but the part that we don't have yet + +01:05:07 is the rich Python-first GUI framework in the browser. + +01:05:13 My hope is that with... + +01:05:15 Oh, you lost me there? I'm not sure. + +01:05:18 My hope is that with inscription in the browser, what we'll get is the ability to build that toolkit. + +01:05:30 I don't know what it is. + +01:05:31 I don't know what will stimulate its development, but it should hopefully make it, or at least makes it possible. + +01:05:36 And once it's possible, then a bunch of people, you know, project people can start playing around with it and actually making things. + +01:05:46 Oh, we lost Russell. + +01:05:49 Well, that's probably a good place to call it, folks. + +01:05:52 Thanks to Russell for being on the show. + +01:05:54 And thank you all for listening. + +01:05:55 Talk to you all later. + +01:05:56 Bye. + +01:05:58 This has been another episode of Talk Python to Me. + +01:06:01 Thank you to our sponsors. + +01:06:02 Be sure to check out what they're offering. + +01:06:04 It really helps support the show. + +01:06:06 This episode is sponsored by Posit and Posit Workbench. + +01:06:10 Posit Workbench allows data scientists to code in Python within their preferred environment + +01:06:15 without any additional strain on IT. + +01:06:18 It gives data scientists access to all the development environments they love, + +01:06:21 including Jupyter Notebooks, JupyterLab, Positron, and VS Code, and helps ensure reproducibility and consistency. + +01:06:28 If you work on a data science team where consistency matters, check out Posit Workbench. + +01:06:33 Visit talkpython.fm/workbench for details. + +01:06:36 Want to level up your Python? + +01:06:38 We have one of the largest catalogs of Python video courses over at Talk Python. + +01:06:42 Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:06:47 And best of all, there's not a subscription in sight. + +01:06:49 Check it out for yourself at training.talkpython.fm. + +01:06:53 Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:06:57 We should be right at the top. + +01:06:58 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +01:07:08 We're live streaming most of our recordings these days. + +01:07:11 If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:07:19 This is your host, Michael Kennedy. + +01:07:21 Thanks so much for listening. + +01:07:22 I really appreciate it. + +01:07:23 Now get out there and write some Python code. + diff --git a/transcripts/499-beeware-and-the-state-of-python-on-mobile.vtt b/transcripts/499-beeware-and-the-state-of-python-on-mobile.vtt new file mode 100644 index 0000000..3b60a0e --- /dev/null +++ b/transcripts/499-beeware-and-the-state-of-python-on-mobile.vtt @@ -0,0 +1,4442 @@ +WEBVTT + +00:00:00.020 --> 00:00:07.940 +This episode is all about Beware, the project that is working towards true native apps built on Python, especially for iOS and Android. + +00:00:08.740 --> 00:00:13.040 +Russell's been at this for more than a decade, and the progress is now hitting critical mass. + +00:00:13.820 --> 00:00:18.440 +We'll cover the Toga GUI toolkit, building and shipping your apps with Briefcase, + +00:00:18.840 --> 00:00:23.900 +and the newly official support for iOS and Android in CPython, and much more. + +00:00:24.700 --> 00:00:29.260 +I can't wait to explore how Beware opens up the entire mobile ecosystem for Python developers, + +00:00:29.620 --> 00:00:30.380 +So let's jump right in. + +00:00:31.100 --> 00:00:35.800 +This is Talk Python to Me, episode 499, recorded February 18th, 2025. + +00:00:37.239 --> 00:00:39.240 +Are you ready for your host, please? + +00:00:40.100 --> 00:00:42.940 +You're listening to Michael Kennedy on Talk Python to Me. + +00:00:43.480 --> 00:00:46.560 +Live from Portland, Oregon, and this segment was made with Python. + +00:00:49.820 --> 00:00:52.680 +Welcome to Talk Python to Me, a weekly podcast on Python. + +00:00:53.280 --> 00:00:54.940 +This is your host, Michael Kennedy. + +00:00:55.270 --> 00:00:57.840 +Follow me on Mastodon, where I'm @mkennedy, + +00:00:58.000 --> 00:01:03.280 +and follow the podcast using @talkpython, both accounts over at Fostadon.org, + +00:01:03.680 --> 00:01:08.180 +and keep up with the show and listen to over nine years of episodes at talkpython.fm. + +00:01:08.780 --> 00:01:12.640 +If you want to be part of our live episodes, you can find the live streams over on YouTube. + +00:01:13.220 --> 00:01:18.920 +Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. + +00:01:19.780 --> 00:01:22.660 +This episode is sponsored by Posit and Posit Workbench. + +00:01:23.500 --> 00:01:28.180 +Posit Workbench allows data scientists to code in Python within their preferred environment + +00:01:28.650 --> 00:01:30.680 +without any additional strain on IT. + +00:01:31.440 --> 00:01:34.640 +It gives data scientists access to all the development environments they love, + +00:01:34.820 --> 00:01:37.720 +including Jupyter Notebooks, JupyterLab, Positron, and VS Code, + +00:01:38.200 --> 00:01:40.960 +and helps ensure reproducibility and consistency. + +00:01:41.360 --> 00:01:43.620 +If you work on a data science team where consistency matters, + +00:01:44.180 --> 00:01:45.540 +check out Posit Workbench. + +00:01:46.180 --> 00:01:48.620 +Visit talkpython.fm/workbench for details. + +00:01:50.280 --> 00:01:52.160 +Russell, welcome to Talk Python to Me. + +00:01:52.430 --> 00:01:53.120 +Great to have you back. + +00:01:53.340 --> 00:01:54.780 +Hey, thank you for having me again. + +00:01:55.160 --> 00:01:56.540 +Yeah, it's good to have you back. + +00:01:56.810 --> 00:02:00.040 +You've been diligently working away on Beware, + +00:02:00.400 --> 00:02:04.080 +and I am very excited to see what you all are up to these days + +00:02:04.230 --> 00:02:06.060 +because I have very high hopes for the project. + +00:02:06.620 --> 00:02:08.179 +I'm multiple angles. + +00:02:08.860 --> 00:02:09.240 +I'm a fan. + +00:02:09.539 --> 00:02:10.200 +Thank you very much. + +00:02:10.800 --> 00:02:14.240 +Yeah, I have been very busy, particularly busy over the last couple of years, + +00:02:14.740 --> 00:02:16.080 +but, yeah, we've made some good progress, + +00:02:16.380 --> 00:02:21.500 +and, yeah, we started to see the fruits of what is now at this point almost a 10-year – + +00:02:21.500 --> 00:02:22.920 +well, it is over a 10-year project. + +00:02:23.020 --> 00:02:23.320 +So + +00:02:23.320 --> 00:02:26.160 +you've been at it for a long time and it's been quite a while since I had you on the show + +00:02:26.520 --> 00:02:30.160 +and Talk Python has been around for almost 10 years. And so I know you've been at it for + +00:02:30.580 --> 00:02:34.860 +quite a while. That's awesome. And it seems like you're starting to get some real traction, + +00:02:35.120 --> 00:02:36.560 +which it couldn't come soon enough. + +00:02:36.800 --> 00:02:40.000 +Yes. And the good, the sort of the reason why that has + +00:02:40.220 --> 00:02:46.199 +happened is basically about three years ago, I was hired by Anaconda to work on their open source + +00:02:46.220 --> 00:02:53.360 +team um so i am employed full-time just to work on beware uh with another guy malcolm smith who + +00:02:53.680 --> 00:02:59.140 +works with me based in based in london um and yeah just like the short version is it is amazing how + +00:02:59.220 --> 00:03:02.780 +much progress you can make when you have two people working full-time on an open source project and + +00:03:02.850 --> 00:03:03.480 +like just a lot + +00:03:03.480 --> 00:03:07.880 +of stuff all the yeah yeah i've always thought that about things it's like there's + +00:03:08.260 --> 00:03:12.519 +the main product or the main project of the main library and then there's all these little + +00:03:13.000 --> 00:03:18.480 +not fun, annoying details that somebody just has to grind through. And when it's two hours in on a + +00:03:18.580 --> 00:03:21.180 +weekend where you're excited about it, it's not the final + +00:03:21.180 --> 00:03:21.920 +push over the + +00:03:21.920 --> 00:03:23.300 +finish line, grindy details + +00:03:23.460 --> 00:03:28.480 +that people want to do. And so when it's, I wake up today and we're pushing it forward every day, + +00:03:28.920 --> 00:03:29.140 +it's + +00:03:29.140 --> 00:03:30.020 +awesome. Yeah. And + +00:03:30.020 --> 00:03:34.180 +a lot of it is also just like CPython is an awesome project, but it's a + +00:03:34.280 --> 00:03:39.459 +big project and getting anything to happen. Like it's a, you've got to move a big project in the + +00:03:39.420 --> 00:03:43.580 +direction of something, that takes a lot of political effort. It takes a lot of time. It + +00:03:43.680 --> 00:03:46.740 +takes a lot of hanging around our mailing lists. And at some point, if you're just trying to do it + +00:03:46.800 --> 00:03:49.860 +on like two hours on a weekend, that just doesn't happen. But + +00:03:49.860 --> 00:03:50.460 +if you can actually + +00:03:50.460 --> 00:03:50.960 +say, okay, + +00:03:51.420 --> 00:03:55.440 +every morning I'm going to spend an hour responding to all the email that I've received about this + +00:03:55.580 --> 00:03:59.260 +project, well, then you can do it. And then you can actually have a meaningful progress. + +00:03:59.880 --> 00:04:05.159 +And then there's also things like, I don't use, I have no disrespect whatsoever for anybody who + +00:04:05.180 --> 00:04:10.320 +uses Windows. I don't use Windows on a day-to-day basis. It is, as a weekend project, very hard to + +00:04:10.320 --> 00:04:16.739 +get me excited about working on Windows app signing as a personal side project. And I am going to see + +00:04:17.040 --> 00:04:21.400 +no benefit out of that whatsoever. But when I've got time for it, a lot of other stuff + +00:04:22.420 --> 00:04:26.420 +becomes more plausible and more palatable as a thing to look at. + +00:04:26.740 --> 00:04:31.280 +Yeah. I want to let you do an introduction for yourself real quick before we dive into it. But I + +00:04:31.260 --> 00:04:35.480 +want to tie this back to PyScript a little bit as well. I think there might be some times that + +00:04:35.620 --> 00:04:35.960 +align there. + +00:04:36.300 --> 00:04:37.860 +There absolutely are. Yeah, + +00:04:37.960 --> 00:04:40.200 +I can imagine. So for folks who haven't listened + +00:04:41.220 --> 00:04:46.220 +over all of the years and have a really great memory, tell us, who are you? What are you up + +00:04:46.300 --> 00:04:48.120 +to these days? Give us a bit of a hint already. + +00:04:48.460 --> 00:04:53.100 +So hi, I'm Russell. I am in days of yore, + +00:04:53.120 --> 00:04:57.419 +I was possibly better known for my contributions to the Django project. I was a very early contributor + +00:04:57.440 --> 00:04:59.140 +or a member of the Django core team. + +00:04:59.960 --> 00:05:01.040 +I don't do anywhere, + +00:05:01.500 --> 00:05:03.260 +well, I basically don't do any Django work at this point. + +00:05:03.380 --> 00:05:05.700 +My interests have moved away from web stuff generally. + +00:05:06.760 --> 00:05:08.300 +For about the last 10 years, + +00:05:08.390 --> 00:05:10.400 +I have been working on the Beware project, + +00:05:10.700 --> 00:05:14.540 +which is a collection of tools and libraries + +00:05:14.940 --> 00:05:17.620 +to support building graphical user interfaces, + +00:05:18.220 --> 00:05:19.440 +native graphical user interfaces + +00:05:19.440 --> 00:05:20.420 +in Python + +00:05:21.020 --> 00:05:23.640 +and targeting that at desktop platforms, + +00:05:23.650 --> 00:05:25.300 +so Windows, Linux, macOS, + +00:05:25.560 --> 00:05:27.960 +but also mobile platforms, iOS and Android, + +00:05:28.510 --> 00:05:31.120 +the web as a GUI user interface, + +00:05:32.050 --> 00:05:33.680 +and more recently also TextUID, + +00:05:33.680 --> 00:05:34.880 +the console-based applications. + +00:05:35.680 --> 00:05:38.100 +Working on the premise that every platform + +00:05:38.600 --> 00:05:41.920 +has a concept of a native interface. + +00:05:41.990 --> 00:05:43.700 +Now, there is a concept of a button + +00:05:44.240 --> 00:05:45.200 +on all of those platforms. + +00:05:45.740 --> 00:05:48.100 +How you access that button is different on every platform. + +00:05:48.720 --> 00:05:51.340 +And Beware's goal is to, one, + +00:05:51.800 --> 00:05:54.180 +put a clean interface around the outside of that + +00:05:54.100 --> 00:05:55.760 +so you can have a common API for, + +00:05:55.800 --> 00:05:57.160 +I just want a button on the screen + +00:05:57.240 --> 00:05:59.260 +so when I push the button, the light goes Bing. + +00:06:00.100 --> 00:06:02.220 +One, provide a common cross-platform API for that. + +00:06:02.680 --> 00:06:05.300 +And then two, provide the mechanisms to say, + +00:06:05.460 --> 00:06:07.300 +okay, I have written my machine + +00:06:07.460 --> 00:06:08.340 +that goes Bing with a button. + +00:06:08.980 --> 00:06:09.920 +I want to give it to someone + +00:06:10.600 --> 00:06:11.900 +and I want to give it to them + +00:06:12.520 --> 00:06:16.060 +using the platform's app infrastructure. + +00:06:16.280 --> 00:06:18.960 +I just want to have it on this person's iPhone + +00:06:19.340 --> 00:06:21.260 +or give it to a Windows user + +00:06:21.400 --> 00:06:23.280 +and have them just run it + +00:06:23.520 --> 00:06:26.000 +without having to go through, okay, well, first you've got to set up + +00:06:26.000 --> 00:06:28.800 +a Python virtual environment, and then you've got to pip install it. + +00:06:29.780 --> 00:06:31.440 +Are you familiar with the terminal? + +00:06:32.010 --> 00:06:32.940 +Why don't you become + +00:06:32.940 --> 00:06:33.880 +familiar with the terminal? + +00:06:33.970 --> 00:06:34.080 +We'll + +00:06:34.080 --> 00:06:34.740 +start there, yeah. + +00:06:35.220 --> 00:06:39.480 +From my perspective, there are absolutely, as a developer, + +00:06:39.650 --> 00:06:40.400 +I need to know that. + +00:06:40.440 --> 00:06:43.200 +As a Python user, I need to know how that stuff works. + +00:06:43.520 --> 00:06:44.060 +If I have + +00:06:44.060 --> 00:06:44.980 +written a program + +00:06:44.980 --> 00:06:46.560 +and I want to give it to someone, + +00:06:47.020 --> 00:06:50.899 +if they need to know what Python is, I have failed as a developer + +00:06:50.920 --> 00:06:53.560 +because as an end user, they don't care. + +00:06:53.920 --> 00:06:54.000 +They + +00:06:54.000 --> 00:06:55.220 +just want to spool the app + +00:06:55.220 --> 00:06:56.200 +the same way as everything. + +00:06:56.320 --> 00:06:58.680 +They want to double click on an icon and make the thing work. + +00:06:59.420 --> 00:07:00.880 +And that's the other part of + +00:07:00.880 --> 00:07:01.220 +doing it. + +00:07:01.340 --> 00:07:01.700 +That's awesome. + +00:07:01.820 --> 00:07:06.120 +And I would say if it's not possible to have a little colorful, + +00:07:06.760 --> 00:07:09.720 +native-looking icon, either in your dock, in your taskbar, + +00:07:10.040 --> 00:07:12.460 +in your app launcher on your phone that you touch, + +00:07:12.740 --> 00:07:13.660 +and the thing comes to life, + +00:07:14.140 --> 00:07:16.140 +that's not a real app delivery platform. + +00:07:16.660 --> 00:07:19.259 +Yeah, it is a very powerful language + +00:07:19.280 --> 00:07:21.060 +with which you can do all sorts of incredible things. + +00:07:21.300 --> 00:07:22.020 +But at the end of the day, + +00:07:22.200 --> 00:07:24.460 +you've got to give that thing to someone to do it + +00:07:24.820 --> 00:07:27.540 +without giving them a three-year degree in computer science + +00:07:27.720 --> 00:07:29.160 +before they can actually do the damn thing. + +00:07:29.340 --> 00:07:29.580 +Sure. + +00:07:29.660 --> 00:07:32.100 +And even if they can, they might not want to, right? + +00:07:32.440 --> 00:07:32.660 +Yeah, exactly. + +00:07:32.720 --> 00:07:33.440 +I don't want to do that. + +00:07:33.620 --> 00:07:33.980 +It's a barrier. + +00:07:34.480 --> 00:07:35.020 +Yeah, yeah. + +00:07:35.360 --> 00:07:36.180 +I love my web browser. + +00:07:36.320 --> 00:07:37.320 +I don't want to start it that way. + +00:07:37.540 --> 00:07:37.720 +Yeah. + +00:07:39.160 --> 00:07:43.760 +Yeah, so the extension of that then is that particularly for the web, + +00:07:44.900 --> 00:07:46.280 +for iOS and for Android, + +00:07:47.080 --> 00:07:50.720 +there's kind of the foundational question then of, well, okay, + +00:07:50.760 --> 00:07:53.940 +but how do I run Python on that device at all? You + +00:07:53.940 --> 00:07:54.000 +know, + +00:07:54.040 --> 00:07:54.680 +how do I + +00:07:54.680 --> 00:07:56.240 +get Python on my iPhone? + +00:07:57.420 --> 00:08:01.240 +And I don't just want to have like a Python interpreter on my iPhone because + +00:08:01.280 --> 00:08:04.000 +you don't like the, an iPhone, everything is inside an app. + +00:08:04.020 --> 00:08:06.840 +So I need to have it inside my app. How do I get it inside the app? + +00:08:06.840 --> 00:08:07.940 +And then when it's inside the app, + +00:08:08.240 --> 00:08:11.480 +how do I invoke the code that's there and so on and so on. + +00:08:12.000 --> 00:08:16.499 +So a lot of the work that's kind of been leading up to this point is like + +00:08:16.520 --> 00:08:22.620 +literally how to make cpython work on ios how do i make cpython work on android how do i make cpython + +00:08:22.740 --> 00:08:23.900 +work in the browser um + +00:08:23.900 --> 00:08:29.220 +right so yeah that's awesome a very very interesting and powerful goal i think + +00:08:29.520 --> 00:08:35.320 +let's not talk about it just yet but i think it's gonna it's it's really it leaves python open to + +00:08:35.560 --> 00:08:40.360 +being not the default choice even though it's a really great programming default choice language + +00:08:40.880 --> 00:08:44.639 +somebody says i want to i want to get started programming awesome you should check out python + +00:08:44.920 --> 00:08:48.600 +yeah i want to build an app you shouldn't check out python right like you don't want that to be the + +00:08:48.820 --> 00:08:53.100 +the answer right and this path that you're you're making you're hacking away + +00:08:53.100 --> 00:08:53.940 +in the forest forest + +00:08:54.760 --> 00:08:56.180 +yeah exactly one + +00:08:56.180 --> 00:09:00.120 +of the one of the the stimulus for me working on this there was a couple + +00:09:00.260 --> 00:09:04.700 +but one of them is that my son who is now first year university but at the time when i was sort + +00:09:04.700 --> 00:09:11.919 +of starting it was you know late primary school um was starting to look at how to program and the + +00:09:11.940 --> 00:09:16.980 +device that he had at school was an iPad. And he says, okay, well, I'm learning to program. Great. + +00:09:17.300 --> 00:09:22.980 +How do I build an iPad app? Well, you don't. Like, it's complicated. Like, maybe. + +00:09:23.860 --> 00:09:24.880 +Have you heard the story for + +00:09:24.880 --> 00:09:25.880 +it? It's kind of complicated. + +00:09:26.740 --> 00:09:32.020 +It's learning. Then you say, okay, well, but I'm learning Python because Python is a good + +00:09:32.140 --> 00:09:38.760 +language for teaching people how to program. Okay, well, now I want to write that iPad app. + +00:09:39.120 --> 00:09:40.140 +Yeah, but you can't. + +00:09:40.320 --> 00:09:42.840 +So maybe you've got to learn a different language for that. + +00:09:42.850 --> 00:09:43.900 +But I don't want to learn a different language. + +00:09:44.020 --> 00:09:45.800 +I've only barely got my hands around Python. + +00:09:45.980 --> 00:09:48.840 +Why do I have to learn another language just to be able to have an app there? + +00:09:49.280 --> 00:09:55.180 +So there's a running sort of semi-joke that goes around that Python is the second best language for everything. + +00:09:55.860 --> 00:10:00.620 +And part of it being the second best language for everything is that it needs to be able to do all these things, + +00:10:01.260 --> 00:10:07.600 +maybe not as well as Swift on an iPhone or Coughlin on an Android machine or whatever, + +00:10:08.040 --> 00:10:10.520 +but well enough that you can get something going. + +00:10:11.280 --> 00:10:13.560 +Yeah, I would be well happy to say + +00:10:13.700 --> 00:10:15.440 +it takes an extra 5% of my battery + +00:10:15.920 --> 00:10:16.740 +compared to a regular app. + +00:10:16.920 --> 00:10:17.780 +They use a Python app. + +00:10:17.840 --> 00:10:18.260 +I don't care. + +00:10:18.580 --> 00:10:18.880 +It's fine. + +00:10:19.160 --> 00:10:20.440 +But it's not that. + +00:10:20.560 --> 00:10:21.860 +It's possible, impossible. + +00:10:22.660 --> 00:10:22.880 +Right? + +00:10:23.020 --> 00:10:23.800 +That's not a great divide. + +00:10:24.440 --> 00:10:25.740 +All right, before we dive into this though, + +00:10:26.080 --> 00:10:28.040 +you mentioned that you work + +00:10:28.280 --> 00:10:30.280 +on the open source team at Anaconda. + +00:10:30.600 --> 00:10:30.960 +So I want to + +00:10:30.960 --> 00:10:31.240 +give you + +00:10:31.240 --> 00:10:31.500 +a + +00:10:31.500 --> 00:10:31.800 +moment + +00:10:31.920 --> 00:10:33.280 +to talk a little bit about what that is. + +00:10:33.420 --> 00:10:35.799 +I had Peter Wang on the show not too long ago + +00:10:36.220 --> 00:10:37.960 +and he's always super fun to talk to. + +00:10:38.380 --> 00:10:41.400 +And I know that they're putting a lot of support + +00:10:42.120 --> 00:10:42.900 +behind PyScript. + +00:10:43.460 --> 00:10:46.380 +And it sounds like the timing of releasing PyScript + +00:10:46.420 --> 00:10:48.940 +and all that is a little bit aligned with when you joined. + +00:10:49.380 --> 00:10:50.780 +Yeah, so maybe talk to us a bit about all this. + +00:10:51.340 --> 00:10:55.240 +Yeah, so Anaconda took a round of funding + +00:10:56.040 --> 00:10:57.880 +a little over three, almost four years ago, + +00:10:57.980 --> 00:10:58.520 +I think at this point. + +00:10:58.940 --> 00:11:01.880 +And part of that was to sort of really start doubling down + +00:11:02.399 --> 00:11:05.000 +on the contribution of open source. + +00:11:05.280 --> 00:11:14.900 +Anaconda is a company who is built upon tools that are open source, most notably Conda, but that's not the only piece of it. + +00:11:15.740 --> 00:11:16.900 +Conda is an open source project. + +00:11:17.620 --> 00:11:18.760 +Someone needs to maintain it. + +00:11:19.100 --> 00:11:33.620 +Anaconda is able to make money selling this open source product, this whole Python ecosystem, into Fortune 500 companies who need all sorts of legal compliance and standards compliance and security reporting and all that kind of stuff. + +00:11:33.700 --> 00:11:38.900 +that your student doing a high school computer programming degree doesn't care about, but + +00:11:38.920 --> 00:11:39.980 +the big end of town does. + +00:11:40.779 --> 00:11:45.380 +Anaconda is able to make money out of that, but recognizes that the only reason they're + +00:11:45.520 --> 00:11:49.120 +able to make money doing that is that there is this open source foundation, that Python + +00:11:49.320 --> 00:11:51.120 +is an open source ecosystem. + +00:11:51.980 --> 00:11:57.120 +And unless people are maintaining that stuff, Anaconda is not going to have a company in + +00:11:57.120 --> 00:11:59.820 +a couple of years because everyone's going to move away from Python or do other things + +00:11:59.880 --> 00:12:01.320 +or find other ways of solving problems. + +00:12:02.120 --> 00:12:04.440 +And, you know, that ultimately is bad news for Anaconda. + +00:12:05.240 --> 00:12:10.180 +The other side of it, and sort of the reason why PyScript is interesting, why Anaconda + +00:12:10.220 --> 00:12:14.900 +is supporting iOS and Android work, is that there is an opportunity here to just literally + +00:12:15.100 --> 00:12:15.820 +make the pie bigger. + +00:12:16.400 --> 00:12:19.860 +That if Anaconda, you know, if someone can come in and say, oh yeah, I want to write + +00:12:20.000 --> 00:12:24.120 +my, you're at a Fortune 500 company, I want to write my app, I want to write it in Python. + +00:12:24.580 --> 00:12:26.740 +Okay, well now I can do it at iOS and Android. + +00:12:27.100 --> 00:12:32.060 +There's now more users for Anaconda's products as a result of this investment that's been + +00:12:32.080 --> 00:12:34.440 +in iOS and Android to make it possible at all. + +00:12:35.260 --> 00:12:36.840 +It is very much a long-term project. + +00:12:37.000 --> 00:12:38.960 +But like this is iOS and Android stuff. + +00:12:39.340 --> 00:12:42.420 +Conda as a project does not have particularly good support + +00:12:42.490 --> 00:12:43.340 +for iOS and Android. + +00:12:43.880 --> 00:12:46.520 +Eventually, hopefully not that far off, it will. + +00:12:47.320 --> 00:12:49.280 +But you don't get to Conda supporting it + +00:12:49.620 --> 00:12:50.680 +until Python supports it. + +00:12:51.220 --> 00:12:52.540 +And you don't get Python supports it + +00:12:52.550 --> 00:12:55.120 +unless someone sits down and actually does the hard work + +00:12:55.300 --> 00:12:58.200 +to make that happen, which is essentially what I was hired to, + +00:12:58.380 --> 00:12:59.760 +myself and Malcolm, who had to do. + +00:13:00.940 --> 00:13:02.300 +And PyScript fits in a very similar boat. + +00:13:02.460 --> 00:13:05.720 +The web is undeniably a platform that is everywhere + +00:13:06.320 --> 00:13:08.820 +that you can't run Python in, or you couldn't run Python, + +00:13:08.910 --> 00:13:10.420 +or it was very hard to run Python in. + +00:13:10.860 --> 00:13:13.740 +And PyScript as an effort is essentially saying, + +00:13:14.230 --> 00:13:16.600 +okay, let's make it as straightforward as possible + +00:13:16.710 --> 00:13:19.380 +for me to just drop some Python into a browser + +00:13:19.640 --> 00:13:22.200 +and do web user interface stuff + +00:13:22.800 --> 00:13:24.460 +where instead of having to learn JavaScript + +00:13:24.610 --> 00:13:25.800 +and learn a whole new ecosystem + +00:13:25.830 --> 00:13:27.120 +and learn a whole bunch of new set of tools, + +00:13:27.240 --> 00:13:30.580 +I can just use Python and open up. + +00:13:30.920 --> 00:13:36.720 +that whole other ecosystem of places where you can run Python code as another place where you + +00:13:36.730 --> 00:13:37.360 +might want to do things. + +00:13:37.670 --> 00:13:40.140 +Yeah. And to give people a sense of what's possible with PyScript, + +00:13:40.940 --> 00:13:46.920 +it's very, very interesting. On iOS and also on Android, I'm sure I just haven't tried it, + +00:13:47.200 --> 00:13:51.920 +you can create progressive web apps and then say, save this to my home screen. And when it launches, + +00:13:52.200 --> 00:13:55.720 +it looks like a native app, but you can't really tell that it's not a native app. + +00:13:56.040 --> 00:14:04.720 +I put together a sample when PyScript first came out that had an offline or offline capable PyScript, all Python written app. + +00:14:04.800 --> 00:14:07.340 +And it was talking to some APIs out on the Internet. + +00:14:07.520 --> 00:14:12.440 +And even just those early stages, it was like, wow, this really makes things possible. + +00:14:12.740 --> 00:14:21.520 +The challenge is, I mean, this is the big challenge, is people don't discover apps on their mobile devices through progressive web apps, right? + +00:14:21.920 --> 00:14:23.440 +Hence your project, right? + +00:14:23.660 --> 00:14:27.180 +They go to the app store and they click get or buy, + +00:14:27.570 --> 00:14:29.040 +and then it shows up on their device, right? + +00:14:29.160 --> 00:14:31.040 +It's kind of like back to like, + +00:14:31.160 --> 00:14:32.460 +well, if you could create a virtual environment, + +00:14:32.840 --> 00:14:33.640 +you could run this thing. + +00:14:33.760 --> 00:14:35.180 +If you can create a progressive web app, + +00:14:35.400 --> 00:14:36.200 +I'm a huge fan of them, + +00:14:36.540 --> 00:14:36.700 +but + +00:14:36.700 --> 00:14:39.480 +my fandom doesn't change people's + +00:14:39.480 --> 00:14:40.120 +behavior, right? + +00:14:40.880 --> 00:14:41.120 +Sure. + +00:14:41.450 --> 00:14:43.840 +But there is also like to defend my own project here, + +00:14:44.200 --> 00:14:45.880 +like I'm a big fan of PyScript. + +00:14:46.030 --> 00:14:46.760 +I love what they're doing. + +00:14:47.280 --> 00:14:49.780 +There are limits to what you can do in a progressive web app. + +00:14:49.880 --> 00:14:52.440 +Those limits are getting extended further and further out, + +00:14:52.620 --> 00:14:52.640 +But + +00:14:52.640 --> 00:14:53.800 +there are things that + +00:14:53.800 --> 00:15:02.460 +if you want to have access to the entire capabilities of your device directly at the hardware layer, you can't do that in a progressive web app. + +00:15:02.460 --> 00:15:07.420 +You can only do it if you're actually writing to the native APIs and you've got direct hardware access. + +00:15:09.220 --> 00:15:14.600 +The web browser sandbox does place some limits on what you can do. + +00:15:15.000 --> 00:15:15.940 +Do those limits matter? + +00:15:16.540 --> 00:15:17.860 +That's an open question. + +00:15:17.920 --> 00:15:21.520 +there are a lot of things that probably should be progressive web apps that are, you know, + +00:15:21.959 --> 00:15:23.560 +or that could be progressive web apps. + +00:15:24.480 --> 00:15:28.780 +But that doesn't mean that there's no market for native apps or that PyScript is like, + +00:15:29.020 --> 00:15:33.000 +PyScript is eventually going to make, you know, Beware or native apps completely irrelevant. + +00:15:33.180 --> 00:15:33.300 +Because + +00:15:33.300 --> 00:15:33.460 +there is... + +00:15:33.460 --> 00:15:34.420 +No, it absolutely won't. + +00:15:34.720 --> 00:15:34.820 +Yeah. + +00:15:35.140 --> 00:15:38.040 +And the other thing is that they actually do complement each other really well. + +00:15:38.200 --> 00:15:44.720 +Like, Beware does have a PyScript web backend or PyScript-based web backend specifically + +00:15:44.900 --> 00:15:45.180 +so you + +00:15:45.180 --> 00:15:47.340 +can say, okay, I want to build an app. + +00:15:47.400 --> 00:15:49.660 +it has a button and when I press it, the dialog button, + +00:15:49.800 --> 00:15:50.700 +my dialog box pops up. + +00:15:51.140 --> 00:15:54.800 +Nothing about what I just said dictated whether it was a web browser + +00:15:55.140 --> 00:15:57.020 +or an iOS app or a desktop app. + +00:15:57.400 --> 00:16:00.400 +I can render that on all of them and then say, okay, + +00:16:00.540 --> 00:16:02.960 +well, it starts as a progressive web app because that's how I want to do it + +00:16:02.980 --> 00:16:04.380 +and then I'm going to spend a little bit more effort + +00:16:04.420 --> 00:16:07.420 +and actually make it into a native app and get it into the iOS app store. + +00:16:07.940 --> 00:16:12.900 +You can progress along that chain without cutting off any options. + +00:16:13.320 --> 00:16:13.500 +Yeah. + +00:16:13.720 --> 00:16:17.040 +Oh, I think your project, the native app side is critical. + +00:16:17.300 --> 00:16:24.760 +I just think there's also a bunch of people out there that say, well, I'm going to build a front end, interactive front end and have a bunch of Django REST framework or something on the back end. + +00:16:25.220 --> 00:16:25.640 +So I'll use + +00:16:25.640 --> 00:16:27.280 +React or I'll use Vue. + +00:16:27.320 --> 00:16:29.240 +And why isn't the answer Python, right? + +00:16:29.320 --> 00:16:29.660 +It + +00:16:29.660 --> 00:16:30.780 +really, really could + +00:16:30.780 --> 00:16:31.200 +be Python. + +00:16:31.480 --> 00:16:33.840 +That's kind of, I think, where PyScript has a lot of possibilities. + +00:16:35.740 --> 00:16:38.920 +This portion of Talk Python to Me is brought to you by the folks at Posit. + +00:16:39.660 --> 00:16:42.280 +Posit has made a huge investment in the Python community lately. + +00:16:42.740 --> 00:16:46.680 +known originally for RStudio, they've been building down a suite of tools and services + +00:16:46.950 --> 00:16:52.480 +for Team Python. Have you ever thought of all the things that go into a Python data science project? + +00:16:53.200 --> 00:16:57.800 +You need your notebook or IDE, sure. Also a server or cloud environment to run it, + +00:16:58.240 --> 00:17:03.860 +a version of Python, packages, access to your databases, and internal APIs. That's a lot of + +00:17:03.950 --> 00:17:08.579 +setup. And if you change any of these things, when you return to your projects a month down the road, + +00:17:08.819 --> 00:17:13.400 +you might get different results. Wouldn't it be nice to have all of this set up for you in one + +00:17:13.920 --> 00:17:19.000 +easy to access place whenever you want to get work done? That's the goal of Posit Workbench. + +00:17:19.699 --> 00:17:23.959 +Posit Workbench allows data scientists to code in Python within their preferred environment + +00:17:24.439 --> 00:17:29.580 +without an additional strain on IT. It gives data scientists access to all the development + +00:17:29.820 --> 00:17:34.540 +environments they love, including Jupyter Notebooks, JupyterLab, Positron, and VS Code. + +00:17:34.860 --> 00:17:37.320 +and yet it helps ensure reproducibility. + +00:17:37.740 --> 00:17:38.360 +Here's how it works. + +00:17:38.740 --> 00:17:41.040 +You or your IT team set up Posit Workbench + +00:17:41.040 --> 00:17:43.660 +on a powerful, dedicated server within your organization + +00:17:44.360 --> 00:17:45.940 +or on the same cloud service + +00:17:46.200 --> 00:17:48.260 +that is hosting your most important data sources, + +00:17:48.800 --> 00:17:52.440 +such as AWS, SageMaker, Azure, GCP, Kubernetes, + +00:17:53.020 --> 00:17:54.160 +or pretty much anywhere. + +00:17:54.740 --> 00:17:57.180 +There, you create dedicated, pre-configured environments + +00:17:57.440 --> 00:17:58.660 +to run your code and notebooks. + +00:17:59.220 --> 00:18:01.480 +And importantly, you also configure access + +00:18:01.820 --> 00:18:04.300 +to proprietary databases and internal APIs. + +00:18:05.100 --> 00:18:13.900 +When it's time to onboard a new data scientist or start a new project, you just fire it up in Workbench and it's fully configured and ready to go, including on the infrastructure side of things. + +00:18:14.460 --> 00:18:17.300 +All of this is securely administered by your organization. + +00:18:18.100 --> 00:18:23.740 +If you work on a data science team where consistency matters, you owe it to you and your org to check out Posit Workbench. + +00:18:24.060 --> 00:18:29.300 +Visit talkpython.fm/workbench today and get a three-month free trial to see if it's a good fit. + +00:18:29.680 --> 00:18:34.280 +That's talkpython.fm/workbench. The link is in your podcast player's show notes. + +00:18:34.940 --> 00:18:36.940 +Thank you to Posit for supporting Talk Python To Me. + +00:18:38.220 --> 00:18:42.540 +I think that brings us a little bit to one of your keynotes from 2019, where you talked + +00:18:42.740 --> 00:18:44.480 +about Black Swan events, right? + +00:18:44.920 --> 00:18:45.160 +Yes. + +00:18:46.240 --> 00:18:49.480 +It's a little bit of the same story, but maybe give people a sense of this and we can link + +00:18:49.560 --> 00:18:52.160 +to this keynote if they want to explore that side of things more. + +00:18:52.440 --> 00:18:52.700 +Sure. + +00:18:52.990 --> 00:18:53.080 +Yeah. + +00:18:53.200 --> 00:18:58.400 +So the idea behind a Black Swan event was Nicholas... + +00:18:58.740 --> 00:19:00.020 +Nassim Talib, I believe it + +00:19:00.020 --> 00:19:00.240 +was. + +00:19:00.460 --> 00:19:01.260 +Nassim Talib, that's the one. + +00:19:01.290 --> 00:19:01.420 +Yep. + +00:19:01.550 --> 00:19:02.160 +I forget his name. + +00:19:02.860 --> 00:19:06.440 +but he identified this idea of what's called a black swan event. + +00:19:07.600 --> 00:19:10.000 +The reason I use this is I'm from Perth and black swans come from Perth. + +00:19:10.140 --> 00:19:11.480 +So it was an interesting + +00:19:11.480 --> 00:19:12.820 +little tie-in for an international + +00:19:12.820 --> 00:19:13.180 +keynote. + +00:19:13.860 --> 00:19:15.440 +You guys actually have black swans all over? + +00:19:15.580 --> 00:19:15.980 +We have + +00:19:15.980 --> 00:19:16.120 +them. + +00:19:16.220 --> 00:19:19.160 +The state emblem of Western Australia is a black swan. + +00:19:19.220 --> 00:19:20.340 +Like that is literally + +00:19:20.340 --> 00:19:22.440 +on our flag. + +00:19:23.000 --> 00:19:23.600 +So the + +00:19:23.600 --> 00:19:27.000 +theory was that the Dutch bumped into Western Australia + +00:19:27.300 --> 00:19:30.820 +accidentally looking for the Spice Islands in the 1600s. + +00:19:31.220 --> 00:19:36.420 +And at the time they did that, they knew with absolute certainty that all swans were white. + +00:19:37.000 --> 00:19:41.000 +And then they bumped into Western Australia and they discovered that all the swans here are black. + +00:19:42.520 --> 00:19:49.880 +The idea behind a black swan event is a thing you don't see coming that completely changes your view of the world. + +00:19:50.380 --> 00:19:53.800 +And black swan events are the sorts of things that completely change markets. + +00:19:53.900 --> 00:20:04.120 +We don't know what, if you're not looking, if you're not ready for a cataclysmic change in your ecosystem, then the change happens and you are left behind. + +00:20:05.179 --> 00:20:05.700 +I + +00:20:05.700 --> 00:20:06.340 +would say + +00:20:06.340 --> 00:20:09.500 +COVID is a good example of what hit people. + +00:20:10.360 --> 00:20:10.940 +Another one's AI. + +00:20:11.960 --> 00:20:14.820 +Yeah, AI is still, how is that going to impact? + +00:20:14.960 --> 00:20:16.560 +It's just a little bit less clear. + +00:20:16.750 --> 00:20:21.640 +And COVID, we're kind of seeing the impacts of that, whether that's going to have long-term impacts. + +00:20:21.880 --> 00:20:22.380 +like it's some + +00:20:22.380 --> 00:20:23.000 +of those things seem + +00:20:23.000 --> 00:20:23.720 +to be kind of fading away. + +00:20:24.580 --> 00:20:25.360 +But the one that I, + +00:20:25.900 --> 00:20:26.000 +yeah, + +00:20:26.280 --> 00:20:26.360 +well, + +00:20:27.100 --> 00:20:27.180 +yeah, + +00:20:27.600 --> 00:20:27.780 +mostly. + +00:20:28.000 --> 00:20:28.120 +I mean, + +00:20:28.620 --> 00:20:28.820 +people, + +00:20:28.820 --> 00:20:28.960 +I mean, + +00:20:29.080 --> 00:20:29.600 +people still get to say, + +00:20:29.760 --> 00:20:30.260 +but like the, + +00:20:30.620 --> 00:20:33.320 +the world has shut down and + +00:20:33.320 --> 00:20:34.060 +will we, + +00:20:34.360 --> 00:20:35.760 +will economies just disappear? + +00:20:36.320 --> 00:20:36.820 +Level of, + +00:20:36.960 --> 00:20:37.480 +Oh my gosh. + +00:20:37.580 --> 00:20:37.920 +It's kind of, + +00:20:37.980 --> 00:20:39.360 +that's the history I was thinking of. + +00:20:39.460 --> 00:20:39.660 +Yeah. + +00:20:39.880 --> 00:20:39.940 +I, + +00:20:40.260 --> 00:20:41.820 +the one that I was highlighting for, + +00:20:41.960 --> 00:20:42.160 +for, + +00:20:43.160 --> 00:20:45.440 +for Python is that Python is a programming language, + +00:20:46.220 --> 00:20:48.600 +but it's a programming language that has historically assumed, + +00:20:48.720 --> 00:20:48.820 +well, + +00:20:48.880 --> 00:20:49.640 +you're running on a laptop, + +00:20:49.860 --> 00:20:50.000 +right? + +00:20:50.380 --> 00:20:51.580 +Like that's where you run computer code. + +00:20:51.680 --> 00:20:56.740 +you always run computer code in a laptop. And then iPhones turned up. And now we've got computing in + +00:20:56.880 --> 00:21:01.620 +our pocket that is as powerful, if not more powerful than the laptops that we had on our desk + +00:21:01.880 --> 00:21:07.020 +10 years ago. And Python can't run there, or at least at the time, couldn't really run there. + +00:21:07.540 --> 00:21:13.520 +So what does that mean for Python? Why am I going to learn Python? If the computing ecosystem moves + +00:21:13.640 --> 00:21:18.840 +to a world where everybody's using phones and tablets, and I can't run Python on a phone or a + +00:21:18.840 --> 00:21:23.740 +tablet, why am I going to learn Python? Python as an ecosystem has an existential risk there + +00:21:23.850 --> 00:21:28.480 +because of this black swan of a change in the way people do computing. + +00:21:29.140 --> 00:21:30.360 +And at + +00:21:30.360 --> 00:21:31.040 +the time I gave that + +00:21:31.180 --> 00:21:35.460 +keynote, what I was essentially saying is, okay, as an ecosystem, we need to take a long, hard look + +00:21:35.470 --> 00:21:40.680 +at ourselves. It's impossible to know what the black swan events are going to be, but we can look + +00:21:40.780 --> 00:21:44.739 +at where the ecosystem is now. We can look at the trends of things that are happening and we can work + +00:21:44.760 --> 00:21:52.520 +out institutionally what are our risks, like what would need to change to make Python a + +00:21:52.720 --> 00:21:53.440 +dead ecosystem. + +00:21:54.320 --> 00:21:59.900 +And the obvious one, the one that I was really pushing at that time, one of which is platforms + +00:22:00.000 --> 00:22:00.700 +that we need to support. + +00:22:00.820 --> 00:22:04.280 +And the second is how do we support this software going forward? + +00:22:04.420 --> 00:22:09.740 +How do we actually guarantee that this community-driven project continues to be maintained? + +00:22:10.080 --> 00:22:13.320 +Because if we don't have maintainers, then the code doesn't get written. + +00:22:14.500 --> 00:22:14.800 +So, yeah. + +00:22:15.000 --> 00:22:15.660 +Yeah, excellent. + +00:22:15.900 --> 00:22:17.060 +I recommend people check it out. + +00:22:17.280 --> 00:22:17.720 +It's on YouTube. + +00:22:17.920 --> 00:22:19.040 +I will link to it. + +00:22:19.580 --> 00:22:20.740 +It's scary the last five years ago. + +00:22:21.540 --> 00:22:21.940 +I know. + +00:22:22.140 --> 00:22:24.520 +I feel like that was pretty recent, but no. + +00:22:25.500 --> 00:22:28.740 +No, that was the last one before COVID, by the way, bringing it full circle. + +00:22:29.500 --> 00:22:35.300 +Tommy out there asked in the audience, is in the iOS Pythonistas app, I had some really good simple use cases in the past. + +00:22:35.600 --> 00:22:42.380 +The GUI layer, a bit of a hassle, but like the library modules, also could not call some not yet exposed APIs. + +00:22:42.460 --> 00:22:47.360 +I guess as we get into this, maybe think, just people probably know about the Pythonista app. + +00:22:47.680 --> 00:22:48.720 +I don't know. + +00:22:49.560 --> 00:22:50.520 +Yeah, it was around. + +00:22:51.460 --> 00:22:52.560 +And it's not the same idea. + +00:22:52.920 --> 00:22:54.120 +It's not what you guys are trying to do. + +00:22:54.500 --> 00:22:57.300 +Yes and no, because I actually worked on Pythonista. + +00:22:57.660 --> 00:23:02.200 +The reason that Pythonista supports Python 3, I was contracted to do the Python 3 port. + +00:23:02.380 --> 00:23:02.600 +So the + +00:23:02.600 --> 00:23:03.780 +core of that library + +00:23:03.780 --> 00:23:05.900 +is essentially the thing that Beware was doing. + +00:23:05.960 --> 00:23:09.000 +And the reason it was able to move to Python 3 was because of the work that I was doing. + +00:23:10.220 --> 00:23:17.940 +It is, yeah, it is very much sort of the Pythonista as an idea is kind of what if idle, but on a phone? + +00:23:18.200 --> 00:23:22.640 +Like here is a development environment where you can write Python code. + +00:23:22.680 --> 00:23:26.720 +You have some access to some libraries and some GUI libraries. + +00:23:27.540 --> 00:23:33.520 +So you can put a button up, but you're not writing, you're not using Pythonista to write an app. + +00:23:33.780 --> 00:23:40.380 +You're using it to put an interface into a running Pythonista sheet, you know, working + +00:23:40.660 --> 00:23:40.880 +environment. + +00:23:41.940 --> 00:23:44.120 +It is absolutely a way to write Python. + +00:23:44.190 --> 00:23:45.940 +It is a way to have + +00:23:45.940 --> 00:23:47.100 +Python running + +00:23:47.100 --> 00:23:47.900 +on your phone. + +00:23:48.450 --> 00:23:52.820 +But it is not something that's going to push something to the iOS app store so that random + +00:23:52.930 --> 00:23:54.660 +other person can buy your app. + +00:23:55.040 --> 00:23:59.620 +You know, to me, it feels a little bit like shortcuts on iOS in that you could kind of + +00:23:59.620 --> 00:24:02.560 +put together some behaviors, but you would never try to make an app. + +00:24:02.660 --> 00:24:07.100 +yeah that's it's not that's not too far from it and it except that it's all inside the sound + +00:24:07.690 --> 00:24:12.320 +yeah and you could do real python which is way better than shortcut but the kind of the type + +00:24:12.320 --> 00:24:16.440 +of thing you accomplish it feels a little bit like i can go find this thing and run it and it'll do + +00:24:16.560 --> 00:24:17.580 +something for me which is cool + +00:24:17.580 --> 00:24:17.780 +but + +00:24:17.780 --> 00:24:18.440 +yeah it's + +00:24:18.440 --> 00:24:22.400 +not going to be the new airbnb implementation no + +00:24:22.400 --> 00:24:22.720 +exactly + +00:24:23.020 --> 00:24:29.959 +and like even like i guess beware as a project isn't it the intention is not that it would be + +00:24:29.980 --> 00:24:33.080 +the next Airbnb as a final product. + +00:24:33.270 --> 00:24:36.940 +It might, however, be the mock-up of the demo app + +00:24:37.080 --> 00:24:40.020 +that you take to your initial seed funding round + +00:24:40.350 --> 00:24:43.080 +to prove, here is the idea we've got. + +00:24:43.380 --> 00:24:44.580 +We've proved it out with some people, + +00:24:44.670 --> 00:24:46.480 +just like a small little sample trial, whatever. + +00:24:47.180 --> 00:24:49.060 +Now give us the money to go and actually hire + +00:24:49.420 --> 00:24:51.720 +a team of iOS developers and a team of Android developers + +00:24:51.770 --> 00:24:53.480 +to build an absolutely kick-ass, + +00:24:54.500 --> 00:24:56.540 +optimized native application of every platform. + +00:24:57.240 --> 00:24:59.940 +Yeah, maybe a little bit like Playgrounds + +00:24:59.960 --> 00:25:01.380 +Well, perhaps in that regard. + +00:25:02.060 --> 00:25:03.440 +Yeah, a little bit, I guess, like Playgrounds, yeah. + +00:25:04.320 --> 00:25:05.600 +Yeah, also a different focus. + +00:25:05.840 --> 00:25:09.500 +But another comment maybe as we dive into this, Joe out there asks us, + +00:25:09.530 --> 00:25:10.260 +hey, great project. + +00:25:10.470 --> 00:25:12.020 +How would you sell this to us? + +00:25:12.400 --> 00:25:14.880 +And that there's a few other frameworks like WXPython, Kivy. + +00:25:15.720 --> 00:25:17.260 +WXPython also renders data of GUIs. + +00:25:18.960 --> 00:25:19.880 +How would I sell it? + +00:25:20.010 --> 00:25:22.480 +Okay, says he, pulling out the sales sheet. + +00:25:23.480 --> 00:25:24.600 +Let me get my pitch deck. + +00:25:24.710 --> 00:25:25.020 +Hold on. + +00:25:25.960 --> 00:25:27.640 +So there's essentially two questions there. + +00:25:27.740 --> 00:25:30.500 +One is WX Python and one is KIVI. + +00:25:31.780 --> 00:25:35.380 +The KIVI is a sort of the two different reasons. + +00:25:35.740 --> 00:25:38.980 +KIVI does work on iOS and Android and also on the desktop platforms. + +00:25:39.580 --> 00:25:43.540 +KIVI is taking a very, very distinctly different approach to user interface design. + +00:25:44.030 --> 00:25:49.560 +So KIVI can run on all those platforms, but the user interface that it puts up is a KIVI + +00:25:49.630 --> 00:25:50.060 +user interface. + +00:25:50.390 --> 00:25:51.580 +It always looks like KIVI. + +00:25:51.580 --> 00:25:55.840 +And it's like if you build a macOS app, it will look the same on Windows. + +00:25:56.100 --> 00:25:58.420 +It'll look the same on GDK, and it will look the same, you know, + +00:25:58.700 --> 00:25:59.720 +size of window notwithstanding. + +00:26:00.040 --> 00:26:01.600 +It will look the same on iOS and Android. + +00:26:02.400 --> 00:26:05.480 +That is a completely viable way to build apps. + +00:26:06.020 --> 00:26:08.600 +It is not a native user interface. + +00:26:10.400 --> 00:26:14.620 +And one of the major reasons that I dislike that as an approach, + +00:26:14.980 --> 00:26:17.300 +I'm not in any way saying that Kibbe is a bad project. + +00:26:17.480 --> 00:26:19.100 +It is a different way to build user interfaces, + +00:26:19.480 --> 00:26:23.420 +and I prefer a different approach, which is to say an actual native button. + +00:26:23.620 --> 00:26:26.260 +So on macOS, it looks like a macOS app. + +00:26:26.480 --> 00:26:28.540 +On iOS, it looks like an iOS app. + +00:26:28.620 --> 00:26:30.520 +On Windows, it looks like a Windows app. + +00:26:32.020 --> 00:26:34.800 +The downside is that if you need to take screenshots, + +00:26:35.080 --> 00:26:36.120 +you've got to have an iOS screenshot + +00:26:36.280 --> 00:26:38.180 +and a macOS screenshot and a Windows screenshot. + +00:26:39.340 --> 00:26:42.660 +The benefit is consistency with the rest of the platform. + +00:26:43.460 --> 00:26:46.180 +And that might seem like a totally cosmetic thing, + +00:26:46.800 --> 00:26:49.660 +but one, consistency between applications + +00:26:50.180 --> 00:26:52.179 +is a mode of user interface discovery + +00:26:52.200 --> 00:26:55.860 +because if people know this is how a button works on MacOS, + +00:26:56.180 --> 00:26:58.100 +they know how your platform does it as well, + +00:26:58.100 --> 00:26:59.000 +how your app does it as well, + +00:26:59.300 --> 00:26:59.780 +rather than + +00:26:59.780 --> 00:27:01.820 +having to discover how you have decided + +00:27:01.820 --> 00:27:04.600 +to implement menus or buttons or anything else. + +00:27:05.140 --> 00:27:06.960 +Yeah, one of the really things that stands out, + +00:27:06.960 --> 00:27:10.620 +I think, super stark a lot of times is the file open save dialogs. + +00:27:10.640 --> 00:27:10.900 +Yes, + +00:27:11.410 --> 00:27:11.540 +yes. + +00:27:11.600 --> 00:27:11.940 +It's like, + +00:27:12.000 --> 00:27:15.460 +whoa, this is clearly just alien versus... + +00:27:15.480 --> 00:27:15.740 +Yeah, and + +00:27:15.740 --> 00:27:16.700 +a lot of cases, + +00:27:17.160 --> 00:27:19.799 +they are missing the one thing on my platform + +00:27:19.820 --> 00:27:22.120 +that makes file dialogues actually useful + +00:27:22.600 --> 00:27:22.720 +because + +00:27:22.720 --> 00:27:23.340 +I can do + +00:27:23.340 --> 00:27:25.680 +this thing that I can't do in your dialogue + +00:27:26.100 --> 00:27:27.120 +because you've had to build it from scratch. + +00:27:27.260 --> 00:27:28.520 +And speaking of building from scratch, + +00:27:28.620 --> 00:27:30.700 +the other big benefit of a native GUI + +00:27:31.140 --> 00:27:33.940 +is that you get all of the platform's native affordances. + +00:27:34.660 --> 00:27:36.320 +One of the most surprising things, + +00:27:36.460 --> 00:27:37.720 +it wasn't necessarily surprising + +00:27:37.860 --> 00:27:39.460 +because I understood why I was doing it, + +00:27:39.560 --> 00:27:41.000 +but things that might be surprising to people + +00:27:41.040 --> 00:27:42.520 +who haven't followed along GUI development, + +00:27:43.400 --> 00:27:45.740 +we have received an inordinate number + +00:27:46.240 --> 00:27:48.379 +of very, very enthusiastic comments + +00:27:48.380 --> 00:27:51.600 +from blind people writing iOS apps. + +00:27:52.410 --> 00:27:54.540 +Because an iOS app built with Beware + +00:27:55.140 --> 00:27:57.880 +fits in directly with screen readers + +00:27:58.380 --> 00:28:01.200 +because it is using the platform's native accessibility. + +00:28:02.000 --> 00:28:04.440 +Kivy has not built explicitly + +00:28:05.090 --> 00:28:06.720 +native screen reader support into their widgets. + +00:28:06.930 --> 00:28:08.100 +To the best of my knowledge, + +00:28:08.150 --> 00:28:09.760 +or at least all the reports that I'm getting + +00:28:09.760 --> 00:28:10.920 +are saying that it's not there + +00:28:11.280 --> 00:28:13.260 +because you have to build it from scratch + +00:28:13.900 --> 00:28:16.120 +on every platform and they haven't done that. + +00:28:16.360 --> 00:28:17.160 +It's a lot of work. + +00:28:17.220 --> 00:28:19.360 +I'm not underselling how big of a project it is, + +00:28:19.520 --> 00:28:23.720 +but Beware got it for free because we are using native widgets, + +00:28:24.300 --> 00:28:26.000 +and that's not an insignificant consideration. + +00:28:28.020 --> 00:28:28.420 +That's + +00:28:28.420 --> 00:28:28.720 +Kivy. + +00:28:28.780 --> 00:28:29.020 +Oh, sorry. + +00:28:29.560 --> 00:28:29.960 +No, no, please + +00:28:29.960 --> 00:28:30.220 +go ahead. + +00:28:30.460 --> 00:28:31.980 +No, I was going to say, that's the Kivy. + +00:28:32.280 --> 00:28:35.320 +So that's the not native user interface but looks the same everywhere + +00:28:35.660 --> 00:28:38.240 +versus Beware's using native widgets everywhere. + +00:28:38.600 --> 00:28:40.880 +The other argument is, okay, but why don't you use WX Windows? + +00:28:41.320 --> 00:28:44.579 +The reason we don't use WX Windows is WX Windows doesn't run on iOS + +00:28:44.600 --> 00:28:47.460 +or Android and trying to make it run on iOS and Android. + +00:28:47.710 --> 00:28:49.540 +Like I did look into it and it was a little bit crunchy. + +00:28:50.740 --> 00:28:53.480 +Qt apparently works on iOS now. + +00:28:53.550 --> 00:28:54.900 +It didn't when I started the project, + +00:28:56.240 --> 00:28:58.280 +but they very much don't want you to write Python. + +00:28:58.350 --> 00:29:00.040 +And one of the things that Beware is trying to push + +00:29:00.080 --> 00:29:01.900 +and Toga as a GUI platform is trying to push + +00:29:01.950 --> 00:29:03.740 +is you're writing Python code. + +00:29:03.810 --> 00:29:04.580 +It looks like Python. + +00:29:04.770 --> 00:29:05.840 +It tastes like Python. + +00:29:06.320 --> 00:29:07.960 +If you don't want to write Python, that's fine. + +00:29:08.160 --> 00:29:09.580 +There are plenty of GUI toolkits out there + +00:29:09.590 --> 00:29:11.200 +that don't require you to write Python. + +00:29:11.590 --> 00:29:14.180 +But Toga is a Python first GUI toolkit. + +00:29:14.280 --> 00:29:24.900 +And it looks like, tastes like Python, we support, we use, actively use in APIs, iterators and generators and asyncio syntax and context managers. + +00:29:25.520 --> 00:29:37.220 +Things that WX Windows as a light wrapper around a C++ framework doesn't really expose because they are limited by what C++ looks like when you program it. + +00:29:37.620 --> 00:29:41.279 +And you also get that to a certain extent in QT as well or PyQT. + +00:29:42.120 --> 00:29:42.920 +Yeah, they're both wrappers. + +00:29:43.420 --> 00:29:43.580 +Interesting. + +00:29:44.200 --> 00:29:48.380 +So let's talk about some of the peps that make this possible, right? + +00:29:48.740 --> 00:29:48.840 +Yeah. + +00:29:49.020 --> 00:29:55.860 +There's the foundational, basically adding platforms as a intentionally supported platform + +00:29:56.460 --> 00:29:58.920 +to the CPython whole process, right? + +00:29:59.020 --> 00:29:59.340 +The core + +00:29:59.340 --> 00:30:00.140 +developers working + +00:30:00.140 --> 00:30:03.340 +on it from the builds and the deployments and all those kinds + +00:30:03.380 --> 00:30:03.680 +of things. + +00:30:03.740 --> 00:30:08.200 +And so you've got a couple of peps where you're the author of them, which is awesome. + +00:30:08.400 --> 00:30:10.180 +I'm strictly the author of one. + +00:30:10.280 --> 00:30:11.720 +Malcolm Smith was the author of the Android one. + +00:30:11.920 --> 00:30:12.240 +Oh, okay. + +00:30:12.660 --> 00:30:12.800 +Got it. + +00:30:13.060 --> 00:30:16.280 +One Android with Malcolm and one iOS with you. + +00:30:16.390 --> 00:30:16.480 +Okay. + +00:30:16.670 --> 00:30:18.220 +I see which side fence you come down on. + +00:30:18.230 --> 00:30:18.620 +I got it. + +00:30:18.820 --> 00:30:18.960 +Yep. + +00:30:19.180 --> 00:30:19.560 +No, no, no, no. + +00:30:20.160 --> 00:30:20.580 +That's all good. + +00:30:20.700 --> 00:30:21.200 +I'm with you. + +00:30:21.240 --> 00:30:21.560 +I'm with you. + +00:30:22.100 --> 00:30:22.240 +All right. + +00:30:22.570 --> 00:30:23.500 +Tell us about these peps. + +00:30:23.960 --> 00:30:24.980 +What role do they have? + +00:30:25.640 --> 00:30:27.140 +And their status is final. + +00:30:27.380 --> 00:30:28.200 +That's good. + +00:30:29.100 --> 00:30:29.460 +So, yeah. + +00:30:29.910 --> 00:30:36.400 +So, the PEP was essentially formalizing the things that Malcolm and I have been maintaining + +00:30:36.700 --> 00:30:40.040 +independently, both of us, at the time for like about eight, nine years. + +00:30:41.280 --> 00:30:41.900 +We had patches. + +00:30:42.340 --> 00:30:46.200 +like I was maintaining a set of patches to make CPython run on iOS, + +00:30:46.360 --> 00:30:48.060 +and I was manually maintaining all those patches. + +00:30:48.160 --> 00:30:50.280 +And every time a new Python release would come out, + +00:30:50.440 --> 00:30:52.980 +I'd have to go and sit down and spend, you know, + +00:30:53.100 --> 00:30:54.420 +a couple of months trying to work out, okay, + +00:30:54.620 --> 00:30:55.620 +what have they changed this time? + +00:30:56.300 --> 00:30:59.840 +And, you know, getting all of those, updating all of the PR, + +00:31:00.000 --> 00:31:03.920 +all of the code to make it work on a new version of Python. + +00:31:04.960 --> 00:31:11.260 +In the 3.13 timeframe, so it's, what, September of 2023, + +00:31:12.080 --> 00:31:17.860 +I went along to the CPython core team summit in the US. + +00:31:18.460 --> 00:31:20.700 +Sorry, it was in Bono. + +00:31:22.580 --> 00:31:24.180 +And basically with the intention of, + +00:31:24.520 --> 00:31:26.060 +I want to upstream all these patches. + +00:31:26.240 --> 00:31:26.840 +We've got these patches. + +00:31:27.000 --> 00:31:27.580 +They're not huge. + +00:31:27.680 --> 00:31:28.960 +They're big, but they're not huge. + +00:31:29.300 --> 00:31:30.300 +I'd like to formalize that. + +00:31:30.460 --> 00:31:32.320 +And so the outcome of that was what we need is a PEP. + +00:31:32.420 --> 00:31:35.520 +We need a formal description of what it is you're going to do, + +00:31:35.800 --> 00:31:36.620 +why it's going to change, + +00:31:37.220 --> 00:31:40.600 +what impact that has on the rest of the ecosystem for you to do that, + +00:31:40.980 --> 00:31:44.480 +and to formalize a couple of the little details + +00:31:45.159 --> 00:31:47.660 +that need to be formally specified + +00:31:47.840 --> 00:31:49.120 +in order for the rest of the ecosystem + +00:31:49.160 --> 00:31:49.960 +to play nice with this. + +00:31:50.240 --> 00:31:50.920 +Little details like, + +00:31:51.200 --> 00:31:54.300 +what does sys.platform return on an iOS machine? + +00:31:54.580 --> 00:31:56.240 +Like it's a really minor detail, + +00:31:56.600 --> 00:31:56.740 +but + +00:31:56.740 --> 00:31:57.460 +it's a question you've + +00:31:57.460 --> 00:31:57.880 +got to answer. + +00:31:58.400 --> 00:31:59.520 +And it's like, it's an answer that even Apple Darwin + +00:31:59.520 --> 00:31:59.780 +is probably + +00:31:59.780 --> 00:32:00.200 +wrong. + +00:32:01.060 --> 00:32:01.160 +Well, + +00:32:01.160 --> 00:32:02.440 +yeah, that's the thing. Like it's + +00:32:02.440 --> 00:32:05.420 +Darwin on macOS for some historical reasons, but like even + +00:32:05.420 --> 00:32:07.400 +on iOS, should it be iOS? Should it be + +00:32:07.400 --> 00:32:09.740 +iOS capitalized? Should it be + +00:32:09.740 --> 00:32:10.460 +iPhone? + +00:32:10.840 --> 00:32:11.800 +Should it be iPhone OS? + +00:32:12.140 --> 00:32:14.080 +Should it be iPhone OS or iPhone simulator? + +00:32:14.440 --> 00:32:18.020 +All of these are answers that Apple itself uses at various points in their API. + +00:32:18.600 --> 00:32:19.360 +And what about iPad? + +00:32:20.600 --> 00:32:21.820 +Well, an iPad is iPad iOS. + +00:32:22.080 --> 00:32:22.680 +Is that a different answer? + +00:32:23.160 --> 00:32:23.980 +And so on. + +00:32:24.900 --> 00:32:32.780 +And then the one that really matters for that is nailing down what is the platform tag that will be used for packaging purposes? + +00:32:32.960 --> 00:32:38.340 +So when a wheel, a binary wheel, eventually lands on PyPI, what's the tag? + +00:32:38.400 --> 00:32:39.780 +Like what is it? + +00:32:40.480 --> 00:32:44.320 +For macOS, it's a macOS X underscore 11 underscore ARM64. + +00:32:44.900 --> 00:32:44.980 +Okay. + +00:32:45.400 --> 00:32:46.120 +What is it for iPhone? + +00:32:46.520 --> 00:32:47.320 +Yeah, maybe we could elaborate. + +00:32:47.520 --> 00:32:50.460 +Maybe you could elaborate just a little bit on like what that even means, right? + +00:32:50.640 --> 00:32:52.880 +Because before Wheels, things were different. + +00:32:52.960 --> 00:32:53.600 +You would download it. + +00:32:53.600 --> 00:32:54.160 +You would build it. + +00:32:54.200 --> 00:32:56.080 +You'd have to have Fortran or some random thing. + +00:32:56.500 --> 00:33:00.500 +Wheels solved that, but that then created a combinatorial explosion of binary assets, + +00:33:00.740 --> 00:33:00.880 +right? + +00:33:01.200 --> 00:33:01.740 +Why does that matter? + +00:33:02.040 --> 00:33:02.640 +Why does that matter? + +00:33:02.660 --> 00:33:14.240 +So the major change with wheels is that it moved from a world where installing a Python package required you to execute code at time of installation. + +00:33:15.840 --> 00:33:25.660 +And that's not a good idea from a security perspective because you then need to audit all of the code that's going to execute when you install the package. + +00:33:26.220 --> 00:33:33.160 +And you can't find anything out about the package without running that code or at least auditing the code that's going to run. + +00:33:34.000 --> 00:33:43.540 +Wheels provides you to say, okay, this is a wheel is basically just a zip file with a bunch of known metadata that describes exactly what is in here. + +00:33:44.000 --> 00:33:49.780 +But part of that specification is also a binary specification that says, what version of CPython is it compatible with? + +00:33:50.040 --> 00:33:52.300 +What version of the operating system is it compatible with? + +00:33:52.700 --> 00:33:54.880 +What CPU architecture is it compatible with? + +00:33:55.240 --> 00:34:06.820 +So that if I look at a wheel that is labeled, for example, macOS 11.0 ARM64, I know that this will run on macOS 11 with an ARM64 CPU. + +00:34:07.500 --> 00:34:09.659 +There are any number of these tags available. + +00:34:09.780 --> 00:34:15.300 +And there's differences like the Universal 2, which will support both x86 and ARM. + +00:34:15.740 --> 00:34:33.980 +And on Linux, there's a thing called a mini Linux wheel, which is a specification for specifically what API guarantees does this Linux platform guarantee in terms of what is available in the CPython, in terms of the C library, the GNU C library, et cetera, et cetera. + +00:34:34.240 --> 00:34:39.780 +You have to be able to specify that so that I know when I get this wheel, I know where it will work and how it will work. + +00:34:39.840 --> 00:34:43.740 +And for the point of view of PIP, pip knowing which version should I download? + +00:34:44.379 --> 00:34:46.980 +I've got an app here that I'm running on my macOS machine. + +00:34:47.520 --> 00:34:48.720 +Which wheel should I download? + +00:34:49.179 --> 00:34:52.120 +Go and grab one and resolve it and find out and solve that question. + +00:34:52.399 --> 00:34:52.600 +Yeah. + +00:34:52.860 --> 00:34:56.760 +So without knowing that answer, you don't know how to install wheels, basically. + +00:34:57.220 --> 00:34:57.360 +Correct. + +00:34:57.460 --> 00:35:12.260 +Part of the wheel specification is which – if the wheel contains all the pre-compiled components, in order for it to have pre-compiled components, there's an API or an ABI specification there of exactly what symbols am I going to be able to link against? + +00:35:12.290 --> 00:35:13.600 +What can I assume exists? + +00:35:14.420 --> 00:35:15.960 +Will this run on my machine? + +00:35:16.980 --> 00:35:17.240 +Yeah. + +00:35:17.840 --> 00:35:18.920 +What's going to happen, right? + +00:35:19.170 --> 00:35:20.520 +Is it even compiled from architecture? + +00:35:21.040 --> 00:35:26.260 +And we have a similar thing with Python standalone builds that is now part of Astral, which is really great. + +00:35:26.320 --> 00:35:30.360 +we can just grab the Python we need because we know the platform tags and all that kind of stuff. + +00:35:30.680 --> 00:35:33.220 +Any difference about the Android one? + +00:35:33.420 --> 00:35:36.240 +I mean, you obviously need one for each platform, I suppose. + +00:35:36.760 --> 00:35:37.640 +We do, yeah, basically. + +00:35:38.040 --> 00:35:40.940 +And there will be at some point in the hopefully not too distant future, + +00:35:40.960 --> 00:35:44.760 +there will be one for Inscripten as well, just formalizing Inscripten support, + +00:35:45.200 --> 00:35:49.100 +which is the Pyodide slash PyScript in the browser version. + +00:35:50.480 --> 00:35:52.780 +Yeah, you need to have one for every platform. + +00:35:53.120 --> 00:35:59.880 +I guess the one notable difference between iOS and Android as a PEP is that in theory, + +00:36:00.960 --> 00:36:04.020 +CPython 12 could compile on Android. + +00:36:04.100 --> 00:36:08.320 +There were some patches which had been applied without going through the formal PEP process + +00:36:08.800 --> 00:36:13.500 +on the basis of, well, it doesn't cost us anything to add this code in. + +00:36:13.500 --> 00:36:16.120 +We're not guaranteeing it's going to work, but it's not breaking anything. + +00:36:16.160 --> 00:36:17.240 +So we might as well have it there. + +00:36:17.840 --> 00:36:24.760 +And so there's one, like the process of formalizing it did change one notable thing, + +00:36:25.300 --> 00:36:29.800 +which is that previously, if you were able to get Sys.platform to compile on Android, + +00:36:29.840 --> 00:36:33.120 +and it was possible and realistically it needed some patches, but it could be done, + +00:36:33.960 --> 00:36:36.180 +Sys.platform would return Linux. + +00:36:37.180 --> 00:36:42.740 +Because technically it is, but not in a way that's even remotely helpful to anyone. + +00:36:43.760 --> 00:36:44.020 +And so + +00:36:44.020 --> 00:36:46.120 +one of the things that the PEP specified, + +00:36:46.270 --> 00:36:47.820 +like, yes, it is a Linux kernel, + +00:36:48.260 --> 00:36:49.700 +but Linux doesn't help you there + +00:36:49.960 --> 00:36:51.920 +because nothing about an Android machine + +00:36:52.520 --> 00:36:53.720 +actually behaves like Linux + +00:36:54.500 --> 00:36:56.780 +once you're past like the kernel level. + +00:36:57.520 --> 00:36:59.380 +The rest of the thing is what you actually care about. + +00:36:59.850 --> 00:37:01.400 +And the rest of the libraries you can guarantee + +00:37:01.600 --> 00:37:02.260 +is what you care about. + +00:37:02.680 --> 00:37:05.880 +And so one of the things that the PEP had to specify + +00:37:06.640 --> 00:37:10.560 +was, okay, we are changing what sys.platform means on Android + +00:37:10.980 --> 00:37:13.720 +because you need to be able to differentiate + +00:37:13.740 --> 00:37:17.060 +I am on an Android machine, not just a desktop Linux box. + +00:37:18.180 --> 00:37:18.340 +And + +00:37:18.340 --> 00:37:18.580 +so, yeah, + +00:37:18.680 --> 00:37:20.360 +so that was one notable change. + +00:37:20.640 --> 00:37:22.880 +It was in that PEP that wasn't in the iOS one. + +00:37:23.220 --> 00:37:23.420 +Interesting. + +00:37:23.700 --> 00:37:25.400 +You mentioned mScripten. + +00:37:25.820 --> 00:37:25.920 +What + +00:37:25.920 --> 00:37:26.900 +an interesting project. + +00:37:27.320 --> 00:37:28.820 +Yeah, it almost + +00:37:28.820 --> 00:37:29.500 +looks like it's + +00:37:29.500 --> 00:37:31.580 +not AI because it's LLVM, + +00:37:32.000 --> 00:37:32.980 +but it has nothing to do with it. + +00:37:33.060 --> 00:37:33.300 +Yeah, yes. + +00:37:33.700 --> 00:37:36.040 +It's a different set of LLs altogether. + +00:37:37.060 --> 00:37:37.160 +Yeah. + +00:37:37.920 --> 00:37:47.180 +So inscription is kind of the tail end, the current manifestation of a long series of + +00:37:47.280 --> 00:37:49.760 +developments that started like 15 years ago. + +00:37:50.300 --> 00:37:58.980 +Someone worked out that modern web browsers have spent so much time optimizing the JavaScript + +00:37:59.260 --> 00:38:06.060 +compilers that there are certain JavaScript constructs that map almost one-to-one to machine + +00:38:06.300 --> 00:38:06.480 +code. + +00:38:06.780 --> 00:38:21.180 +Like you can say, okay, add these two integers in JavaScript or to these two numbers in JavaScript that if you know the numbers are integers, will map through the JavaScript jits that are running. + +00:38:21.250 --> 00:38:24.580 +It will map to a literal, you know, add instruction + +00:38:24.580 --> 00:38:25.260 +on your CPU. + +00:38:25.540 --> 00:38:26.840 +Like two registers coming together. + +00:38:27.010 --> 00:38:27.760 +Yeah, two registers + +00:38:27.760 --> 00:38:28.200 +coming together. + +00:38:28.900 --> 00:38:40.120 +And so if you restrict yourself to handwriting just that subset of JavaScript, you can essentially write assembler in JavaScript. + +00:38:40.880 --> 00:38:51.640 +Now, in practice, nobody can actually do that because it's just way too constraining to know that you actually have to put the brackets around the outside of here or else the jit won't optimize it in just the right way. + +00:38:52.140 --> 00:38:53.380 +But that's what compilers are for. + +00:38:53.760 --> 00:38:59.340 +Compilers are really good at taking a set of arbitrary input and following a set of arbitrary rules to produce a consistent set of output. + +00:38:59.640 --> 00:39:10.140 +And so this thing called ASM.js was born, which is essentially an input format that could be compiled to a JIT-able version of JavaScript that would run as a native platform. + +00:39:10.740 --> 00:39:18.340 +And then from that, you can then get to WASM, which is WebAssembly, which is a formal text specification of the sort of things that ASM.js was working on. + +00:39:18.760 --> 00:39:23.740 +And then Inscripten is a compiler tool chain that plugs into LLVM. + +00:39:23.740 --> 00:39:33.640 +So if you've used a compiler like Clang, it is essentially a wrapper around a set of compiler building tools that produces output. + +00:39:34.000 --> 00:39:39.040 +So you are used to calling GCC or Clang on a C program. + +00:39:39.900 --> 00:39:48.600 +Clang calls into your C program, compiles it into an LLVM format, and LLVM converts it into actual runnable executables. + +00:39:49.140 --> 00:39:59.800 +And so it's an LLVM plugin that instead of spitting out macOS-compatible binary executable assembly, it spits out WASM. + +00:39:59.940 --> 00:40:03.420 +It spits out JavaScript-optimized code that can run in the browser. + +00:40:03.860 --> 00:40:05.620 +That's all very much high level. + +00:40:05.760 --> 00:40:07.960 +There's a lot of very, very complicated details that go on in the internals. + +00:40:08.140 --> 00:40:14.120 +But the end point is that Enscripten lets you take any arbitrary C code and compile it to + +00:40:14.720 --> 00:40:18.800 +JavaScript that will run in a browser at nearish native speed. + +00:40:19.920 --> 00:40:22.060 +Good news is that CPython is written in C. + +00:40:22.390 --> 00:40:27.720 +And so you can compile CPython into a JavaScript version or into this JavaScript subset, and + +00:40:27.770 --> 00:40:28.880 +it will run in the browser. + +00:40:29.260 --> 00:40:31.060 +There's a bunch of other stuff you need to do as well. + +00:40:31.110 --> 00:40:35.100 +And Enscripten is providing a bunch of other things like guaranteeing a bunch of Unix system + +00:40:35.300 --> 00:40:35.840 +calls are available. + +00:40:36.440 --> 00:40:38.000 +But broadly, that's the idea here. + +00:40:38.000 --> 00:40:40.860 +So you can take any C code and have it run in the browser. + +00:40:41.400 --> 00:40:43.880 +And at that point, Python is just C code. + +00:40:44.320 --> 00:40:44.420 +So + +00:40:44.420 --> 00:40:45.100 +all of + +00:40:45.100 --> 00:40:45.400 +a sudden, + +00:40:45.750 --> 00:40:49.060 +yeah, but the best news is because the browser is everywhere, because + +00:40:49.300 --> 00:40:53.480 +JavaScript interpreters are everywhere, you've now got cross-platform binaries. + +00:40:53.880 --> 00:40:58.640 +You've got a format that you can compile to once, and it will run pretty much everywhere + +00:40:59.120 --> 00:41:03.640 +because existentially, every platform that matters has a web browser now, which means + +00:41:03.800 --> 00:41:04.920 +it's got the ability to run JavaScript. + +00:41:05.600 --> 00:41:09.460 +And so JavaScript becomes this cross-platform format + +00:41:09.810 --> 00:41:11.940 +without ever actually being JavaScript the language. + +00:41:12.160 --> 00:41:14.200 +It's just kind of exploiting some weird quirks + +00:41:14.300 --> 00:41:14.400 +of the + +00:41:14.400 --> 00:41:14.880 +way JavaScript + +00:41:14.880 --> 00:41:15.600 +works as a language. + +00:41:16.080 --> 00:41:17.820 +It's like saying you never write assembly, + +00:41:18.080 --> 00:41:19.980 +but kind of still do machine instructions. + +00:41:20.200 --> 00:41:21.320 +But you're always writing + +00:41:21.320 --> 00:41:21.660 +assembly. + +00:41:21.880 --> 00:41:22.340 +Yeah, exactly. + +00:41:22.880 --> 00:41:27.200 +There's a really, if you go back, oh gosh, 12, 13 years, + +00:41:27.440 --> 00:41:29.240 +Gary Bernhardt gave a talk at PyCon + +00:41:29.980 --> 00:41:31.540 +called The Birth and Death of JavaScript. + +00:41:32.440 --> 00:41:33.040 +So good. + +00:41:33.390 --> 00:41:34.180 +It was so good. + +00:41:34.660 --> 00:41:35.500 +It is hilarious. + +00:41:36.180 --> 00:41:39.260 +At the time, it was just straight up, oh, this is a funny gag, + +00:41:39.860 --> 00:41:43.280 +except that he has accurately predicted the next 15 years + +00:41:43.480 --> 00:41:46.540 +of software development in terms of the way everything has gone. + +00:41:46.850 --> 00:41:48.780 +And I've never spoken to Gary about it, + +00:41:48.840 --> 00:41:52.400 +so I don't know how much of this was just for the giggles + +00:41:53.030 --> 00:41:56.620 +and how much of it was accurately like this is where things are going. + +00:41:57.740 --> 00:41:58.720 +But as it + +00:41:58.720 --> 00:41:58.860 +turns out It + +00:41:58.860 --> 00:41:59.160 +was + +00:41:59.160 --> 00:42:00.900 +both really, really entertaining + +00:42:00.900 --> 00:42:04.080 +and really technically well-presented + +00:42:04.340 --> 00:42:05.780 +and predicted, like you said. + +00:42:05.900 --> 00:42:06.080 +Yes. + +00:42:06.460 --> 00:42:06.800 +It's something. + +00:42:07.240 --> 00:42:08.940 +And for people who are not familiar with this stuff, + +00:42:09.040 --> 00:42:10.280 +like one of the examples he showed, + +00:42:10.360 --> 00:42:11.300 +I don't know if he came up with it + +00:42:11.340 --> 00:42:13.140 +or he was just quoting and showing it off, + +00:42:13.460 --> 00:42:16.380 +but somebody took Mozilla Firefox, I think, + +00:42:16.480 --> 00:42:18.280 +and compiled that into Scriptum, + +00:42:18.320 --> 00:42:19.420 +was running inside of Chrome, + +00:42:19.940 --> 00:42:21.200 +but then inside the embedded, + +00:42:21.620 --> 00:42:23.820 +Firefox was running like Doom compiled, + +00:42:23.880 --> 00:42:26.280 +Doom Scriptum or some crazy 3D. + +00:42:26.500 --> 00:42:27.200 +I mean, it was wild. + +00:42:27.500 --> 00:42:27.660 +Yeah. + +00:42:27.820 --> 00:42:34.660 +The list of things you can do once you've got the ability to compile C is truly phenomenal, + +00:42:35.320 --> 00:42:35.580 +purely + +00:42:35.580 --> 00:42:36.120 +because + +00:42:36.120 --> 00:42:43.220 +the C compiler toolchain for all of the flaws of C as a language, C is + +00:42:43.320 --> 00:42:43.780 +lingua franca. + +00:42:44.160 --> 00:42:44.280 +C + +00:42:44.280 --> 00:42:45.180 +is where everything + +00:42:45.180 --> 00:42:45.620 +is done. + +00:42:46.420 --> 00:42:47.380 +And we're + +00:42:47.380 --> 00:42:50.640 +now kind of removing, like the whole thing with LRVM is essentially we're removing + +00:42:50.920 --> 00:42:53.880 +C from the front end, letting you put any language you want on the front. + +00:42:54.420 --> 00:42:57.200 +But now we can use the same toolchain to target any platform we want. + +00:42:57.320 --> 00:43:05.060 +And so you divorce the specific language from the tool chain that is producing platform-specific binaries. + +00:43:05.340 --> 00:43:05.560 +Nice. + +00:43:05.920 --> 00:43:08.340 +All right, let's talk about some of the projects and get some updates on them. + +00:43:08.580 --> 00:43:09.300 +I know it's been a while. + +00:43:09.960 --> 00:43:10.360 +And, + +00:43:10.880 --> 00:43:14.280 +you know, you've got a lot of support from Anaconda, which, you know, thank you again. + +00:43:15.060 --> 00:43:15.300 +Yes, + +00:43:15.440 --> 00:43:15.760 +absolutely. + +00:43:16.020 --> 00:43:16.460 +That's + +00:43:16.460 --> 00:43:16.640 +awesome. + +00:43:17.080 --> 00:43:20.580 +So we'll maybe touch on some of the high-level or most important ones. + +00:43:20.960 --> 00:43:21.100 +Toga? + +00:43:21.780 --> 00:43:22.180 +So + +00:43:22.180 --> 00:43:24.300 +Toga, the cross-platform GUI toolkit. + +00:43:24.520 --> 00:43:29.420 +So this is the analog of a Kibbe or a QT or a WX Windows. + +00:43:30.040 --> 00:43:33.080 +It's the API that lets you say, I want a window that has a button. + +00:43:33.360 --> 00:43:36.620 +And when I press the button, the light goes off, whatever + +00:43:36.620 --> 00:43:37.500 +you want it to do. + +00:43:37.660 --> 00:43:39.500 +Put this text in the label or whatever. + +00:43:39.780 --> 00:43:40.000 +Exactly, + +00:43:40.280 --> 00:43:40.520 +exactly. + +00:43:41.600 --> 00:43:46.020 +So it is cross-platform, pure Python, runs cross-platform. + +00:43:47.040 --> 00:43:49.440 +And by cross-platform, I mean all the desktop platforms, + +00:43:49.490 --> 00:43:52.760 +so Windows, macOS, Linux with GTK. + +00:43:53.280 --> 00:43:54.700 +It runs as a web. + +00:43:55.220 --> 00:43:56.640 +There's a web backend and a textual backend. + +00:43:56.760 --> 00:43:58.180 +Both of those are very early prototypes, + +00:43:58.340 --> 00:44:00.100 +but I can't, like there are demos I can do + +00:44:00.240 --> 00:44:01.220 +that I demonstrate that it works. + +00:44:02.220 --> 00:44:03.400 +And also iOS and Android. + +00:44:04.380 --> 00:44:06.800 +So essentially anywhere that you might want plausibly + +00:44:06.940 --> 00:44:07.820 +to have a user interface, + +00:44:08.420 --> 00:44:10.700 +I can give you a user interface purely written in Python. + +00:44:11.460 --> 00:44:14.820 +We also cover a bunch of hardware services as well. + +00:44:15.000 --> 00:44:18.040 +So for example, you can, using a pure Python interface, + +00:44:18.580 --> 00:44:19.720 +get a GPS read. + +00:44:19.780 --> 00:44:21.080 +You can work out where you are at a GPS + +00:44:21.580 --> 00:44:22.540 +and we have a native WAP widget. + +00:44:22.620 --> 00:44:26.480 +So you can say, get a GPS read and put it by center of the map of where I am currently standing. + +00:44:26.920 --> 00:44:29.720 +And that works on iOS, on Android, on macOS. + +00:44:30.400 --> 00:44:35.000 +We've got it working on GTK and we've got a PR in flight to make it work on Windows as well, I think. + +00:44:35.900 --> 00:44:39.780 +That, you know, it just, same code just runs everywhere and just, where am I? + +00:44:39.850 --> 00:44:40.400 +Give me an answer. + +00:44:40.710 --> 00:44:41.020 +Give me a + +00:44:41.020 --> 00:44:41.420 +ping every + +00:44:41.420 --> 00:44:42.840 +time I move a significant distance. + +00:44:43.700 --> 00:44:45.380 +And also for cameras, similar, similar thing for cameras. + +00:44:46.540 --> 00:44:49.940 +There is a question out of the audience, which I don't know the answer, but I'll throw that for you. + +00:44:50.020 --> 00:44:53.440 +Is there any more love coming for the Beware WebView widget? + +00:44:55.680 --> 00:44:58.960 +Having a case, not what they're referring to with Ceph, + +00:44:59.060 --> 00:44:59.160 +but + +00:44:59.160 --> 00:45:01.680 +is there + +00:45:01.680 --> 00:45:02.860 +more love coming? + +00:45:03.480 --> 00:45:05.820 +I mean, sure, as soon as somebody wants to pay attention to it. + +00:45:05.880 --> 00:45:08.860 +Am I actively intending in the very near future to do anything with it? + +00:45:09.420 --> 00:45:10.140 +Probably not. + +00:45:10.170 --> 00:45:15.760 +The most likely change that's on the horizon is to close the loop on callbacks. + +00:45:15.840 --> 00:45:18.080 +So at the moment, you can, from Python, + +00:45:18.620 --> 00:45:20.720 +inject arbitrary JavaScript into a web view, + +00:45:21.660 --> 00:45:24.480 +but you can't have something happen in the JavaScript + +00:45:24.700 --> 00:45:26.740 +to trigger an action outside in the Python, + +00:45:26.840 --> 00:45:29.660 +which is essentially kind of the last piece of the puzzle you need + +00:45:29.690 --> 00:45:33.140 +to have a full kind of electron clone. + +00:45:33.270 --> 00:45:34.800 +Yeah, I was going to say, that's very electron. + +00:45:35.700 --> 00:45:36.580 +Yeah, we + +00:45:36.580 --> 00:45:39.280 +actually have a bunch of tooling in place. + +00:45:39.500 --> 00:45:41.280 +We call it positron because it's like electron, + +00:45:41.460 --> 00:45:42.400 +but positive because it's Python. + +00:45:42.510 --> 00:45:42.960 +I love it. + +00:45:43.490 --> 00:45:43.820 +I love it. + +00:45:43.960 --> 00:45:47.860 +Yeah, so we can take a website and turn it into a native app. + +00:45:48.240 --> 00:45:50.720 +The only thing you can't do is then close the loop back out + +00:45:50.780 --> 00:45:55.860 +and call native hardware type stuff back into the Python API. + +00:45:56.320 --> 00:45:57.220 +Not impossible to do. + +00:45:57.520 --> 00:45:59.680 +I know exactly how to do it on iOS and macOS. + +00:45:59.820 --> 00:46:01.400 +I think I know how to do it on GTK. + +00:46:01.420 --> 00:46:02.580 +I don't know how to do it on Windows. + +00:46:03.980 --> 00:46:07.000 +So that might happen at some point in the not too distant future. + +00:46:08.360 --> 00:46:11.860 +Beyond that, oh, Seth, are they referring to the Chromium framework? + +00:46:12.440 --> 00:46:15.180 +No intention to, but if someone wants to write that up, + +00:46:15.680 --> 00:46:17.760 +there's no reason you couldn't be a standalone widget. + +00:46:17.920 --> 00:46:23.760 +This is very much from the Django days of if there is no, + +00:46:24.040 --> 00:46:27.780 +just because something exists doesn't mean it needs to be in the core. + +00:46:27.860 --> 00:46:30.040 +So if someone wants to write a third-party widget + +00:46:30.240 --> 00:46:32.620 +that wraps the Chromium embedded framework + +00:46:33.340 --> 00:46:36.480 +and turn it into a Toga widget, Toga will support that. + +00:46:36.660 --> 00:46:39.280 +And to the extent that if there's something that makes it a little bit crusty, + +00:46:39.320 --> 00:46:41.180 +there's probably a couple of areas where it could be a little bit smoother. + +00:46:41.720 --> 00:46:44.340 +If there's a way we can make it smoother, I'm totally on board with that. + +00:46:44.840 --> 00:46:47.100 +Doesn't mean it needs to be in Toga core itself. + +00:46:47.760 --> 00:46:51.360 +Django has a huge ecosystem of Django compatible plugins. + +00:46:52.060 --> 00:46:53.620 +Toga very much can be in that same space. + +00:46:54.640 --> 00:46:57.880 +Everything doesn't have to be in Toga for it to be useful in Toga. + +00:46:58.200 --> 00:46:58.580 +That's awesome. + +00:46:58.720 --> 00:46:59.120 +That's good to know. + +00:46:59.480 --> 00:47:04.580 +So I guess maybe the big updates here are the textual and the web targets for Toga. + +00:47:04.610 --> 00:47:04.960 +Is that right? + +00:47:05.660 --> 00:47:09.440 +They've been the biggest changes in the last eight years, I guess, is that those two now + +00:47:10.210 --> 00:47:10.320 +exist. + +00:47:11.480 --> 00:47:15.140 +And obviously you still need a lot of work, but like proof of concept, you can do it. + +00:47:15.840 --> 00:47:16.280 +And it's really + +00:47:16.280 --> 00:47:18.720 +just kind of turn the handle and make more widgets come out. + +00:47:19.620 --> 00:47:22.380 +The biggest sort of impediment to that at the moment is actually testing. + +00:47:22.860 --> 00:47:28.700 +One of the changes that has come on since I've been working at Anaconda full time is we've + +00:47:28.840 --> 00:47:29.620 +gone and got test coverage. + +00:47:29.720 --> 00:47:34.300 +We've got 100% test coverage of all of the core and all of the desktop and mobile platforms. + +00:47:34.580 --> 00:47:39.440 +So we have a very, very solid regression test suite to know that things aren't going to + +00:47:39.460 --> 00:47:42.600 +break as a result of this weird little tweak over here that we make. + +00:47:42.680 --> 00:47:44.120 +We've actually got validation of all that. + +00:47:44.540 --> 00:47:47.260 +We don't have that yet on web and textual, + +00:47:47.820 --> 00:47:50.240 +mostly because of quirks of how those platforms actually run. + +00:47:50.380 --> 00:47:53.840 +There's a kind of a foundational question we need to answer on the web one + +00:47:53.840 --> 00:47:54.320 +in particular. + +00:47:55.120 --> 00:47:56.940 +Yeah, I can imagine how the textual one works. + +00:47:57.480 --> 00:48:00.720 +What's the runtime look for part of the web? + +00:48:00.820 --> 00:48:02.680 +Is that PyScript or is that PyOxid? + +00:48:03.700 --> 00:48:04.940 +It is PyScript, + +00:48:04.990 --> 00:48:05.100 +yeah. + +00:48:05.130 --> 00:48:08.400 +So it's a briefcase, which I guess we'll get to in a moment, + +00:48:08.560 --> 00:48:10.420 +But Briefcase has the ability to target a web deployment. + +00:48:10.540 --> 00:48:17.140 +It generates a HTML page that injects a load this wheel. + +00:48:17.600 --> 00:48:18.960 +And this wheel is your app. + +00:48:19.420 --> 00:48:24.720 +But it's a Python wheel that just contains your code for your running app. + +00:48:25.340 --> 00:48:25.540 +And, yeah, + +00:48:25.600 --> 00:48:26.400 +but it's PyScript + +00:48:26.400 --> 00:48:27.160 +that's making that work. + +00:48:27.500 --> 00:48:28.560 +Yeah, PyScript is neat. + +00:48:28.800 --> 00:48:29.280 +Definitely neat. + +00:48:29.600 --> 00:48:30.040 +Just + +00:48:30.040 --> 00:48:31.640 +the fact that it's opening up these types of things. + +00:48:31.820 --> 00:48:34.500 +So Briefcase, yeah, let's dive a little more into that one. + +00:48:34.840 --> 00:48:37.880 +Yeah, so Briefcase is then sort of the other part of the story. + +00:48:38.100 --> 00:48:39.500 +you know, I've written this out. + +00:48:39.500 --> 00:48:39.760 +It gives you the runtime, + +00:48:40.210 --> 00:48:41.800 +but you're still in the terminal to launch it + +00:48:41.820 --> 00:48:43.840 +with virtual environments and all that kind of business, right? + +00:48:44.080 --> 00:48:44.360 +Yeah, okay. + +00:48:44.760 --> 00:48:45.800 +Yeah, so briefcase is, + +00:48:46.220 --> 00:48:47.760 +the scary thing is at the end of the day, + +00:48:47.860 --> 00:48:50.440 +briefcase is really just kind of three templates + +00:48:50.590 --> 00:48:51.260 +in a trench coat. + +00:48:51.660 --> 00:48:52.560 +There's not a lot of, + +00:48:52.830 --> 00:48:55.380 +like, there's not a lot that it's doing beyond that + +00:48:55.560 --> 00:48:58.440 +and also encoding all of the annoying details + +00:48:58.820 --> 00:49:01.940 +that are embedded 15 pages deep in Microsoft's + +00:49:02.120 --> 00:49:05.140 +how to write an app, you know, documentation + +00:49:06.500 --> 00:49:10.240 +around code signing and packaging and building an installer + +00:49:10.560 --> 00:49:12.000 +and all of that kind of stuff. + +00:49:12.680 --> 00:49:17.200 +So on the theory that 99% of Python users have code + +00:49:17.300 --> 00:49:21.620 +that runs as Python minus M, my app, make that be a macOS app. + +00:49:21.870 --> 00:49:22.780 +And that's what Briefcase does. + +00:49:22.980 --> 00:49:25.800 +It says, okay, how do I turn this into an icon on my desktop + +00:49:26.000 --> 00:49:28.420 +that I can double click and the app will just start. + +00:49:28.760 --> 00:49:29.540 +And it will include + +00:49:29.540 --> 00:49:30.640 +an embedded + +00:49:30.640 --> 00:49:31.500 +version of Python. + +00:49:32.290 --> 00:49:34.960 +So I don't need to tell my end user how to install Python. + +00:49:35.060 --> 00:49:41.060 +it will be signed and notarized in accordance with Apple's latest specifications it might come + +00:49:41.180 --> 00:49:44.500 +inside a DMG because I need to be able to distribute it along with a readme or it might + +00:49:44.500 --> 00:49:49.760 +come as a.pkg file because I actually need like in the case of non-GUI apps you actually like you + +00:49:49.800 --> 00:49:53.420 +want to be able to put something on your path and you have to have like a post install script and + +00:49:53.620 --> 00:49:59.420 +pkgs how you do that on macOS similarly on Windows it produces an MSI and it drops a Python + +00:49:59.660 --> 00:50:04.240 +embedded and all that kind of stuff make sure your dependencies are there on macOS it makes sure that + +00:50:04.220 --> 00:50:09.020 +of you install. If I'm building it on ARM machine, the app will still run on x86 because it'll install + +00:50:09.500 --> 00:50:13.820 +both versions of both the x86 and the ARM binaries and merge them where they're necessary and do all + +00:50:13.820 --> 00:50:18.560 +that kind of stuff. So it's all the little fiddly details. It is essentially just running a bunch + +00:50:18.580 --> 00:50:24.100 +of command line, like everything Briefcase does is just a command line tool or rolling out a template. + +00:50:25.180 --> 00:50:34.180 +If you turn up the verbosity, it will tell you in the prompt, like this is what I'm running. I'm + +00:50:34.200 --> 00:50:35.780 +notary tool and so on and so on. + +00:50:36.460 --> 00:50:37.520 +But you don't have to worry about that. + +00:50:37.640 --> 00:50:39.580 +Like it's just going to, this is what will encode. + +00:50:39.580 --> 00:50:45.760 +It'll do it all for you and just spit out either a DMG, a.zip, an MSI, + +00:50:47.140 --> 00:50:51.640 +IPA files on iOS, APK files or AAB files on Android, + +00:50:51.960 --> 00:50:52.900 +whatever the native platform is. + +00:50:53.140 --> 00:50:56.480 +And then on Linux, one of the things we've added more recently, + +00:50:56.820 --> 00:51:02.620 +we can do flat packs, we can do Debian, RPM, PKG zip, + +00:51:03.160 --> 00:51:05.340 +PKG files for Arch and Manjuro, + +00:51:05.840 --> 00:51:08.680 +like all of the common packaging formats we try to cover. + +00:51:08.840 --> 00:51:10.440 +The only one that I can think of that's notable + +00:51:10.560 --> 00:51:11.660 +that we don't cover at the moment. + +00:51:12.480 --> 00:51:14.520 +We don't do NCS installers on Windows + +00:51:15.280 --> 00:51:19.360 +and we don't do Snap packages on Linux. + +00:51:19.840 --> 00:51:22.800 +Not because we can't, just because I've only got two arms + +00:51:23.080 --> 00:51:25.460 +and there's a limit to how much I can practically do. + +00:51:25.700 --> 00:51:28.200 +If someone wants those, I can point you at the code + +00:51:28.200 --> 00:51:29.140 +and show you what you're going to do. + +00:51:29.300 --> 00:51:31.720 +And it's a moderately... + +00:51:31.740 --> 00:51:35.760 +If you know the commands you need to run to make a thing happen, I can show you how to + +00:51:35.920 --> 00:51:36.640 +plug that into Briefcase. + +00:51:36.720 --> 00:51:39.740 +And like Briefcase's infrastructure is designed to be pluggable. + +00:51:39.780 --> 00:51:41.780 +So it doesn't even need to be in Briefcase. + +00:51:41.780 --> 00:51:44.800 +You can, again, do it as an external plugin and have it plug into Briefcase. + +00:51:45.080 --> 00:51:46.380 +There are two things I want to ask you about here. + +00:51:46.760 --> 00:51:47.000 +One, + +00:51:47.160 --> 00:51:48.520 +notarization and signing. + +00:51:48.960 --> 00:51:50.080 +I remember how + +00:51:50.080 --> 00:51:54.920 +long ago I built C++ desktop apps and.NET desktop apps. + +00:51:55.100 --> 00:51:58.760 +And you could just compile them and give them to somebody and let them run it. + +00:51:59.120 --> 00:52:01.240 +And those sweet days are long gone. + +00:52:01.680 --> 00:52:01.720 +You + +00:52:01.720 --> 00:52:03.600 +know, it'll get flagged with big warnings. + +00:52:03.830 --> 00:52:07.940 +Like this looks like a virus because it didn't go through our digital signature system. + +00:52:08.620 --> 00:52:08.720 +Yep. + +00:52:09.120 --> 00:52:10.360 +What a frustrating thing. + +00:52:10.640 --> 00:52:11.860 +So this has support for that. + +00:52:12.160 --> 00:52:12.540 +It does. + +00:52:12.840 --> 00:52:12.900 +Yeah. + +00:52:14.200 --> 00:52:14.380 +Yeah. + +00:52:14.780 --> 00:52:18.160 +It turns out there are bad people on the internet and they're willing + +00:52:18.160 --> 00:52:18.620 +to do bad + +00:52:18.620 --> 00:52:18.900 +things. + +00:52:19.590 --> 00:52:19.840 +Who knew? + +00:52:20.780 --> 00:52:25.460 +So yeah, macOS has a bunch of processes for signing all of the binaries in an application + +00:52:26.020 --> 00:52:30.420 +and then stamping what's called a notarization certificate to the installer to say, + +00:52:30.600 --> 00:52:33.760 +This is a package in a format that hasn't been modified + +00:52:34.040 --> 00:52:36.440 +that I am willing to verify as me as a person. + +00:52:37.040 --> 00:52:39.160 +This is what the version of the package looks like + +00:52:39.580 --> 00:52:40.440 +when it's correctly installed. + +00:52:41.440 --> 00:52:43.000 +And then Sorry, + +00:52:43.160 --> 00:52:44.160 +probably in practice, if you + +00:52:44.160 --> 00:52:47.260 +want to distribute stuff to Mac and Windows, you probably + +00:52:47.260 --> 00:52:49.400 +need to create a developer account and sign + +00:52:49.400 --> 00:52:50.940 +these things, unless your + +00:52:50.940 --> 00:52:53.740 +users are willing to run with the big scary, this looks + +00:52:53.740 --> 00:52:55.240 +unsigned and untrusted. We've + +00:52:55.240 --> 00:52:56.680 +moved it to the trash, or you + +00:52:56.680 --> 00:52:57.740 +can go in there and run it. You know + +00:52:57.740 --> 00:52:58.220 +what I mean? Yeah, + +00:52:58.460 --> 00:52:59.340 +like on macOS, + +00:52:59.460 --> 00:53:01.160 +It like literally has to be at this point. + +00:53:01.200 --> 00:53:02.480 +You actually can't get away + +00:53:02.780 --> 00:53:04.740 +with not getting a developer certificate + +00:53:04.920 --> 00:53:07.580 +because the secure stuff on the M1 ship + +00:53:08.100 --> 00:53:09.160 +literally will not run. + +00:53:09.440 --> 00:53:11.620 +You can do cheap and nasty ad hoc signing, + +00:53:11.740 --> 00:53:13.100 +which means it will run on my machine. + +00:53:14.080 --> 00:53:15.520 +But if you want to give it to anybody else, + +00:53:16.160 --> 00:53:17.240 +it is now at the point + +00:53:17.400 --> 00:53:18.880 +where you've got to go into the settings, + +00:53:19.420 --> 00:53:20.000 +explicitly say, + +00:53:20.240 --> 00:53:21.300 +I'm willing to trust content + +00:53:21.480 --> 00:53:22.640 +that comes from an unknown provider. + +00:53:23.180 --> 00:53:26.360 +Start the app by pressing option and double click, + +00:53:26.880 --> 00:53:28.960 +selecting the option of I'm willing to accept this. + +00:53:29.260 --> 00:53:32.960 +Like it can be done, but it's like nobody is actually ever going to do that. + +00:53:33.960 --> 00:53:34.900 +So you basically, yeah, + +00:53:34.960 --> 00:53:35.320 +you've got to + +00:53:35.320 --> 00:53:38.380 +sign up for a $99 a year developer account + +00:53:38.780 --> 00:53:39.820 +and then go through the signing process. + +00:53:40.060 --> 00:53:40.460 +And signing + +00:53:40.460 --> 00:53:41.160 +is + +00:53:41.160 --> 00:53:42.380 +a non-trivial activity. + +00:53:42.560 --> 00:53:43.340 +It is difficult. + +00:53:43.860 --> 00:53:45.780 +There are like you've got to do things in the right order. + +00:53:46.280 --> 00:53:50.060 +And there is like even you have to submit the app in a zip format. + +00:53:50.800 --> 00:53:53.980 +But you can't just use PK zip or your standard command line zip. + +00:53:54.140 --> 00:53:57.460 +You've got to use Ditto, which is Mac's version of zip, + +00:53:57.840 --> 00:54:04.880 +Because otherwise, file system properties can end up being corrupted because the UTF-18 coding won't go quite reversible in the way. + +00:54:04.980 --> 00:54:07.060 +Like, it is a nightmare. + +00:54:07.420 --> 00:54:09.080 +Briefcase solves all those problems for you. + +00:54:09.220 --> 00:54:14.500 +It should be as straightforward as just package this thing and sign and it works. + +00:54:15.780 --> 00:54:16.180 +Similarly, + +00:54:16.180 --> 00:54:17.800 +it's not quite as bad on Windows. + +00:54:18.060 --> 00:54:21.640 +Like, Windows will let you install an unsigned binary. + +00:54:21.720 --> 00:54:24.600 +It just comes up with sort of the, this comes from an unknown developer. + +00:54:24.740 --> 00:54:25.920 +Are you sure you want to do this? + +00:54:26.060 --> 00:54:26.620 +And you say, okay. + +00:54:27.540 --> 00:54:34.800 +But realistically, like people's willingness to click the this looks like it might be insecure is low and decreasing. + +00:54:35.400 --> 00:54:39.060 +And so, yeah, you've just got to go get a certificate, sign the app, validate that it's you. + +00:54:39.780 --> 00:54:42.240 +It's kind of it's the price of doing app business at this point. + +00:54:42.560 --> 00:54:46.140 +Yeah, I guess it's sort of the equivalent of you have to have SSL to run a website. + +00:54:46.620 --> 00:54:46.760 +However, + +00:54:47.010 --> 00:54:47.140 +exactly. + +00:54:47.370 --> 00:54:48.540 +It is exactly that. + +00:54:48.920 --> 00:54:50.420 +Yes, but we're missing Let's Encrypt. + +00:54:50.650 --> 00:54:51.360 +We're missing Let's Encrypt. + +00:54:51.400 --> 00:54:51.560 +Yes. + +00:54:51.650 --> 00:54:52.220 +That's that part. + +00:54:52.240 --> 00:54:52.300 +Yeah. + +00:54:52.860 --> 00:55:04.400 +From the Windows point of view, there is absolutely a window of opportunity to do a Let's Encrypt because Windows doesn't specify who you've got to get your certificate from. + +00:55:04.710 --> 00:55:06.880 +It just has to be signed by a root certificate that they honor. + +00:55:07.580 --> 00:55:10.780 +So, like, essentially the instructions in briefcase kind of go get a certificate. + +00:55:10.960 --> 00:55:11.460 +Where do you get it from? + +00:55:12.340 --> 00:55:12.600 +You know, + +00:55:12.880 --> 00:55:13.280 +here + +00:55:13.280 --> 00:55:13.880 +are some options. + +00:55:14.030 --> 00:55:14.520 +Go have fun. + +00:55:15.300 --> 00:55:19.540 +So there is absolutely an opportunity for someone to do a Let's Encrypt here for app signing. + +00:55:19.900 --> 00:55:27.420 +Unfortunately for Apple, that doesn't apply because Apple does insist that the certificate needs to be issued by them, signed by them, so that they can revoke it. + +00:55:28.010 --> 00:55:34.980 +And maybe the EU changes around app distribution will relax that somewhat, but at least for the moment, you've got to go through Apple. + +00:55:35.440 --> 00:55:35.500 +Yeah. + +00:55:35.890 --> 00:55:38.540 +As salty as I am at the EU about the cookie warnings. + +00:55:39.660 --> 00:55:39.880 +Yeah. + +00:55:40.210 --> 00:55:40.360 +Yeah. + +00:55:40.620 --> 00:55:41.660 +But I do have + +00:55:41.660 --> 00:55:45.540 +hope that they can bring some reason to some of the other App Store type behaviors. + +00:55:45.750 --> 00:55:46.700 +It would certainly be. + +00:55:46.940 --> 00:55:47.660 +I would not. + +00:55:47.740 --> 00:55:48.740 +I welcome that development. + +00:55:49.180 --> 00:55:50.800 +I am waiting to see what happens. + +00:55:51.680 --> 00:55:53.100 +I had a lot of hope that the App Store stuff, + +00:55:53.660 --> 00:55:55.620 +I had a lot of hope opening the App Store thing + +00:55:55.800 --> 00:55:58.140 +would provide a way to make it easier + +00:55:58.340 --> 00:55:59.520 +for Beware to distribute apps. + +00:56:00.540 --> 00:56:01.920 +But in practice, it doesn't look like + +00:56:02.020 --> 00:56:02.800 +that's actually going to happen. + +00:56:03.140 --> 00:56:03.920 +So yeah. + +00:56:04.500 --> 00:56:05.440 +It's, yeah. + +00:56:05.760 --> 00:56:08.520 +Nice try, but they squeeze through another + +00:56:08.520 --> 00:56:09.820 +track, + +00:56:10.240 --> 00:56:10.360 +basically. + +00:56:11.180 --> 00:56:11.760 +It is what it is. + +00:56:12.180 --> 00:56:12.820 +It is what it is. + +00:56:14.180 --> 00:56:15.620 +The gatekeeper stuff drives me crazy. + +00:56:15.820 --> 00:56:18.179 +I've had so much trouble getting my mobile apps + +00:56:18.200 --> 00:56:20.260 +in the app store, not for reasons technical, + +00:56:20.540 --> 00:56:21.460 +just for reasons of... + +00:56:21.890 --> 00:56:24.360 +Yeah, and I guess that's the + +00:56:24.360 --> 00:56:24.540 +thing. + +00:56:24.540 --> 00:56:27.040 +It's like, I will, to a limited extent, + +00:56:28.360 --> 00:56:33.400 +defend the, not so much the gatekeeping aspect of it, + +00:56:33.480 --> 00:56:35.660 +but the fact that there is a process involved + +00:56:36.000 --> 00:56:39.340 +to protect against completely arbitrary content. + +00:56:40.320 --> 00:56:41.660 +That side of it is useful. + +00:56:42.300 --> 00:56:45.280 +But the fact that there's, like, it is that locked down + +00:56:45.460 --> 00:56:46.959 +and only from Apple is the part + +00:56:46.980 --> 00:56:48.360 +where it becomes a little bit onerous + +00:56:48.500 --> 00:56:48.640 +and that + +00:56:48.640 --> 00:56:49.460 +there's no competition + +00:56:49.640 --> 00:56:50.680 +to make Apple work better, + +00:56:50.900 --> 00:56:51.740 +I guess is the real thing. + +00:56:51.780 --> 00:56:51.900 +Yeah, yeah. + +00:56:52.160 --> 00:56:53.140 +You can't go into a + +00:56:53.140 --> 00:56:53.700 +competition. + +00:56:54.140 --> 00:56:54.580 +Yeah, and + +00:56:54.580 --> 00:56:54.920 +the rent + +00:56:54.920 --> 00:56:55.540 +is a, yeah. + +00:56:55.840 --> 00:56:56.400 +Yeah, yeah, yeah. + +00:56:56.540 --> 00:56:57.180 +From my app there. + +00:56:57.480 --> 00:56:59.220 +You don't have in-app purchasing. + +00:56:59.440 --> 00:57:00.900 +Well, I don't want to sell anything in my app. + +00:57:00.900 --> 00:57:02.200 +I just want to let people view the stuff + +00:57:02.380 --> 00:57:03.060 +that we already got online. + +00:57:03.440 --> 00:57:04.540 +Well, you're going to have to add it. + +00:57:04.800 --> 00:57:08.260 +So weeks and weeks of paying developers + +00:57:08.680 --> 00:57:09.520 +and stuff to build it + +00:57:09.840 --> 00:57:11.680 +just so they can have their few bucks, + +00:57:11.840 --> 00:57:12.260 +you know what I mean? + +00:57:12.300 --> 00:57:12.780 +And it's just like, + +00:57:12.900 --> 00:57:14.060 +you have no idea how + +00:57:14.060 --> 00:57:15.080 +much pain this is. + +00:57:15.420 --> 00:57:16.380 +Yeah, and like, I guess, + +00:57:16.640 --> 00:57:21.500 +I could almost, I don't like it, but I could almost grant them that, you know, the golden rule of + +00:57:22.140 --> 00:57:26.120 +has the gold makes the rules, but it's their store. They get to decide what's in there, + +00:57:26.180 --> 00:57:27.720 +but I don't have the option of another store. + +00:57:28.400 --> 00:57:30.260 +And in + +00:57:30.260 --> 00:57:32.880 +particular, the review process is so + +00:57:33.320 --> 00:57:37.100 +inscrutable. Like Google is no better in this regard. If something + +00:57:37.100 --> 00:57:37.720 +goes wrong + +00:57:37.720 --> 00:57:38.340 +with your Google + +00:57:38.540 --> 00:57:46.600 +application, good luck with that. For all the money they're taking, they're not providing good + +00:57:46.600 --> 00:57:49.220 +customer service. They're providing no as a customer service. + +00:57:50.300 --> 00:57:51.520 +Exactly. Yeah. They're both + +00:57:51.640 --> 00:57:57.220 +bad. I'm not saying Google's good. They're both bad. Okay. But for a briefcase here, we have, + +00:57:57.520 --> 00:58:03.220 +we have the targets that I would expect Linux, Mac, Windows. Yes. But iPad, iPhone, Android, + +00:58:03.460 --> 00:58:05.600 +tell us about that. Even watch. + +00:58:06.500 --> 00:58:09.320 +So we, yeah, so we don't, we don't actually have it working for + +00:58:09.440 --> 00:58:09.700 +watchOS. + +00:58:09.880 --> 00:58:10.160 +I have + +00:58:10.160 --> 00:58:10.860 +a proof of, + +00:58:11.180 --> 00:58:14.960 +yeah, I have a proof of concept that it can work on Apple TV. I have + +00:58:14.980 --> 00:58:16.800 +written a Python application for Apple TV. + +00:58:17.730 --> 00:58:19.740 +Web deployment, actually, that's that website's out of date. + +00:58:19.810 --> 00:58:20.560 +I should need to update that. + +00:58:20.630 --> 00:58:21.620 +We do have web deployments. + +00:58:21.840 --> 00:58:23.900 +Put that in the proper bullet list, right? + +00:58:23.940 --> 00:58:24.140 +Yes. + +00:58:24.500 --> 00:58:25.500 +Yes, it should be on the proper bullet list. + +00:58:26.620 --> 00:58:31.060 +So, yeah, but submitting to an app store, there are iOS apps, + +00:58:31.110 --> 00:58:33.880 +there are Android apps in the Apple and Google app stores + +00:58:34.900 --> 00:58:36.540 +that are written with BWare. + +00:58:37.480 --> 00:58:39.020 +So you are producing a binary. + +00:58:39.110 --> 00:58:41.200 +It is signed and validated and then sent to Apple, + +00:58:41.310 --> 00:58:43.880 +and Apple says, yes, that's fine, and Google says, yes, that's fine, + +00:58:44.020 --> 00:58:45.280 +and it's available for purchase. + +00:58:45.540 --> 00:58:46.100 +So, yeah. + +00:58:46.240 --> 00:58:46.420 +Awesome. + +00:58:46.560 --> 00:58:49.140 +So what do you get, like an APK for Android? + +00:58:49.440 --> 00:58:50.640 +And what is the AP? + +00:58:50.820 --> 00:58:51.540 +It's an IPA. + +00:58:51.720 --> 00:58:53.340 +Yeah, it's an IPA, but you don't ever + +00:58:53.340 --> 00:58:53.960 +see it. + +00:58:54.120 --> 00:58:56.520 +Like you basically have to, for all practical purposes, + +00:58:56.660 --> 00:58:57.920 +you have to submit it through Xcode. + +00:58:58.340 --> 00:59:00.740 +That might be something we can resolve, but I haven't. + +00:59:00.980 --> 00:59:01.240 +They don't + +00:59:01.240 --> 00:59:02.200 +make it easy to find out + +00:59:02.200 --> 00:59:02.740 +how to not + +00:59:02.740 --> 00:59:03.300 +use Xcode + +00:59:03.300 --> 00:59:03.700 +to do it. + +00:59:04.420 --> 00:59:06.500 +No, I think you're probably committed. + +00:59:06.680 --> 00:59:09.660 +Even for my Flutter-based app, I've got to go do a build + +00:59:09.900 --> 00:59:11.860 +and an app store submission through Xcode. + +00:59:12.020 --> 00:59:12.180 +Yeah, yeah. + +00:59:12.380 --> 00:59:13.120 +The tools are + +00:59:13.120 --> 00:59:14.700 +there, but yeah. + +00:59:15.860 --> 00:59:17.700 +But yeah, you get an AAB for Android. + +00:59:17.800 --> 00:59:22.880 +You get effectively an IPA, but an Xcode project you can submit for iPhone. + +00:59:23.640 --> 00:59:29.320 +For web, you get a tarball that is basically upload to your web provider of choice, + +00:59:29.400 --> 00:59:34.120 +and it would work, and it would end up being similar for Apple TV + +00:59:34.240 --> 00:59:36.480 +or WatchOS or WearOS eventually. + +00:59:36.620 --> 00:59:36.740 +Awesome. + +00:59:37.380 --> 00:59:40.540 +Well, that's really great that you've been making so much progress on this, + +00:59:40.960 --> 00:59:43.860 +And is it Steven, the other guy you're working with? + +00:59:44.920 --> 00:59:45.280 +Malcolm. + +00:59:45.800 --> 00:59:46.340 +I'm sorry, + +00:59:46.460 --> 00:59:46.580 +Malcolm. + +00:59:47.040 --> 00:59:48.660 +You and Malcolm have been making so much progress. + +00:59:48.860 --> 00:59:49.720 +That's really, really great. + +00:59:50.000 --> 00:59:53.660 +And I think only good things are going to come from having the backing of Anaconda. + +00:59:54.239 --> 00:59:55.860 +So we've got just a couple of minutes left. + +00:59:55.980 --> 00:59:58.600 +What do you want to tell people about this project, these projects, + +00:59:58.750 --> 01:00:00.220 +how they should adopt them, where + +01:00:00.220 --> 01:00:00.940 +they're ready for use, + +01:00:01.450 --> 01:00:02.360 +and so on, things like that? + +01:00:02.620 --> 01:00:05.620 +I guess the quick thing I'd say is that we're not done yet. + +01:00:05.820 --> 01:00:06.600 +There's still more to come. + +01:00:07.939 --> 01:00:10.380 +Just a week and change ago, PIP. + +01:00:10.760 --> 01:00:15.480 +sorry, PyPI officially started accepting iOS and Android wheels. + +01:00:15.640 --> 01:00:17.700 +So you can upload an iOS and Android wheel. + +01:00:18.040 --> 01:00:20.680 +There aren't any on there yet that I'm aware of, + +01:00:20.740 --> 01:00:22.060 +but I'm actively working on that. + +01:00:22.120 --> 01:00:24.060 +So just so I can prove, hey, look, yes, there's one over there. + +01:00:24.100 --> 01:00:24.500 +Look at it. + +01:00:25.880 --> 01:00:27.420 +But part of the reason there's not one there yet + +01:00:27.600 --> 01:00:31.220 +is that the tooling to build iOS and Android wheels is not quite there. + +01:00:31.620 --> 01:00:33.160 +And that's what I'm actively working on right now. + +01:00:33.280 --> 01:00:36.760 +I would be deeply surprised if by the end of this year, + +01:00:36.860 --> 01:00:39.180 +there aren't multiple high profile projects that have. + +01:00:39.300 --> 01:00:45.900 +I have the patches for Pillow to literally produce iOS wheels as part of their normal release process. + +01:00:46.380 --> 01:00:49.480 +They haven't even seen the PRs yet, but I'm + +01:00:49.480 --> 01:00:50.440 +certainly hoping to + +01:00:50.440 --> 01:00:50.960 +work with them. + +01:00:51.280 --> 01:00:52.300 +Yeah, I mean, they know it's coming. + +01:00:52.500 --> 01:00:53.160 +Hugo knows it's coming. + +01:00:53.380 --> 01:01:00.500 +So there is a lot of work still to do in terms of getting that patch accepted, but at least I know it can be done. + +01:01:00.640 --> 01:01:15.560 +And more importantly, things like CI build wheels, like the infrastructure for putting a build all these wheels as part of CI is something where I have the patch and I'm like this close to being able to submit a pull request to CI build wheel to say, here's the back end to let you generate iOS wheels. + +01:01:16.300 --> 01:01:18.300 +Similarly, we've got to do all that work for Android as well. + +01:01:18.460 --> 01:01:20.100 +So there is still a lot of work to come. + +01:01:20.440 --> 01:01:32.360 +My hope is that sort of by the end of this year, says he, being very hopeful, we'll be in a position where like anything you can do on a desktop platform, you can basically do on iOS and Android as well. + +01:01:33.140 --> 01:01:40.600 +With the caveat that there might be, if there's C code involved, you might need to actually like work out how to compile this thing for iOS and Android. + +01:01:41.200 --> 01:01:43.860 +For a lot of simple projects, literally there's nothing you've got to do. + +01:01:43.920 --> 01:01:46.200 +You've just cranked the handle and make the compiler do its thing. + +01:01:46.480 --> 01:01:47.880 +Yeah, especially if you have Python. + +01:01:48.380 --> 01:01:48.940 +Yeah, exactly. + +01:01:49.620 --> 01:01:51.540 +The other part that I'm sort of playing around with this year + +01:01:51.860 --> 01:01:54.840 +as a sort of a background project is doing for Enscripten + +01:01:55.320 --> 01:01:58.980 +what I did last year or I'm up and I did last year for iOS and Android, + +01:01:59.120 --> 01:02:01.400 +which is get Enscripten up to tier three support. + +01:02:01.550 --> 01:02:05.580 +So there will be a PEP and official supported platform. + +01:02:05.750 --> 01:02:09.900 +Like Enscripten will be an officially supported platform in CPython + +01:02:10.360 --> 01:02:12.160 +in the same way that iOS and Android are. + +01:02:13.000 --> 01:02:16.520 +I'm mostly shepherding that as sort of a member of the CPython core team + +01:02:17.839 --> 01:02:21.040 +and Hood Chatham is actually doing most of the heavy lifting on that. + +01:02:21.580 --> 01:02:24.780 +He's a PyDyde maintainer and is involved with PyScript as well + +01:02:24.900 --> 01:02:25.980 +to a lesser extent. + +01:02:27.380 --> 01:02:29.520 +And, yeah, so he is working on the patches for that + +01:02:29.550 --> 01:02:31.920 +and I'm kind of shepherding those patches into core. + +01:02:32.620 --> 01:02:35.500 +We are about to stand up and build bot to actually make the kind of the – + +01:02:35.880 --> 01:02:39.060 +make sure every pull request is validated to make sure it still works + +01:02:39.180 --> 01:02:41.640 +on Inscripten essentially, which is one of the big sort of milestones + +01:02:42.020 --> 01:02:43.480 +for official platform support. + +01:02:43.740 --> 01:02:44.900 +Yeah, that's super neat. + +01:02:45.640 --> 01:02:46.240 +What about CI? + +01:02:46.260 --> 01:02:46.980 +for people. + +01:02:47.590 --> 01:02:48.260 +Yeah, what about CI + +01:02:48.330 --> 01:02:49.200 +for people who want to build + +01:02:49.720 --> 01:02:51.700 +wheels for iOS and Android? + +01:02:52.300 --> 01:02:53.000 +That's essentially + +01:02:53.050 --> 01:02:53.760 +where CI build rule + +01:02:53.760 --> 01:02:54.100 +comes in. + +01:02:54.280 --> 01:02:55.240 +So you can do it right now. + +01:02:55.320 --> 01:02:56.500 +So every time we + +01:02:56.500 --> 01:02:56.720 +make + +01:02:56.720 --> 01:02:57.420 +a change to Toga, + +01:02:57.880 --> 01:02:59.400 +we run a full test suite + +01:02:59.820 --> 01:03:00.980 +on iOS, on Android, + +01:03:01.720 --> 01:03:02.520 +on macOS + +01:03:02.660 --> 01:03:03.220 +and all the other + +01:03:03.380 --> 01:03:03.880 +desktop platforms. + +01:03:04.220 --> 01:03:06.120 +You can use like GitHub Action CI + +01:03:06.340 --> 01:03:07.280 +will let you start + +01:03:07.430 --> 01:03:08.500 +an iPhone simulator + +01:03:08.940 --> 01:03:09.600 +and run the tests + +01:03:09.780 --> 01:03:10.560 +on that iPhone simulator. + +01:03:10.610 --> 01:03:12.300 +You can start an Android simulator + +01:03:12.680 --> 01:03:13.540 +and it will run a simulator + +01:03:13.550 --> 01:03:14.020 +on those platforms. + +01:03:14.400 --> 01:03:15.440 +It's like it takes three minutes + +01:03:15.460 --> 01:03:21.240 +to start the simulator, but that's just the price of doing business for iPhone, basically. + +01:03:21.700 --> 01:03:25.620 +Yeah, I guess they probably already have CI for iOS and Android because there's probably + +01:03:25.620 --> 01:03:27.340 +a ton of mobile projects on there. + +01:03:27.440 --> 01:03:30.240 +It's kind of unusual to complain it with Python, right? + +01:03:30.540 --> 01:03:31.240 +Yes, absolutely. + +01:03:31.420 --> 01:03:33.460 +And so you can do it. + +01:03:33.880 --> 01:03:34.940 +You can run those tests. + +01:03:35.200 --> 01:03:38.440 +There are probably some things that could be done at an institutional level at GitHub + +01:03:38.780 --> 01:03:40.460 +to make those tests run faster. + +01:03:41.800 --> 01:03:48.420 +But that's a, you know, now I've just got to convince Microsoft to implement an entirely new backend for GitHub Actions. + +01:03:48.860 --> 01:03:51.020 +And they don't support like Windows on ARM yet. + +01:03:51.080 --> 01:03:52.700 +So I'm not holding their breath thinking I support iOS. + +01:03:53.640 --> 01:03:54.860 +Yeah, yeah, I hear + +01:03:54.860 --> 01:03:55.040 +that. + +01:03:55.420 --> 01:03:57.080 +One final forward-looking question, I suppose. + +01:03:57.440 --> 01:04:01.340 +What do you see support for MScript in giving us? + +01:04:01.640 --> 01:04:06.440 +You know, for example, like having the JavaScript stuff, you'd see like, okay, well, great, I can run it in a browser. + +01:04:06.600 --> 01:04:10.940 +But well, then Node comes along and now I can run servers on it or I can distribute it or there's tool. + +01:04:11.240 --> 01:04:14.060 +Like there's these sort of ways in which these environments + +01:04:14.460 --> 01:04:17.080 +grow unexpectedly, at least for most people, I think. + +01:04:17.130 --> 01:04:19.080 +And how do you think InScript in my... + +01:04:19.280 --> 01:04:19.740 +So + +01:04:19.740 --> 01:04:23.200 +my hope is that what we'll end up seeing + +01:04:23.680 --> 01:04:28.160 +is the analog of React and Vue, but for Python. + +01:04:29.500 --> 01:04:31.720 +So you can right now, or you have been able to write + +01:04:31.890 --> 01:04:33.860 +a Django server-side website, + +01:04:34.080 --> 01:04:36.240 +pick any other GUI, any other web framework you want. + +01:04:36.980 --> 01:04:38.560 +You can write your server-side code, excuse me, + +01:04:38.570 --> 01:04:40.280 +you write your server-side code completely in Python. + +01:04:40.490 --> 01:04:40.820 +It's great. + +01:04:41.840 --> 01:04:43.940 +but then you've got to push your code to the front end + +01:04:44.340 --> 01:04:47.620 +and you have to write JavaScript on the front end. + +01:04:48.020 --> 01:04:49.760 +And so that's kind of essentially where Node came from + +01:04:49.900 --> 01:04:51.380 +is realizing that we've got to write this code, + +01:04:51.430 --> 01:04:52.420 +like we've got validation code, + +01:04:52.540 --> 01:04:53.380 +you've got to run it around the client + +01:04:53.520 --> 01:04:54.200 +and run it around the server. + +01:04:54.520 --> 01:04:57.000 +So I can't put Python in the client, + +01:04:57.420 --> 01:04:59.000 +so I might as well bring JavaScript back to the server. + +01:04:59.320 --> 01:05:01.400 +And so having that same language everywhere + +01:05:01.550 --> 01:05:02.420 +is a real advantage. + +01:05:03.260 --> 01:05:05.120 +InScripten lets us get Python into the browser, + +01:05:05.560 --> 01:05:06.940 +but the part that we don't have yet + +01:05:07.160 --> 01:05:11.780 +is the rich Python-first GUI framework in the browser. + +01:05:13.260 --> 01:05:15.000 +My hope is that with... + +01:05:15.170 --> 01:05:16.640 +Oh, you lost me there? I'm not sure. + +01:05:18.260 --> 01:05:24.120 +My hope is that with inscription in the browser, + +01:05:24.740 --> 01:05:29.620 +what we'll get is the ability to build that toolkit. + +01:05:30.120 --> 01:05:31.080 +I don't know what it is. + +01:05:31.110 --> 01:05:33.360 +I don't know what will stimulate its development, + +01:05:33.530 --> 01:05:36.080 +but it should hopefully make it, or at least makes it possible. + +01:05:36.680 --> 01:05:43.000 +And once it's possible, then a bunch of people, you know, project people can start playing around with it and actually making things. + +01:05:46.140 --> 01:05:47.300 +Oh, we lost Russell. + +01:05:49.779 --> 01:05:52.020 +Well, that's probably a good place to call it, folks. + +01:05:52.720 --> 01:05:53.940 +Thanks to Russell for being on the show. + +01:05:54.360 --> 01:05:55.060 +And thank you all for listening. + +01:05:55.600 --> 01:05:56.040 +Talk to you all later. + +01:05:56.450 --> 01:05:56.540 +Bye. + +01:05:58.180 --> 01:06:00.560 +This has been another episode of Talk Python to Me. + +01:06:01.360 --> 01:06:02.380 +Thank you to our sponsors. + +01:06:02.770 --> 01:06:04.000 +Be sure to check out what they're offering. + +01:06:04.130 --> 01:06:05.420 +It really helps support the show. + +01:06:06.300 --> 01:06:09.620 +This episode is sponsored by Posit and Posit Workbench. + +01:06:10.380 --> 01:06:15.100 +Posit Workbench allows data scientists to code in Python within their preferred environment + +01:06:15.570 --> 01:06:17.580 +without any additional strain on IT. + +01:06:18.340 --> 01:06:21.540 +It gives data scientists access to all the development environments they love, + +01:06:21.720 --> 01:06:24.640 +including Jupyter Notebooks, JupyterLab, Positron, and VS Code, + +01:06:25.100 --> 01:06:27.740 +and helps ensure reproducibility and consistency. + +01:06:28.260 --> 01:06:30.540 +If you work on a data science team where consistency matters, + +01:06:31.080 --> 01:06:32.540 +check out Posit Workbench. + +01:06:33.100 --> 01:06:35.540 +Visit talkpython.fm/workbench for details. + +01:06:36.640 --> 01:06:37.560 +Want to level up your Python? + +01:06:38.020 --> 01:06:41.660 +We have one of the largest catalogs of Python video courses over at Talk Python. + +01:06:42.140 --> 01:06:46.820 +Our content ranges from true beginners to deeply advanced topics like memory and async. + +01:06:47.260 --> 01:06:49.440 +And best of all, there's not a subscription in sight. + +01:06:49.840 --> 01:06:52.360 +Check it out for yourself at training.talkpython.fm. + +01:06:53.080 --> 01:06:57.240 +Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +01:06:57.700 --> 01:06:58.560 +We should be right at the top. + +01:06:58.680 --> 01:07:07.920 +You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +01:07:08.600 --> 01:07:10.840 +We're live streaming most of our recordings these days. + +01:07:11.180 --> 01:07:18.680 +If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:07:19.700 --> 01:07:20.820 +This is your host, Michael Kennedy. + +01:07:21.240 --> 01:07:22.080 +Thanks so much for listening. + +01:07:22.240 --> 01:07:23.240 +I really appreciate it. + +01:07:23.580 --> 01:07:25.180 +Now get out there and write some Python code. + From 654669f28cdf2fa195f4077791b922833da80259 Mon Sep 17 00:00:00 2001 From: Michael Kennedy Date: Thu, 10 Apr 2025 20:33:05 -0700 Subject: [PATCH 2/2] transcripts --- ...-Pyramid-Web Framework-Chris-McDonough.txt | 2 +- ...-Pyramid-Web Framework-Chris-McDonough.vtt | 2 +- transcripts/028_profiling.txt | 2 +- transcripts/028_profiling.vtt | 2 +- ...ef-data-scientist-of-the-united-states.txt | 4 +- ...ef-data-scientist-of-the-united-states.vtt | 4 +- ...57-the-journal-of-open-source-software.txt | 2 +- ...57-the-journal-of-open-source-software.vtt | 2 +- transcripts/496-scaf-deploys.vtt | 2 +- ...ware-and-the-state-of-python-on-mobile.vtt | 2 +- transcripts/500-django-simple-deploy.txt | 1332 +++++++++ transcripts/500-django-simple-deploy.vtt | 2150 ++++++++++++++ ...1-marimo-reactive-notebooks-for-python.txt | 1532 ++++++++++ ...1-marimo-reactive-notebooks-for-python.vtt | 2633 +++++++++++++++++ .../378-flet-flutter-apps-in-python.vtt | 2 +- youtube_transcripts/390-mastodon.vtt | 2 +- .../391-pyscript-powered-by-micropython.vtt | 2 +- .../462-pandas-and-beyond-with-wes.vtt | 2 +- 18 files changed, 7663 insertions(+), 16 deletions(-) create mode 100644 transcripts/500-django-simple-deploy.txt create mode 100644 transcripts/500-django-simple-deploy.vtt create mode 100644 transcripts/501-marimo-reactive-notebooks-for-python.txt create mode 100644 transcripts/501-marimo-reactive-notebooks-for-python.vtt diff --git a/transcripts/003-Pyramid-Web Framework-Chris-McDonough.txt b/transcripts/003-Pyramid-Web Framework-Chris-McDonough.txt index 470adfa..2c1eb12 100644 --- a/transcripts/003-Pyramid-Web Framework-Chris-McDonough.txt +++ b/transcripts/003-Pyramid-Web Framework-Chris-McDonough.txt @@ -718,7 +718,7 @@ 00:27:59 So so if you if you look inside of easy install dot PTH thing, Philip Eby took serious advantage of that, you know, when he wrote setup tools. -00:28:08 And it's got it's got some some Python, inscrutable Python in there that does some fun stuff. +00:28:08 And it's got it's got some Python, inscrutable Python in there that does some fun stuff. 00:28:16 I think it sets up package resources on there. diff --git a/transcripts/003-Pyramid-Web Framework-Chris-McDonough.vtt b/transcripts/003-Pyramid-Web Framework-Chris-McDonough.vtt index 164ad7f..6df6aa3 100644 --- a/transcripts/003-Pyramid-Web Framework-Chris-McDonough.vtt +++ b/transcripts/003-Pyramid-Web Framework-Chris-McDonough.vtt @@ -1087,7 +1087,7 @@ Oh, boy. So so if you if you look inside of easy install dot PTH thing, Philip Eby took serious advantage of that, you know, when he wrote setup tools. 00:28:08.700 --> 00:28:16.560 -And it's got it's got some some Python, inscrutable Python in there that does some fun stuff. +And it's got it's got some Python, inscrutable Python in there that does some fun stuff. 00:28:16.560 --> 00:28:19.500 I think it sets up package resources on there. diff --git a/transcripts/028_profiling.txt b/transcripts/028_profiling.txt index 1633b48..342a072 100644 --- a/transcripts/028_profiling.txt +++ b/transcripts/028_profiling.txt @@ -422,7 +422,7 @@ 00:27:28 a lot of cool advances in python 3 around this type of parallelism right and they just added async and -00:27:37 await the the new keywords to uh is that three five i think it was right yeah just three fives just +00:27:37 await the new keywords to uh is that three five i think it was right yeah just three fives just 00:27:43 came out like two days yeah two days ago so yeah i mean that's super new but these are the types diff --git a/transcripts/028_profiling.vtt b/transcripts/028_profiling.vtt index ef95d2e..00cdde9 100644 --- a/transcripts/028_profiling.vtt +++ b/transcripts/028_profiling.vtt @@ -865,7 +865,7 @@ i mean as long as everything else is not the bottleneck right and you know we've a lot of cool advances in python 3 around this type of parallelism right and they just added async and 00:27:37.600 --> 00:27:43.960 -await the the new keywords to uh is that three five i think it was right yeah just three fives just +await the new keywords to uh is that three five i think it was right yeah just three fives just 00:27:43.960 --> 00:27:49.100 came out like two days yeah two days ago so yeah i mean that's super new but these are the types diff --git a/transcripts/089-a-conversation-with-the-chief-data-scientist-of-the-united-states.txt b/transcripts/089-a-conversation-with-the-chief-data-scientist-of-the-united-states.txt index eb17b96..83e76e6 100644 --- a/transcripts/089-a-conversation-with-the-chief-data-scientist-of-the-united-states.txt +++ b/transcripts/089-a-conversation-with-the-chief-data-scientist-of-the-united-states.txt @@ -874,7 +874,7 @@ 00:30:59 All of that falls on us for the implications of these things. -00:31:03 And we just have to get ready to drive that world because the the time is now for us to do that. +00:31:03 And we just have to get ready to drive that world because the time is now for us to do that. 00:31:10 But let's actually, you know, one of the things I think given your audience, I'd actually be really curious to hear what the audience has to say on this. @@ -890,7 +890,7 @@ 00:31:40 That's you know, that's one that I think the community has an incredible opportunity to stand up and say this is where we believe. -00:31:45 And I have no idea what the the broad consensus of data scientists are thinking, except for the ones that we've had a lot of interactions with our our very traditional White House processes. +00:31:45 And I have no idea what the broad consensus of data scientists are thinking, except for the ones that we've had a lot of interactions with our our very traditional White House processes. 00:31:58 And that makes sense. diff --git a/transcripts/089-a-conversation-with-the-chief-data-scientist-of-the-united-states.vtt b/transcripts/089-a-conversation-with-the-chief-data-scientist-of-the-united-states.vtt index cee972c..bab6450 100644 --- a/transcripts/089-a-conversation-with-the-chief-data-scientist-of-the-united-states.vtt +++ b/transcripts/089-a-conversation-with-the-chief-data-scientist-of-the-united-states.vtt @@ -1321,7 +1321,7 @@ And what are the implications of that? All of that falls on us for the implications of these things. 00:31:03.520 --> 00:31:10.620 -And we just have to get ready to drive that world because the the time is now for us to do that. +And we just have to get ready to drive that world because the time is now for us to do that. 00:31:10.620 --> 00:31:19.440 But let's actually, you know, one of the things I think given your audience, I'd actually be really curious to hear what the audience has to say on this. @@ -1345,7 +1345,7 @@ Where do we stand? That's you know, that's one that I think the community has an incredible opportunity to stand up and say this is where we believe. 00:31:45.920 --> 00:31:58.660 -And I have no idea what the the broad consensus of data scientists are thinking, except for the ones that we've had a lot of interactions with our our very traditional White House processes. +And I have no idea what the broad consensus of data scientists are thinking, except for the ones that we've had a lot of interactions with our our very traditional White House processes. 00:31:58.660 --> 00:32:00.640 And that makes sense. diff --git a/transcripts/157-the-journal-of-open-source-software.txt b/transcripts/157-the-journal-of-open-source-software.txt index 2d8f7fd..291810f 100644 --- a/transcripts/157-the-journal-of-open-source-software.txt +++ b/transcripts/157-the-journal-of-open-source-software.txt @@ -560,7 +560,7 @@ 00:24:30 ecosystem. I was looking today, and there was a paper, it was a scikit-learn -00:24:36 contript package. I think it was the HDB scan, some some new, you know, implementation of a of a of an +00:24:36 contript package. I think it was the HDB scan, some new, you know, implementation of a of a of an 00:24:44 algorithm. And it was somebody at a university and somebody from Spotify, I think, or, or no, Shopify, diff --git a/transcripts/157-the-journal-of-open-source-software.vtt b/transcripts/157-the-journal-of-open-source-software.vtt index e30ae57..f699f53 100644 --- a/transcripts/157-the-journal-of-open-source-software.vtt +++ b/transcripts/157-the-journal-of-open-source-software.vtt @@ -844,7 +844,7 @@ career progressions. But we do have people in commercial companies as well, espe ecosystem. I was looking today, and there was a paper, it was a scikit-learn 00:24:36.260 --> 00:24:44.560 -contript package. I think it was the HDB scan, some some new, you know, implementation of a of a of an +contript package. I think it was the HDB scan, some new, you know, implementation of a of a of an 00:24:44.560 --> 00:24:51.260 algorithm. And it was somebody at a university and somebody from Spotify, I think, or, or no, Shopify, diff --git a/transcripts/496-scaf-deploys.vtt b/transcripts/496-scaf-deploys.vtt index 83e2113..edd7421 100644 --- a/transcripts/496-scaf-deploys.vtt +++ b/transcripts/496-scaf-deploys.vtt @@ -148,7 +148,7 @@ What have you been doing all this time? - I don't know, not being on your show, I guess. 00:04:09.160 --> 00:04:17.100 -I actually was super excited when you asked that I could be on here, This has been like a life goal, at least a goal since Talk Python to me has ever existed. +I actually was super excited when you asked that I could be on here, This has been like a life goal, at least a goal since Talk Python To Me has ever existed. 00:04:17.269 --> 00:04:18.959 >> I really appreciate that. That's very kind. diff --git a/transcripts/499-beeware-and-the-state-of-python-on-mobile.vtt b/transcripts/499-beeware-and-the-state-of-python-on-mobile.vtt index 3b60a0e..c58ac85 100644 --- a/transcripts/499-beeware-and-the-state-of-python-on-mobile.vtt +++ b/transcripts/499-beeware-and-the-state-of-python-on-mobile.vtt @@ -553,7 +553,7 @@ in the forest forest yeah exactly one 00:08:56.180 --> 00:09:00.120 -of the one of the the stimulus for me working on this there was a couple +of the one of the stimulus for me working on this there was a couple 00:09:00.260 --> 00:09:04.700 but one of them is that my son who is now first year university but at the time when i was sort diff --git a/transcripts/500-django-simple-deploy.txt b/transcripts/500-django-simple-deploy.txt new file mode 100644 index 0000000..dfa56fb --- /dev/null +++ b/transcripts/500-django-simple-deploy.txt @@ -0,0 +1,1332 @@ +00:00:00 We're sitting down with Eric Matthews, the educator, author, and developer behind Django Simple Deploy. + +00:00:06 If you've ever struggled with taking the final step of getting your Django app onto a live server without spending days wrestling with DevOps complexities, then give Django Simple Deploy a look. + +00:00:17 Eric shares how Django Simple Deploy automates away the boilerplate parts of deployment so you can focus on building features instead of deciphering endless configs. + +00:00:26 We'll talk about the new project's journey to 1.0, the range of hosting platforms it supports, and why it's not just for beginners. + +00:00:34 This is Talk Python to Me, episode 500, recorded February 20th, 2025. + +00:00:42 Are you ready for your host, please? + +00:00:44 You're listening to Michael Kennedy on Talk Python to Me. + +00:00:48 Live from Portland, Oregon, and this segment was made with Python. + +00:00:54 Welcome to Talk Python to Me, a weekly podcast on Python. + +00:00:57 This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is sponsored by worth recruiting worth recruiting specializes in placing senior level python developers and data scientists let worth help you find your next python opportunity at talkpython.fm/worth eric welcome back to talk python great to have you here nice to see you again michael nice to see you as well last time we talked about your book right and now your project your deployment project i love talking about deployment because i think it's such an enabling technology or enabling skill and your project Django Simple Deploy is certainly in that realm. + +00:02:00 Yes, I have appreciated your comments about deployment over the years. + +00:02:04 Thanks. Well, they're a hard one. My skills in deployment are a hard one. + +00:02:09 I kind of want to talk a little bit about the journey for people, but not yet. + +00:02:16 Let's get a quick introduction of who is Eric. What do you do? + +00:02:20 What's new in your life since the last time you were on the show? + +00:02:22 Oh, gosh, yeah. It's all context for this project because this project ties together a lot of the different things that I've done. + +00:02:29 Short version, I grew up in New Hampshire, moved to New York City, then moved to Alaska. + +00:02:33 Just last year, I moved to Western North Carolina. + +00:02:38 And I had a background in science. I studied physics in college and got pulled into teaching. + +00:02:43 spent a long time teaching math and science, fifth grade through twelfth grade, which was super fun because it's not at the level of undergrad or grad work, but the questions young people ask always make things interesting. And I got pulled into programming because I always did it on the side. And I kind of felt like, I always felt like at some point, I might focus more on programming and less on classroom teaching. So for the past five, six years I've been a full-time writer and programmer. + +00:03:15 It's been enjoyable. Yeah. And congrats on the move. You really like to sample different weather systems, different environments. + +00:03:26 You're going to get a different summer experience in North Carolina than you might in Alaska. I know it gets warm somewhat there, but + +00:03:32 it's warm a lot in North Carolina. + +00:03:34 Yeah. We're in the mountains, so we can have cool nights at least. + +00:03:37 Yeah. Lovely. My brother lives there and I think North Carolina is a really neat place. + +00:03:41 It is, yes. + +00:03:43 Yeah, yeah, yeah. So let's talk about this going from science to teaching to programming just a little bit, because I did a very similar journey. I think mine was accelerated in terms of its time frame, but I was in grad school. I studied math and then taught math, calculus, linear algebra, those kinds of things at universities at GTA, and then kind of got into programming as part of it. And then I'm like, wow, this is really fun. How do I, how do I do more of this? You know what I mean? And after evaluating it, just academia didn't seem like a thing I really wanted to try to chase. And so I basically, I took one or two programming classes, but otherwise, other than that, I taught myself programming a number of years ago. That is kind of scary. Last century, let's say. And I don't know, I feel totally confident about it these days, but at first it was, you know, do I belong here? + +00:04:35 All these things seem hard. What am I doing wrong? Should I have gotten a CS degree? I don't think any of those things are true. I think just, you know, as I learned hanging out with CS people, it was hard for them too. Like it's, but maybe talk a little bit about your journey, right? Because I'm sure it would be helpful to other people. There's many people out there who are doing something somewhat similar. + +00:04:55 Yeah. Questions about academia kind of steered me this way as well. I studied physics in undergrad because I started as a chemical engineer because I had a great chemistry class in high school. But I found that engineering classes felt like being taught how to solve other people's problems. And my favorite class as an undergrad or as a freshman was physics because it was the only class that really seemed about truly trying to understand the world. And I love that. So that's what led me to focus on physics. And I was going to be a particle physicist, but I did not want to + +00:05:26 be... + +00:05:26 Wanted to spend some time at sun or maybe outside of Chicago or wherever. Yeah. Okay. + +00:05:31 Yeah. And I, yeah, when I was really young, I heard about atom smashers. I thought somebody like literally had hammers where they smash atoms. + +00:05:40 So every stage, yes, every stage I learned more about it was more and more fascinating, but I didn't want to be a student forever. And so I want to do something outside of being a student before pursuing the graduate work that would get to get me to be a part of a physicist. + +00:05:57 So when I was looking what to do, I had been a tutor in undergrad, and I tutored calculus and physics. + +00:06:03 And I tutored those classes because I watched classmates start to hate math and physics because of how they were taught. + +00:06:12 And that's the only reason people hated those topics. + +00:06:14 And it was really humbling because I was 18, 19 years old and tutoring people in their 30s and 40s who were doing mid-career changes. + +00:06:23 Speak of the field, like you maybe don't necessarily belong there telling someone how to do something. + +00:06:29 But at the same time, you probably knew it really well, right? + +00:06:32 Yeah. + +00:06:33 Yeah. + +00:06:34 And I felt like I belonged, but it was really humbling as far as just sorting out like school systems and where to go. + +00:06:40 So anyway, that led me into teaching. + +00:06:42 And I thought I would just teach for a couple of years and go back to science. + +00:06:46 But I found that the challenge of reaching everybody in the classroom was as hard and as satisfying as hard science. + +00:06:53 And so I just love teaching. + +00:06:55 I love what it brings up for other people. + +00:06:57 And, yeah, it's always stayed interesting. + +00:07:02 For me, the transition into more of a focus on programming was my father died. + +00:07:07 I've told this story a few times over the years about the origin of the book Python Crash Course. + +00:07:11 But my father died in 2011. + +00:07:13 And my mom asked. + +00:07:14 He was a programmer. + +00:07:15 That's how I first learned to program. + +00:07:17 My mom asked me to go through his computer and she found all these, or she asked me to go through his computer to see if there was anything worth keeping. + +00:07:24 And there wasn't, but for me, it was a really personal experience of seeing all these projects he had started. + +00:07:30 They would never get finished. + +00:07:31 And I realized that I had a bunch of those programming projects as well. + +00:07:35 And so that kind of was a turning point for me to get me out of the practice of just building a bunch of side projects and start to focus on one or two real world projects that other people would benefit from. + +00:07:47 That's a great story. + +00:07:48 That's certainly one of the challenges of you can build a project and you can get it to run, but you're maybe halfway there, getting it out to the world in a real world sort of way. + +00:08:00 You've got all these challenges of, I need to pick a place to run my app. + +00:08:05 Wait a minute, databases? + +00:08:06 Why are these migrations so hard? + +00:08:08 How do I do migrations? + +00:08:10 I got the first one up and now I can't change it. + +00:08:12 just breaks it doesn't run anymore and you know there's just all these these aspects not even taking into account the stuff you got to add to go from a toy project to making it real logging error handling etc etc right yeah + +00:08:25 i mean this is all a good segue into the topic of today i mean django simple deployed to kind of put it out there for people who haven't heard of it um it's a um package that automates deployment uh django deployments so if your project runs locally on your computer and you get to the point where you want to make it public for other people to use if you haven't already gone through the deployment process for most people that's a cliff and they kind of just fall off i spend all this time learning django all the time building this private they really care about solves the problem does it better than other people's work because you know it's new and whatnot and then they go to deploy it and they find that they have to spend so much time reading a platform's documentation um tweaking their settings um adding this file file and then kind of crossing your fingers and hoping that it works. + +00:09:14 Every time I've gone through that process, I've kind of thought to myself, this is all just boilerplate. + +00:09:20 Like we're just being told to, everybody's being told to add the same settings, make the same modifications. + +00:09:25 And so I had that shower thought of like, automation. + +00:09:29 Yeah. + +00:09:29 Yeah. + +00:09:30 I had that shower thought of like, okay, how could we do this configuration from the Django side? + +00:09:35 So when we're just looking at the user's computer, you've got a project that's working We know it works. + +00:09:42 How can we do all the configuration automatically so that you then just push your automatically, correctly configured project to your hosting platform? + +00:09:51 So Jango Simple Deploy does that. + +00:09:52 You pip install on Jango Simple Deploy. + +00:09:54 You need to have a plugin for the platform that you're using. + +00:09:57 So Fly.io is a go-to example. + +00:10:01 And then you add Jango Simple Deploy to installed apps. + +00:10:04 And then there's two modes for this project to work. + +00:10:08 The fully automated mode is easier to talk about because you run one command, bifimanage.py deploy --automate all. + +00:10:14 And if you do that and you have that platform CLI installed, it configures your project, makes a git commit, and pushes your project, and your project appears in a new browser window, which feels like magic. + +00:10:28 That's kind of magical, actually. + +00:10:30 Yeah, yeah. + +00:10:32 All right. + +00:10:32 So I want to talk about what platforms are supported. + +00:10:36 But before we do, maybe we could talk more broadly about just hosting platforms that you've considered, that you see out there, that people publish Django apps to, right? + +00:10:47 On one hand, we've got really, really straightforward platforms. + +00:10:50 On the other, you can get as complicated as you want, basically. + +00:10:53 You play the game of how many AWS services could you combine into one Django app, which, you know, that's kind of like the obfuscated code contest. + +00:11:04 How much can you fit under one line that's unlegible but still does the thing or whatever, right? + +00:11:09 You know, we started this with a conversation about our backgrounds. + +00:11:13 And so kind of weaving this for a little bit back into that background story, I do remember one of my transitions to doing more professional programming work was when I was teaching, I would write programs to address inefficiencies and difficulties we had in our school for our staff. + +00:11:28 And so I remember like exactly this, getting something that like automated report card generation for our staff. + +00:11:36 And I got it working on my computer. + +00:11:38 So like for me, writing report cards was so much easier than for everybody else. + +00:11:42 And so I learned Django. + +00:11:43 I'm like, cool, I can share this with everybody else. + +00:11:45 And I tried to deploy it and it just, I could not do it. + +00:11:49 It just fell apart. + +00:11:51 And I sort of figured it out. + +00:11:54 So my first deployment platform was a laptop in a closet in my school connected directly to the school network. + +00:12:02 So the deployment worked for people on our network, and it was only out of the kindness of our IT staff that would allow me to do that. + +00:12:09 Clearly not a solution for most people. + +00:12:12 It's really hard to deploy. Let me just deploy my machine to the school. + +00:12:17 Yes, yes. Update. + +00:12:18 Let me go to that closet. + +00:12:21 Yeah, yeah, yeah. + +00:12:23 I'm going to be in here for a few hours. I'm programming on the server. + +00:12:26 Yeah. + +00:12:27 So, okay, Heroku is the first one to talk about because Heroku was one of the first platforms that really automated a lot of the DevOps work and allowed us the classic sales pitch for Heroku was run and get push Heroku main and your project is live. + +00:12:43 There's always a bit more to it than that, including that configuration to make your project work on Heroku. + +00:12:49 So the platforms I start thinking about are the ones that manage the DevOps aspect for you. + +00:12:56 Heroku, Flyto.io, Platform.sh, Python Anywhere is in there. + +00:13:03 So to take a step back even further, this project started because I used Heroku for a while. + +00:13:10 In Python Crash Course, the first edition, I taught people, I walked people through deploying a Django project to Heroku. + +00:13:17 is the only part of the book that felt like I'm not really teaching anything. + +00:13:20 I'm just telling people to write certain things and add certain files. + +00:13:24 A bit of explanation about what it does, but mostly it's, you know, make these changes and cross your fingers that you type them all right. + +00:13:31 So I wrote a Heroku build pack that did the configuration automatically, but it did it on Heroku servers. + +00:13:41 So you'd specify this build pack. + +00:13:42 It would load that build pack onto Heroku servers, and then all your configuration changes would happen on their end. + +00:13:48 Tell us what is a build pack. + +00:13:50 I'm not sure that I can tell you. + +00:13:52 It's basically, I said my short take is... + +00:13:54 I'll tell you, I've never built a build pack myself, and I've deployed a lot of Python, so... + +00:13:58 Yeah, my short description is it's a set of programs, body of code that configures your project for running on that specific platform. + +00:14:10 Got it, okay. + +00:14:11 And so typically it's generic and it's not targeting, it's not trying to do any changes to your project. + +00:14:19 We won't go further. + +00:14:20 I'll say something wrong. + +00:14:21 Yeah, yeah, yeah, sounds good. + +00:14:22 But the takeaway for that is it felt like doing Heroku's work for them. + +00:14:26 And so it was nice for anybody trying to push Django to Heroku, but that boundary of like, what's our work? + +00:14:31 What's their work? + +00:14:31 Are we doing work free for them? + +00:14:34 That's an interesting question. + +00:14:35 And it really felt like it was skewed towards doing their work. + +00:14:38 Right. + +00:14:39 Well, I've done it once or twice, deployed an app to Heroku, never for real work, but to, you know, play around with it and sort of check it out. + +00:14:47 Because Heroku was definitely a darling of the Python space and certainly of the Python tutorials, right? + +00:14:56 Because for a long time, they had a free tier. + +00:14:59 And so everybody who needed to deploy something was like, here's how you put it on Heroku to sort of complete the loop and get it out there in the world, right? + +00:15:09 Right. And then, yeah, you know, a testament to Heroku is how long it has worked without a lot of attention from them. And so they lost a lot of trust in the community, not just from doing away with their free tier. It's really hard to maintain a free tier. It's just puts a big target on the platform for abuse. + +00:15:29 But it's about some reliability issues and communication issues around issues on the platform. + +00:15:37 So there was an exodus from Heroku for a while. + +00:15:40 And I was working on this automation problem for a while. + +00:15:44 And I had that shower thought of like, okay, all that work that I'm doing through the Buildpack on Heroku's end could be done through a management command with NJNGO. + +00:15:52 You can do all the changes to your project on your computer if you know what to do. + +00:15:58 So internally, Django simple deploy is just a single management command, that deploy command. + +00:16:04 And when you run that command, based on the plugin that you have installed, it knows which platform you're targeting. + +00:16:10 And then basically, I've just spent hours pouring over the documentation for a few different platforms. + +00:16:16 And so we pick up the platform that you're trying to deploy to, make all the changes necessary for a basic initial deployment to that platform. + +00:16:24 And then there's two ways you can push your project. + +00:16:26 The fully automated mode puts all those changes into a single commit, runs that platform's deploy command, and pops up your project in a new window. + +00:16:35 Yeah, and the transport mechanism is Git, right? + +00:16:41 Good question. + +00:16:43 Not necessarily. + +00:16:44 The transport mechanism is whatever that platform uses. + +00:16:49 So for Fly, Django simple deploy makes the configuration changes for your project. + +00:16:56 and then it calls the fly deploy command. + +00:16:59 So I don't even have to know what fly deploy is using behind the scenes. + +00:17:04 You have the CLI installed, simple deploy calls that deploy command. + +00:17:09 For Heroku, when it gets to that deployment stage, in the automated mode, Jingo simple deploy runs git push Heroku main. + +00:17:17 And so it's one of the nice things about this project, and part of the reason it has taken so long to get to 1.0 is quite adaptable to different approaches. + +00:17:27 Yeah, that's pretty wild because you're just automating the CLI for that platform, right? + +00:17:32 Right. + +00:17:34 And Chris is commenting, there should be one obvious way to deploy your Django project to a service. + +00:17:40 Yeah, he's right. + +00:17:41 The nice thing about this is Django Simple Deploy becomes this abstraction layer between your work and the platform. + +00:17:49 So it creates a nice, consistent interface for deploying to a wide range of platforms. + +00:17:54 And when a platform inevitably changes their deployment process or configuration requirements, as long as one person makes that update to the Django SEP deploy plugin, everybody else gets to just keep using that consistent interface for how you do your initial deployment, + +00:18:10 which is really, really nice. + +00:18:12 Yeah, it is really nice. + +00:18:14 You don't have to worry about all the details and also the ability to move without losing all of your expertise. + +00:18:23 This portion of Talk Python to Me is brought to you by Worth Recruiting. + +00:18:27 Are you tired of applying for jobs and never hearing back? + +00:18:30 Have you been getting the runaround or having trouble making it past the AI resume screeners that act as the new gatekeepers for your next level Python job? + +00:18:39 You should reach out to Worth Recruiting. + +00:18:41 Worth Recruiting specializes in placing senior level Python developers and data scientists. + +00:18:46 They work directly with hiring managers at startups, helping them grow their software engineering and data science teams. + +00:18:53 With Worth, it's not just connecting you with the company. + +00:18:56 It will guide you through the interview process and help make sure you're ready with their detailed preparation approach. + +00:19:01 They can even coach you on salary negotiations and other important decision-making processes. + +00:19:07 So if you're ready to see what new opportunities are out there for you, reach out to Worth Recruiting. + +00:19:12 Let them be your partner and specialist to find the right Python developer or data scientist position for you. + +00:19:18 Fill out their short contact form at talkpython.fm/worth. + +00:19:23 It only takes a minute. + +00:19:24 That's talkpython.fm/worth. + +00:19:27 The link is in your podcast player's show notes. + +00:19:30 Thank you to Worth Recruiting for supporting the show. + +00:19:33 This is where it gets really fun to talk about this project because, you know, people think when people first hear about this project, they think it's just for beginners. + +00:19:42 It's just for people who already know deployment. + +00:19:46 But where is I going with that? + +00:19:48 Oh, so thinking about writers, authors, myself first, but content creators, tutorial writers, presenters. + +00:19:56 If you're wanting to include deployment as part of your process that you're teaching people, then a lot of people have chosen not to do that because as soon as you include deployment, you're now tied to that platform's stability. + +00:20:13 As soon as they change their process, you've got to update your materials or it's broken. + +00:20:17 and that's not appealing. + +00:20:19 So for somebody who uses Django Simply Deploy in their tutorial, again, as long as one of us keeps the plugin updated to work on that platform, your learning resources stays the same. + +00:20:31 That is really nice. + +00:20:33 And whether you're teaching a college class or you're just writing something for your blog, it is really nice. + +00:20:38 Yeah, and I think about the Django Girls workshops. + +00:20:41 And so part of that, the goal of that conclusion of deploying your project is just to see a live. + +00:20:47 Like, why are we doing this if we can't see a line? + +00:20:49 You don't necessarily want to spend a whole bunch of time focused on deployment. + +00:20:52 And so if you can kind of really crunch that into like, okay, we're going to run these commands. + +00:20:57 We have a working deployment. + +00:20:58 It's not about hiding deployment from people. + +00:21:01 Because one of the coolest parts of this project is that it contains all those configuration changes in a single Git commit. + +00:21:07 Is there a verbose mode where you get to show all the little steps it's doing and stuff? + +00:21:11 Yeah, it is verbose by default. + +00:21:15 So you get to basically it streams and dumps all the platform's output as it's deploying, which is kind of important because if you don't do that, you just see this blank screen for two, three, seven + +00:21:26 minutes. + +00:21:27 Yeah, it's just sitting there spinning. + +00:21:28 Yeah. + +00:21:31 But to slow this down, take a step back for a minute. + +00:21:33 I've mentioned the automated mode where you include this stash-tash automate all. + +00:21:37 That's easiest to talk about because it boils down this deployment process to three steps. + +00:21:43 I don't actually recommend that for most people because it makes a Git commit for you, runs a deploy command, and maybe you want to have a little more control about that process. + +00:21:53 There's a configuration-only mode, which is every bit as helpful, but it lets you kind of take a little more control. + +00:22:01 And so what you do for most platforms, you have to create a resource, basically a deployed project, an empty project on that platform, and then you run deploy. + +00:22:11 And Django Simple Deploy recognizes the resource you just created on that platform, configures your project, and then it stops. + +00:22:21 And so you now have a configured project, and you can run git diff, and you get to see all the configuration + +00:22:27 changes that were made on your behalf. + +00:22:30 So you don't have to just trust it. + +00:22:32 Yeah, yeah. + +00:22:33 So when I shared this on the Django Discord yesterday, somebody was excited about it, and then I said something about that commit that was made. + +00:22:40 And they said, oh, it makes commits for you. + +00:22:41 I'm not sure I want to use that. + +00:22:43 Is it okay? + +00:22:43 You can do the longer mode where you get to review that. + +00:22:48 And then you can see all the files that were created for that platform. + +00:22:52 You can see the settings changes that were made to make your project work. + +00:22:56 And then you can choose to make that commit. + +00:22:58 You can choose to push it or revert it. + +00:23:01 Even if you use the automated mode, it's still all the configuration is one commit. + +00:23:05 And so you can just revert that commit and you're right back to your project. + +00:23:09 I'll throw one more piece in there right away. + +00:23:10 one of the priorities of the project is that it does not change the local behavior of your project. + +00:23:16 And so when I run the test suite, it tests that all the configuration changes are correct for the target platform, but it also runs run server again locally and makes sure that we haven't impacted the local running of your project. + +00:23:32 Oh, interesting. It'll run it to make sure it basically starts. Nice. + +00:23:36 Yeah, you know, this automated versus config sort of deal, it reminds me a little bit of how maybe DBAs feel about ORMs. + +00:23:45 You know, like they're like, oh, you can't just let it run arbitrary queries against my database. + +00:23:50 Like, how do we know it's doing the right thing? + +00:23:52 I'm sure you feel that way for a while. + +00:23:55 But eventually, all right, we've reviewed them. + +00:23:58 They're fine, right? + +00:23:59 So maybe this config thing you do a couple of times, you're like, yeah, that's + +00:24:03 what I would do anyway. + +00:24:04 Just run it, right? + +00:24:05 Because what else are you going to do? + +00:24:07 You're probably going to create a script that does the things you expect it to do anyway for yourself as a bash script or something, right? + +00:24:13 And so might as well be a Python script. + +00:24:15 Yeah. + +00:24:15 I mean, it's funny hearing you talk about this because this project started as a pile of bash scripts. + +00:24:20 And so, yeah. + +00:24:22 I got a pile of bash scripts on my server. + +00:24:24 They do things. + +00:24:25 Yeah. + +00:24:25 One of the pre-1.0 tasks was a long issue that was just about converting all these shell scripts to Python. + +00:24:33 And mostly that was about creating, making this work cross-platform. + +00:24:37 So it's basically taken me like four years to get this to 1.0. + +00:24:40 And it's that 80-20 principle turned into like 595 principle. + +00:24:49 So another reason, so a good thing to kind of move into, another reason it took so long to get to 1.0 was originally this was all one library. + +00:24:58 So it was just Django simple deploy. + +00:25:00 And originally you would run managed.py deploy --platform by.io. + +00:25:06 And so the entire project contained all the code for all the different platforms. + +00:25:12 And that felt better than the original project just targeting Heroku because you have choice. + +00:25:17 But it felt like a nightmare for maintenance. + +00:25:19 Anytime any project changes, it's going to need an update and it's just going to grow exponentially. + +00:25:27 Yeah, it becomes combinatorially complex in terms of dependencies, which + +00:25:31 is not a good order of magnitude. + +00:25:35 It's not good. + +00:25:36 Yeah. + +00:25:37 So I had some people kind of very kindheartedly nudging me towards just doing a 1.0 release to get over that fear of a 1.0 release. + +00:25:46 But it was always about having – I had a vision that the mature version of this project should be plug-in based. + +00:25:54 So the core project, Django Simple Deploy, is small. + +00:25:58 It's platform agnostic. + +00:25:59 All it does is inspect your project, inspect your system, and then hand out the rest of the work to a plugin. + +00:26:05 And so currently there's a plugin targeting apply.io, plugin for platform.sh, platform for Heroku. + +00:26:12 I have a proof of concept plugin for CodeRed, and I'm working on a proof of concept for DigitalOcean. + +00:26:20 Yeah, so many cool things to talk about with this. + +00:26:23 Yeah, yeah. + +00:26:24 Before we go, I do want to maybe just go through some of these hosts and let you talk about maybe why they're included. + +00:26:32 Which ones are, there's stuff out there, obviously, that's not included. + +00:26:35 Why is it not included? + +00:26:37 So Heroku, obvious, especially when you started, when they had the free tier and stuff like that. + +00:26:43 I kind of want to get them, I know people are angry at them. + +00:26:45 I understand that. + +00:26:46 But also, I kind of want to give them a little bit of a pass, like offering free hosting to the world where people just put up malware and other badness and use the trust of your domain to sort of leverage that, it's challenging. + +00:27:00 And I don't know. + +00:27:01 I understand people are frustrated with them, but that's tough. + +00:27:05 So Heroku, platform as a service, you push your stuff up, you say, here's my code, here's how you start it, run it for me, please. + +00:27:12 There's not a lot of Linux. + +00:27:13 There's not a lot of shell, right? + +00:27:15 They kind of take care of that for you, yes? + +00:27:17 They do. + +00:27:18 And there's still, yeah. + +00:27:20 I've had conversations with people who, with founders of a number of these platforms over the years, because it's always interesting to talk about this. + +00:27:28 And platforms would love to see an easier way to deploy Jango projects. + +00:27:32 One of those founders said something really insightful at one point, and I appreciated the honesty, is that we're all trying to be second best for beginners. + +00:27:39 Because if you draw that crowd, you're drawing more support than benefit in the short term. + +00:27:45 And so, yeah, for a long time, Foroku paid that cost. + +00:27:49 they were willing to be the go-to for everybody. + +00:27:51 So there's still a lot of projects running long-term on Heroku. + +00:27:55 I know some important Python ones that are there. + +00:27:57 Yeah. + +00:27:57 Yeah. + +00:27:58 Okay. + +00:27:58 And then we got fly.io and platform.sh. + +00:28:03 What's the story of these? + +00:28:05 Honestly, I haven't used either of them. + +00:28:06 I've heard of them. + +00:28:07 I haven't used them. + +00:28:08 So the selling point for fly.io is they're an edge-based platform. + +00:28:14 And so they take your, if I'm saying this right, They take your project and they push your application code to a bunch of different nodes. + +00:28:25 And so your code is... + +00:28:26 Kind of like a CDN, right? + +00:28:28 Yeah, CDN for your application code. + +00:28:30 And so your code is right close to your end users. + +00:28:33 And so I don't need it, but they've built it well enough that I can just push a project there and it works. + +00:28:39 And I'm a big fan of, you know, if your project does what you need it to, is it a good question? + +00:28:43 Is it good enough? + +00:28:43 And so it's been good enough for me for a lot. + +00:28:46 And I know it's targeting some more complex use cases. + +00:28:50 I have so many questions. + +00:28:52 It's so interesting. + +00:28:53 Well, the first thing that comes to mind is what about databases, right? + +00:28:57 If your app is running, yeah, you've got to have some way to correlate and centralize and make the currency of the database and all that work, yeah. + +00:29:05 They've had some fantastic people working for them, and they have some fantastic blog posts. + +00:29:09 So they've done really interesting cutting-edge work around distributed SQLite. + +00:29:16 databases. And so, yeah, they are quite familiar with that question and they + +00:29:20 have answered it well and they are answering it well. + +00:29:24 If you can answer that question well, then you can get some stuff that flies, you know? I mean, I guess, hence the name. I didn't even do that on purpose. + +00:29:32 I've been using bunny.net, which is a really nice CDN platform, but just for static content, not for application code that talks to the database. + +00:29:46 And I understand how you would replicate the database, but the challenge is if you have chatty rights, all of a sudden you're pushing your database away from your app, which has its own sort of latency and stuff. + +00:30:00 But what an interesting idea. + +00:30:01 Okay, and Platform.sh? + +00:30:03 Platform.sh started as a PHP, I believe, started as a PHP-only platform, but then expanded to support other languages. + +00:30:12 And so they have been sponsors of DjangoCon for a long time. + +00:30:17 And I really enjoy working with their platform. + +00:30:21 Deployment is pretty straightforward. + +00:30:23 And then, you know, one of the things I look for in these platform as a service is, so once you push it, I'm very happy to have them managing the DevOps work. + +00:30:32 I also want to be able to like get into the server and run commands and see what's going on. + +00:30:36 And so they make it easy to do that. + +00:30:38 Yeah, if things are not starting or they're crashing or something, sometimes just go, just 30 seconds at a glance at the log file or something. + +00:30:46 Oh, yeah, or maybe why is that file? + +00:30:49 That file is actually missing. + +00:30:50 Oh, that's weird. + +00:30:51 Let me see if I can just get that. + +00:30:52 And then you're back to like, okay, back to platform as a service, and you're good, you know? + +00:30:55 Yeah, and that's, you know, platform as a service certainly works for people who know nothing about, you know, hopping into a server and running commands. + +00:31:03 But they're also for, like, people who know how to do that and don't want to do that. + +00:31:05 And so a service like platform.sh that makes it easy to do both is also, I see it as an onboarding for people who have something worth deploying, don't know DevOps yet, but letting them hop in and start to do that is pretty interesting. + +00:31:21 Yeah, it's very interesting. + +00:31:22 Okay, so I can run Python manage.py deploy --automate all, and that will take my app and put it into whatever destination you pick, right, of those three. + +00:31:32 For these platforms as a service, a lot of times you have to have a database. + +00:31:36 Sometimes that's a managed Postgres or whatever you might have data that has to import initially. + +00:31:44 You know, you're like, well, here's all the counties in the United States and we have to query them for stuff. + +00:31:48 So we preload that at first. + +00:31:50 How does all that get connected and happen, right? + +00:31:53 If I had like no database and I just have a pure Django app that just runs Python, easy. + +00:31:58 I see how this works, but there's settings and stuff here that vary by production versus dev. + +00:32:04 Yeah, fun question. + +00:32:06 So I've done most of the development work on Django Simple Deploy. + +00:32:10 I've actually run sprints at several PyCons and several Django Cons. + +00:32:14 And this project has brought together like 8 to 15 people at each sprint. + +00:32:19 It draws some interest. + +00:32:20 And so the project would not be where it is without those contributions. + +00:32:24 A lot of those people have made some contributions and not done a whole lot outside of the conferences, but that whole group of people has been available to just bounce ideas off of. + +00:32:34 And so a lot of the pre-1.0 work was also like me getting rid of all the baggage that just kind of only made sense to me, or you really need to have years of contact with the project to make sense of it. + +00:32:44 And so now it's at a place where the structure is pretty consistent. + +00:32:49 And so one of the things I'm looking forward to is somebody else who knows Fly.io managing the Fly.io plugin more than me. + +00:32:57 And so they don't need to go deal with Chango Simple Deploy. + +00:33:01 They can focus on the parts of the script that do the things you were just talking about. + +00:33:06 So all the configuration has been largely my choice. + +00:33:09 You know, okay, on Heroku, we'll choose this Postgres instance because it's cheaper, because it lets people get started. + +00:33:17 It's possible for somebody to either take the Heroku plugin and add a few more CLI flags to give you a bit more choice, or just write your own Heroku plugin. + +00:33:28 And so right now the plugin for Heroku is dsd-heroku. + +00:33:33 Somebody else could do like DSD Heroku-large, and it's meant for scaling large or deploying large-scale Heroku projects. + +00:33:43 And so that's not a, I don't want to do that work. + +00:33:45 They should write their own. + +00:33:46 That's if you have a use case that you know well, you want to automate. + +00:33:51 Simple Deploy just becomes this, again, abstraction layer for, you know, you have a consistent interface for running that deployment from within Django. + +00:34:00 Yeah, there's a nice plugin model, which really opens things up. + +00:34:04 I suppose the short answer is probably, well, the Django ORM will create the database structure for you if you could just tell it where the database is, right? + +00:34:14 Right. + +00:34:15 And I know certain apps I've built, if they have data that has to be preloaded, I'll just have a little check like, hey, has this table got any data in it? + +00:34:24 If not, run the load up the database the first time. + +00:34:27 Otherwise, assume it's all good and keep going. + +00:34:30 that pretty much handles it? + +00:34:31 I think, you know, it's kind of two directions I'll take this. + +00:34:35 You know, a short list of the projects I've done over the years, they tend to be smaller, but they tend to be impactful. + +00:34:40 So one project, automated report cards for students. + +00:34:45 I used to work with struggling students, students who were plenty smart, but life had happened and they had gotten off track with school. + +00:34:52 And so they just had holes in their, in their graduation requirements. + +00:34:58 And so those students wanted to graduate, but they didn't know where to focus their efforts. + +00:35:02 And so looking at a transcript was really hard because it's just a bunch of letters and numbers. + +00:35:08 So I wrote a program that automated a generation of visual transcripts. + +00:35:13 So I'd read their text-based transcript and then give them basically a chart, a bunch of gauges. + +00:35:17 You just look at it in 30 seconds, know which classes you needed to focus on. + +00:35:22 And it also went from deficiencies to strengths. + +00:35:25 So traditionally, we talked to students who are struggling and saying, you need to do this, you need to do that. + +00:35:30 You could see where they needed to do their work, but you could also see where they had been shining. + +00:35:34 You could see where the strengths were. + +00:35:36 And so I wrote a project. + +00:35:38 I worked with whale researchers. + +00:35:39 And people don't know this. + +00:35:41 Sperm whales have learned to pick the fish off of – I'll take a step back. + +00:35:47 People catch halibut, fish for halibut by putting literally miles long cables on the ocean floor or lines with a hook every six to 10 feet. + +00:35:57 And so you put a thousand hooks on the ocean floor, leave it overnight or pull it. + +00:36:01 And that's how you catch halibut. + +00:36:04 And sperm whales have learned to listen for the sound of the transmission changes when somebody puts their boat into gear to pull those hooks. + +00:36:13 They swim towards that sound and they just pick the fish off those lines. + +00:36:16 Wow. + +00:36:17 And so these researchers were starting to put transponders on boats that would allow anybody who saw a sperm whale to report that. + +00:36:26 And then you could kind of time your fishing sets to be away from the sperm whales. + +00:36:31 And so I wrote a project that just like these researchers didn't know how to get their data from one place to another. + +00:36:36 And so I just wrote a small Django project that listened for that data and gave them a visible place where they could go to it and share it. + +00:36:45 And one other, oh, I lived in Southeast Alaska, and we were having increasing landslides, more frequent landslides, and more impactful. + +00:36:54 And so I wrote a small project that examines real-time stream gauge data and turns that into a measure of landslide risk. + +00:37:03 And they gave a talk about that last year at PyCon. + +00:37:06 All those projects are small. + +00:37:07 They're never going to scale to millions of users. + +00:37:10 They all need a database. + +00:37:11 And so they all fit this model of something like I personally would build. + +00:37:15 And if I didn't know deployment, those would not have ended up being useful. + +00:37:18 And so this tool takes all those people out there right now who are getting the skills to build something that works locally. + +00:37:24 And they don't have to go through that three weeks to three months of learning how to deploy things. + +00:37:30 So I appreciate you bringing up the questions of what happens with these larger scale projects. + +00:37:36 But there's also a whole class before you even get there of projects that would make the world better if they could be easily deployed. + +00:37:43 Such a challenge because if you go down the, there's so many paths that people get led down that are not good paths. + +00:37:50 You know, it's sort of the architectural equivalent. + +00:37:52 I guess even just the DevOps equivalent of you're not Microsoft, you're not Google, you're not Netflix. + +00:37:59 You probably don't need the chaos monkey running around your whale research project. + +00:38:03 Right, right. + +00:38:04 But people read that and they're like, well, I saw a really awesome conference talk and people loved it about how they had like geo-replication of their database and stuff. + +00:38:11 You're like, yes. + +00:38:12 Interesting. + +00:38:13 not for you. + +00:38:14 So the other direction that's interesting to finish out this discussion, so I'm targeting a VPS right now. + +00:38:20 I have deployed a Django project to DigitalOcean just on the bare VPS. + +00:38:25 For people less familiar with that virtual private server, it's basically a brand new virtual Linux machine that you can push your project to. + +00:38:33 And so this is more complicated because rather than just like making a few configuration changes to your project and then running a platform's deploy command, you have to do that whole update the server, install Django, set up a Git push if you want to do that approach, make the right port changes and whatnot, expose that. + +00:38:54 So it's more work for me to get that proof of concept done. + +00:38:57 But once it's done, the work I'm doing to write a plugin for DigitalOcean would work just as well for Hetzner, just as well for Linode. + +00:39:05 And then it opens up to all the different approaches that people take to deployment. + +00:39:11 One of the interesting questions about Django Simple Deploy is what are the boundaries? + +00:39:14 So what is simple? + +00:39:16 And I don't know the answer to that yet. + +00:39:19 It shouldn't become tube loaded. + +00:39:21 It shouldn't re-implement the entire CLI for a platform. + +00:39:26 But the bounds are pretty large. + +00:39:29 So you could have a DigitalOcean that targets NGINX Benicorn. + +00:39:36 You could have one that targets using the Doku approach. + +00:39:40 You could have one that targets external databases. + +00:39:43 There's really, it's a pretty open platform for automating deployment processes. + +00:39:51 Yeah, absolutely. + +00:39:52 Lots of people. + +00:39:53 I mean, once you knock out one of those virtual servers, it's really, really close to seeing, well, anything that can run a Linux VM pretty much can do it. + +00:40:02 And you were talking about Nginx. + +00:40:03 You know, you might, I don't know, have you considered Caddy? + +00:40:06 CaddyServer.com? + +00:40:07 As I was saying Nginx, I was like, oh, wait, there's another one that I should be saying. + +00:40:11 Well, I use Nginx for all of my stuff because it's gotten over the years, gotten more and more complicated. + +00:40:18 However, you've got to run, you've got to independently manage Let's Encrypt, and there's this sort of scaling up process. + +00:40:27 First, you've got to put the non-encrypted version up, so that cert bot and all the Let's Encrypt stuff can find it. + +00:40:34 Then you create the SSL certificate. + +00:40:36 Then you change the configuration again, because it won't run saying it's SSL until it actually has this, like there's this scaffolding step that's always messy. + +00:40:45 And if I think like easy getting started for people, Caddy, which manages the security and all that and has a real simple configuration language, it seems like it might be a good option. + +00:40:56 And it has 62,000 GitHub stars. + +00:40:59 It's, I think a person or two uses it, you know? + +00:41:02 That's kind of neat. + +00:41:03 Yeah, last week, before I started the DigitalOcean plugin, I asked on Mastodon how people are deploying to VPS these days. + +00:41:13 And Cady was one of the things that came up. + +00:41:14 It was super interesting to see the range of approaches that people are using. + +00:41:18 So, you know, as I go through this, I'm kind of asking the question for basic initial deployment for somebody new to this platform, what are some sane choices? + +00:41:28 So Cady or Nginx, Postgres or SQLite. + +00:41:32 So I'll make those choices. + +00:41:34 And one of the open questions that I don't have clear answers to is, you know, where are the boundaries around those plugins? + +00:41:41 So I'll certainly implement a --DB flag that lets you choose like Postgres or SQLite. + +00:41:48 But I don't want to do a flag for like --approach, daku, whatnot. + +00:41:53 At some point, it makes more sense to have other people write other plugins and just foster that plugin ecosystem. + +00:42:00 so another consideration and kim out in the audience was hinting at this before and it's certainly the way that i'm doing things these days is docker so let me just give you an example like yeah you can go all in and you could run on like the doc like eks and all these kinds of things you can go way way deep on it or you can just do stuff as simple as like hey i want i want the front end web server to be caddy i could install caddy on the server using apt or whatever but then you those kind of update more slowly, you have less control over it. You could just as equally well run a catty Docker image on that machine and have basically a zero touch experience on the server itself, other than just having a Docker thing there, right? And you can do whatever you want to that Docker image. You could even have a catty Django simple deployed one based off their original one or, you know, things like that where you've got it exactly configured. What's the story with Docker? And how have you been thinking about that? Because once you start to go to virtual or you start to get into some of these processes and some of these ideas? + +00:43:03 Yeah, great question. + +00:43:04 And it certainly has come up multiple times. + +00:43:06 My main experience with Docker is working with Fly because Fly is a Docker-based approach. + +00:43:15 I have not written my own Docker files, but I have learned about them. + +00:43:19 And so short answer, Docker is an approach to deployment, and there's nothing stopping anybody from writing a plugin that has a Docker-based approach. + +00:43:28 this kind of gets back to the question of like what's next yeah I'm going to finish the plugin for a proof of concept plugin for DigitalOcean I'll take it to the 1.0 version of that approach so people can reliably deploy to DigitalOcean with a particular articulated approach it'll probably be catty choice of SQLite or Progres Whereas, and then I'm probably like, my best approach is probably to back off of writing more plugins because I could spend the next few years just enjoyably exploring all these platforms and approaches. + +00:44:09 But I really should back off, just continue to let people know the project exists, make sure these set of plugins that I've started continue to work, and then just continue to tighten up the internals so that it's easier and easier for other people to write plugins. + +00:44:24 I can already tell from the way you're talking, like you could, I have a DSD plugin template repository. + +00:44:31 It's in the Django Simple Deploy organization. + +00:44:34 And so if you want to write a plugin, you don't start from scratch. + +00:44:37 You download that repository for the DSD plugin template. + +00:44:43 You don't clone it because there's no upstream for it. + +00:44:46 Download it, install it alongside Simple Deploy in development mode, and you can run tests right away. + +00:44:52 So you run an integration test and it just verifies it. + +00:44:55 It gives you a couple of questions. + +00:44:56 What's the name of your platform? + +00:44:57 What's the command you want people to use? + +00:45:00 Do you support automate all? + +00:45:02 And then it writes a new plugin for you with all the plumbing there. + +00:45:08 You run tests, verify that it works, and then you start writing your platform-specific code. + +00:45:13 And so there's a bit of a learning curve for developers where you got to learn, like, what... + +00:45:17 You know, Django's node deploy has a module called plugin utils. + +00:45:22 And so that's how you write output in a way that's consistent with the rest of the project, feeds into its logging. + +00:45:27 There's a run quick command for small changes you're making to the project. + +00:45:32 There's a run slow command for things like deploy to make sure we're capturing output, streaming it, whatnot. + +00:45:37 So you learn those utilities, but you pretty quickly get to a place where you're just focusing on your expertise or opinion about how to approach deployment for a particular platform. + +00:45:47 Yeah. Okay. I'm sure there's a ton out there. + +00:45:51 People can be like, well, what about my platform, right? + +00:45:53 You know, I'm sure there's some folks who are experts at wherever they're deploying their code that could maybe come and create a plugin, right? + +00:46:00 Absolutely. + +00:46:00 And somebody tried to write in support for Google. + +00:46:08 What was it? + +00:46:09 Google Cloud? + +00:46:10 Yeah, Google. + +00:46:11 Yeah, GCP. + +00:46:12 Google Cloud. + +00:46:13 A couple years ago. + +00:46:15 But that was back when there was no plugin system. + +00:46:17 And so it got bogged down in just like all the back and forth between core and that platform. + +00:46:24 These days, that person could just download that plugin template and start focusing on the platform. + +00:46:30 Yeah, awesome. + +00:46:31 I'm sure that there's people out there thinking, hey, we could do this for Azure. + +00:46:34 Hey, we could do this for AWS as well, right? + +00:46:36 You know, I'm really blown away about how many people choose the hyperscale clouds, given how complicated and expensive they are. + +00:46:45 There was some stat in one of the recent surveys. + +00:46:48 I don't know if it was the ESF chip brains one or just a more general developer one. + +00:46:54 Anyway, it doesn't matter. + +00:46:54 One of the bigger surveys, 75, 78% of the people used either Azure, AWS, or GCP. + +00:47:02 And I look at those, I'm like, geez, I don't think 75% of the apps out there need the stuff that those are offering. + +00:47:09 And it's so, so much more expensive. + +00:47:11 I don't think people fully understand how much more expensive it is to get like a VM on one versus the other. + +00:47:18 I did some, I think, you know, it always depends on which one you pick or something. + +00:47:23 But for a $20 Hetzner server, it's, I think, $60 on DigitalOcean, $200 something dollars on AWS, and I think pushing up towards $300 for Azure. + +00:47:36 And what you get when you go up to these prices, you don't get a better experience. + +00:47:40 you get a worse one because all the complexity, all the stuff that you got to deal with, it's not like, well, just here's your server, go with it. + +00:47:46 Like, oh, no, no. + +00:47:46 There's all these other things you got to bring in. + +00:47:48 And it just, it super surprises me that that's the case. + +00:47:52 And so, yeah, I don't know. + +00:47:54 Yeah. + +00:47:54 I built working support for Azure early on. + +00:47:58 My goal, you know, I worked for Vork originally, then I made it work for one other platform, which was Fly. + +00:48:03 Like, okay, cool. + +00:48:04 This isn't tied to one platform. + +00:48:06 And then the rule of three, I did a platform in a stage. + +00:48:07 Like, okay, now I've gotten all the generalities out. + +00:48:11 moving over to VPS as opposed to platform as a service opens up a few new things, but I can put those in internally without making breaking changes in the project. + +00:48:22 Yeah. And Azure does have a platform as a service offering, for example. I'm not, I think, I'm sure AWS does as well, although I haven't tried to try to use it. Yeah. So, I mean, they're analogies. I don't know. + +00:48:34 Well, I also focused on Heroku. + +00:48:37 One of the things for Heroku is it's one of the fastest deployments I've used. + +00:48:41 It kind of got a dialed. + +00:48:44 Azure was taking like 10 or 15 minutes, so it was hard to do development work against that. + +00:48:49 Kim posted this interesting question or comment. + +00:48:52 The various deployment mechanisms presumably all need credentials. + +00:48:55 Are these provided as NV variables? + +00:48:57 Many beginners perhaps don't know what has to do with these. + +00:49:00 This project may help. + +00:49:01 I agree completely. + +00:49:03 So this project is like I'm approaching VPS credentials around SSH as ENV variables. + +00:49:10 That's what I've used in the past. + +00:49:12 ENV variables being files that have basically key values that you don't put into Git, but you create on the platform, right? + +00:49:19 Something like that? + +00:49:20 Yeah. + +00:49:20 And so the quick start guide will say like export these two environment variables, you know, your IP address and your host password, maybe a username if you've already created a non-root user. + +00:49:31 But it goes through and it basically, if you don't give it a username, it tries the default Django user username and it goes through root and whatnot. + +00:49:41 It's a little humbling to make these plugins because it shows you exactly where my current knowledge is at. + +00:49:47 But it's also a starting place that works. + +00:49:49 And so if somebody looks at this and says like, hey, you really shouldn't be using the NB variables. + +00:49:52 Here's the current best way to deal with credentials. + +00:49:55 we can write that into the plugin and as kim is pointing out it steers all beginners all people from that point forward um into that better usage and it also comes back to that idea that this isn't just for beginners think about all the people out there who are using the inv variables because that's what they learned and maybe if there's something better and the latest version of the plugin steers them towards that as well i've often found there are things that are good for beginners that end up being really helpful for everybody else as well um and this project feels like one of those. + +00:50:24 That's very cool. Certainly making your life easier. Everybody wants that, right? Yeah. + +00:50:28 Automated a lot of things. Okay. Well, I mean, I'll throw one more thought out there. Like as you're working on the server, the virtual server side of things, I think there's a lot of opportunity to leverage Docker to say like, if you could build an image that is 90% the way there, you could just say, run this Docker image, run this Docker container based on this image. + +00:50:48 And what, it doesn't even matter what server it's on. I think there's, I think there's some interesting stuff you could do with docker here but + +00:50:53 oh absolutely yes i'll let you go down that path but i i certainly have enjoyed it um yeah so what's next what do you have to you this is by the way i don't know if we called it out that way this is kind of the 1.0 you mentioned it in passing but this is the 1.0 milestone sort of thing right + +00:51:11 yeah the 1.0 for this is really about like people should be able to trust that it's reliable from this point forward it does work works on all three OSs for the officially supported plugins. + +00:51:23 I'm going to try to keep a list of plugins that I'm aware of that are being worked on and sort out that whole, you know, how do you manage an ecosystem? But, you know, I've also thought of this as like a gift to the Django community. + +00:51:36 I wanted to release 1.0 on Christmas. I couldn't quite get there. + +00:51:41 You know, if you listen to Django chat, when the word deployment comes up, there's always this Corona. + +00:51:46 of oh um and i've been there for 20 years excited as well over he loves it yes um + +00:51:52 but so this problem has been in jango in the jango world for 20 years how do we deal with the fact that it's hard to deploy jango harder than php harder than some other things and i really think this project kind of answers that question or goes a long way towards entering it um and there's a there are people in the django community who have been kind of cheerleading this project for a while um because if they've seen it i've announced it in public i've kind of done the development in public and at django cons whatnot + +00:52:22 yeah so awesome i'm gonna i'm gonna step back from creating more plugins i'm gonna continue to refine the internals and i'm gonna go back to my own like actual django projects you're + +00:52:33 gonna use it rather than build it yes yes you know that's the way to do it you know Well, I think one of the bits of magic of Python and why it's so popular is it's very welcoming to beginners, right? + +00:52:45 People coming from science, like we talked about, or other things, they kind of get sucked in and they're like, huh, it's nice here. + +00:52:50 Maybe I'll stay. + +00:52:51 And, you know, these kind of things sort of tell that same story for web developers, right? + +00:52:57 There might be a point where people outgrow this or they don't, but they might. + +00:53:01 But it might have gotten them that year's worth of experience running a server or running a platform as a service app and dealing with all of that so that they're then ready to take whatever next step that needs to be taken, if it needs to. + +00:53:13 Yeah, it really smooths out that learning process because you just described it. + +00:53:18 Python's welcoming people to learn Python. + +00:53:20 They say, oh, I want a web front end for this. + +00:53:23 Going back to my father not releasing his projects, he was a software developer in the 70s and 80s. + +00:53:30 And so for him, deployment was, I need to make my project so good, it can be printed to like 10,000 physical copies. + +00:53:39 So it's no fault to him for having those unreleased projects. + +00:53:44 So the pathway these days is people learn Python, say, okay, I'm solving my problem with Python. + +00:53:48 I want a web front end so other people can access it. + +00:53:51 They learn Django because they already know Python. + +00:53:53 That's all smooth. + +00:53:55 Django is a bit hard, but you can learn it if you know Python. + +00:53:58 And then they get to deployment and it's this entirely different class of things to learn. + +00:54:03 And that's where that cliff comes in. + +00:54:05 And so simple deploy tends to smooth out that. + +00:54:09 It basically removes that cliff. + +00:54:11 And so you go right from where you are. + +00:54:13 I have a project working on my system. + +00:54:15 Now it's working on a remote server. + +00:54:17 I don't fully understand it, but it's much easier to poke around at the working deployment than poke around at something that you're trying to do that's broken and people end up giving up. + +00:54:28 I'm excited about like, yeah, I'm excited about like opening that pathway for people. + +00:54:32 Yeah, absolutely. + +00:54:33 That's awesome. + +00:54:33 Nice work. + +00:54:34 And thanks for doing it. + +00:54:35 There's already so much to learn to build a web app to even go from, I know Python to a web app because CSS databases, you know, HTML, JavaScript, there's already enough that you don't need to throw Linux and CDNs and all that at people at the same time, right? + +00:54:52 Let them go across the chasm and they'll figure it out on the other side. + +00:54:56 All right. + +00:54:57 Well, thanks for this project. + +00:54:58 Maybe final call to action. + +00:55:00 People want to get started with it. + +00:55:00 What do you tell them? + +00:55:02 Go take a look at the docs. + +00:55:03 There's friendly quick start docs, show you how it's used. + +00:55:06 And there's pretty comprehensive contributing docs if you're curious to write a plugin or how about otherwise? + +00:55:11 Yeah, I guess let's close it out with this. + +00:55:13 If you got a platform out there, if you're somebody like fly.io, but a different, consider creating one of these plugins. + +00:55:20 It blows my mind, Eric. + +00:55:22 I don't know how you feel, but there'll be some major, major service. + +00:55:25 And you'll go there and they're like, well, you can use our JavaScript API or you can just write it from scratch. + +00:55:31 Like how much effort is it for one of these companies to spend a week, create a package for their thing and make it go? + +00:55:37 And I think, you know, a sort of a similar call to action, like, hey, if you want to make it real easy for people to just switch to your platform, you know, consider following the steps that you talked about to build a plugin for this, right? + +00:55:48 Yeah. + +00:55:49 Yeah. + +00:55:49 And for anybody getting, anybody interested in writing a plugin, feel free to reach out. + +00:55:52 I'm happy to make sure that works first time people do it. + +00:55:55 Yeah, absolutely. + +00:55:56 All right. + +00:55:57 Thanks for being here again. + +00:55:58 Talk to you later. + +00:55:58 Thank you, Michael. + +00:55:59 Bye. + +00:56:00 Bye. + +00:56:01 This has been another episode of Talk Python to Me. + +00:56:05 Thank you to our sponsors. + +00:56:06 Be sure to check out what they're offering. + +00:56:07 It really helps support the show. + +00:56:10 This episode is sponsored by Worth Recruiting. + +00:56:13 Worth Recruiting specializes in placing senior level Python developers and data scientists. + +00:56:18 Let Worth help you find your next Python opportunity at talkpython.fm/worth. + +00:56:23 Want to level up your Python? + +00:56:25 We have one of the largest catalogs of Python video courses over at Talk Python. + +00:56:29 Our content ranges from true beginners to deeply advanced topics like memory and async. + +00:56:34 And best of all, there's not a subscription in sight. + +00:56:37 Check it out for yourself at training.talkpython.fm. + +00:56:40 Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +00:56:44 We should be right at the top. + +00:56:46 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +00:56:55 We're live streaming most of our recordings these days. + +00:56:58 If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +00:57:06 This is your host, Michael Kennedy. + +00:57:08 Thanks so much for listening. + +00:57:09 I really appreciate it. + +00:57:10 Now get out there and write some Python code. + diff --git a/transcripts/500-django-simple-deploy.vtt b/transcripts/500-django-simple-deploy.vtt new file mode 100644 index 0000000..a45ef3c --- /dev/null +++ b/transcripts/500-django-simple-deploy.vtt @@ -0,0 +1,2150 @@ +WEBVTT + +00:00:00.020 --> 00:00:05.400 +We're sitting down with Eric Matthews, the educator, author, and developer behind Django Simple Deploy. + +00:00:06.180 --> 00:00:16.900 +If you've ever struggled with taking the final step of getting your Django app onto a live server without spending days wrestling with DevOps complexities, then give Django Simple Deploy a look. + +00:00:17.200 --> 00:00:26.200 +Eric shares how Django Simple Deploy automates away the boilerplate parts of deployment so you can focus on building features instead of deciphering endless configs. + +00:00:26.780 --> 00:00:34.200 +We'll talk about the new project's journey to 1.0, the range of hosting platforms it supports, and why it's not just for beginners. + +00:00:34.880 --> 00:00:40.440 +This is Talk Python to Me, episode 500, recorded February 20th, 2025. + +00:00:42.320 --> 00:00:43.860 +Are you ready for your host, please? + +00:00:44.740 --> 00:00:47.680 +You're listening to Michael Kennedy on Talk Python to Me. + +00:00:48.220 --> 00:00:51.300 +Live from Portland, Oregon, and this segment was made with Python. + +00:00:54.560 --> 00:00:57.420 +Welcome to Talk Python to Me, a weekly podcast on Python. + +00:00:57.740 --> 00:02:00.080 +This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is sponsored by worth recruiting worth recruiting specializes in placing senior level python developers and data scientists let worth help you find your next python opportunity at talkpython.fm/worth eric welcome back to talk python great to have you here nice to see you again michael nice to see you as well last time we talked about your book right and now your project your deployment project i love talking about deployment because i think it's such an enabling technology or enabling skill and your project Django Simple Deploy is certainly in that realm. + +00:02:00.480 --> 00:02:03.100 +Yes, I have appreciated your comments about deployment over the years. + +00:02:04.100 --> 00:02:09.280 +Thanks. Well, they're a hard one. My skills in deployment are a hard one. + +00:02:09.800 --> 00:02:16.160 +I kind of want to talk a little bit about the journey for people, but not yet. + +00:02:16.520 --> 00:02:19.740 +Let's get a quick introduction of who is Eric. What do you do? + +00:02:20.200 --> 00:02:21.980 +What's new in your life since the last time you were on the show? + +00:02:22.440 --> 00:02:28.460 +Oh, gosh, yeah. It's all context for this project because this project ties together a lot of the different things that I've done. + +00:02:29.140 --> 00:02:33.080 +Short version, I grew up in New Hampshire, moved to New York City, then moved to Alaska. + +00:02:33.880 --> 00:02:36.200 +Just last year, I moved to Western North Carolina. + +00:02:38.019 --> 00:02:43.640 +And I had a background in science. I studied physics in college and got pulled into teaching. + +00:02:43.900 --> 00:03:15.280 +spent a long time teaching math and science, fifth grade through twelfth grade, which was super fun because it's not at the level of undergrad or grad work, but the questions young people ask always make things interesting. And I got pulled into programming because I always did it on the side. And I kind of felt like, I always felt like at some point, I might focus more on programming and less on classroom teaching. So for the past five, six years I've been a full-time writer and programmer. + +00:03:15.760 --> 00:03:24.420 +It's been enjoyable. Yeah. And congrats on the move. You really like to sample different weather systems, different environments. + +00:03:26.240 --> 00:03:32.360 +You're going to get a different summer experience in North Carolina than you might in Alaska. I know it gets warm somewhat there, but + +00:03:32.360 --> 00:03:33.220 +it's warm a + +00:03:33.220 --> 00:03:33.780 +lot in North Carolina. + +00:03:34.140 --> 00:03:37.180 +Yeah. We're in the mountains, so we can have cool nights at least. + +00:03:37.500 --> 00:03:41.680 +Yeah. Lovely. My brother lives there and I think North Carolina is a really neat place. + +00:03:41.940 --> 00:03:42.800 +It is, yes. + +00:03:43.060 --> 00:04:34.880 +Yeah, yeah, yeah. So let's talk about this going from science to teaching to programming just a little bit, because I did a very similar journey. I think mine was accelerated in terms of its time frame, but I was in grad school. I studied math and then taught math, calculus, linear algebra, those kinds of things at universities at GTA, and then kind of got into programming as part of it. And then I'm like, wow, this is really fun. How do I, how do I do more of this? You know what I mean? And after evaluating it, just academia didn't seem like a thing I really wanted to try to chase. And so I basically, I took one or two programming classes, but otherwise, other than that, I taught myself programming a number of years ago. That is kind of scary. Last century, let's say. And I don't know, I feel totally confident about it these days, but at first it was, you know, do I belong here? + +00:04:35.300 --> 00:04:35.759 +All these things + +00:04:35.780 --> 00:04:54.780 +seem hard. What am I doing wrong? Should I have gotten a CS degree? I don't think any of those things are true. I think just, you know, as I learned hanging out with CS people, it was hard for them too. Like it's, but maybe talk a little bit about your journey, right? Because I'm sure it would be helpful to other people. There's many people out there who are doing something somewhat similar. + +00:04:55.060 --> 00:05:26.140 +Yeah. Questions about academia kind of steered me this way as well. I studied physics in undergrad because I started as a chemical engineer because I had a great chemistry class in high school. But I found that engineering classes felt like being taught how to solve other people's problems. And my favorite class as an undergrad or as a freshman was physics because it was the only class that really seemed about truly trying to understand the world. And I love that. So that's what led me to focus on physics. And I was going to be a particle physicist, but I did not want to + +00:05:26.140 --> 00:05:26.400 +be... + +00:05:26.420 --> 00:05:27.500 +Wanted to spend some time at sun + +00:05:27.500 --> 00:05:29.740 +or maybe outside of + +00:05:29.740 --> 00:05:31.140 +Chicago or wherever. Yeah. Okay. + +00:05:31.580 --> 00:05:39.920 +Yeah. And I, yeah, when I was really young, I heard about atom smashers. I thought somebody like literally had hammers where they smash atoms. + +00:05:40.960 --> 00:05:42.440 +So every stage, + +00:05:42.920 --> 00:05:56.800 +yes, every stage I learned more about it was more and more fascinating, but I didn't want to be a student forever. And so I want to do something outside of being a student before pursuing the graduate work that would get to get me to be a part of a physicist. + +00:05:57.700 --> 00:06:03.260 +So when I was looking what to do, I had been a tutor in undergrad, and I tutored calculus and physics. + +00:06:03.780 --> 00:06:12.140 +And I tutored those classes because I watched classmates start to hate math and physics because of how they were taught. + +00:06:12.440 --> 00:06:13.980 +And that's the only reason people hated those topics. + +00:06:14.980 --> 00:06:22.460 +And it was really humbling because I was 18, 19 years old and tutoring people in their 30s and 40s who were doing mid-career changes. + +00:06:23.660 --> 00:06:28.520 +Speak of the field, like you maybe don't necessarily belong there telling someone how to do something. + +00:06:29.560 --> 00:06:32.240 +But at the same time, you probably knew it really well, right? + +00:06:32.960 --> 00:06:33.140 +Yeah. + +00:06:33.820 --> 00:06:33.880 +Yeah. + +00:06:34.070 --> 00:06:40.260 +And I felt like I belonged, but it was really humbling as far as just sorting out like school systems and where to go. + +00:06:40.680 --> 00:06:42.120 +So anyway, that led me into teaching. + +00:06:42.770 --> 00:06:45.900 +And I thought I would just teach for a couple of years and go back to science. + +00:06:46.400 --> 00:06:53.300 +But I found that the challenge of reaching everybody in the classroom was as hard and as satisfying as hard science. + +00:06:53.830 --> 00:06:54.800 +And so I just love teaching. + +00:06:55.800 --> 00:06:57.600 +I love what it brings up for other people. + +00:06:57.820 --> 00:06:59.540 +And, yeah, it's always stayed interesting. + +00:07:02.060 --> 00:07:06.960 +For me, the transition into more of a focus on programming was my father died. + +00:07:07.220 --> 00:07:10.920 +I've told this story a few times over the years about the origin of the book Python Crash Course. + +00:07:11.620 --> 00:07:13.160 +But my father died in 2011. + +00:07:13.550 --> 00:07:14.320 +And my mom asked. + +00:07:14.370 --> 00:07:15.000 +He was a programmer. + +00:07:15.880 --> 00:07:16.800 +That's how I first learned to program. + +00:07:17.180 --> 00:07:23.440 +My mom asked me to go through his computer and she found all these, or she asked me to go through his computer to see if there was anything worth keeping. + +00:07:24.780 --> 00:07:29.840 +And there wasn't, but for me, it was a really personal experience of seeing all these projects he had started. + +00:07:30.220 --> 00:07:31.140 +They would never get finished. + +00:07:31.620 --> 00:07:34.320 +And I realized that I had a bunch of those programming projects as well. + +00:07:35.300 --> 00:07:47.080 +And so that kind of was a turning point for me to get me out of the practice of just building a bunch of side projects and start to focus on one or two real world projects that other people would benefit from. + +00:07:47.360 --> 00:07:48.040 +That's a great story. + +00:07:48.520 --> 00:08:00.500 +That's certainly one of the challenges of you can build a project and you can get it to run, but you're maybe halfway there, getting it out to the world in a real world sort of way. + +00:08:00.820 --> 00:08:05.020 +You've got all these challenges of, I need to pick a place to run my app. + +00:08:05.200 --> 00:08:05.940 +Wait a minute, databases? + +00:08:06.660 --> 00:08:08.340 +Why are these migrations so hard? + +00:08:08.340 --> 00:08:08.620 +How do I + +00:08:08.620 --> 00:08:09.260 +do migrations? + +00:08:10.060 --> 00:08:12.180 +I got the first one up and now I can't change it. + +00:08:12.320 --> 00:08:25.560 +just breaks it doesn't run anymore and you know there's just all these these aspects not even taking into account the stuff you got to add to go from a toy project to making it real logging error handling etc etc right yeah + +00:08:25.560 --> 00:09:13.040 +i mean this is all a good segue into the topic of today i mean django simple deployed to kind of put it out there for people who haven't heard of it um it's a um package that automates deployment uh django deployments so if your project runs locally on your computer and you get to the point where you want to make it public for other people to use if you haven't already gone through the deployment process for most people that's a cliff and they kind of just fall off i spend all this time learning django all the time building this private they really care about solves the problem does it better than other people's work because you know it's new and whatnot and then they go to deploy it and they find that they have to spend so much time reading a platform's documentation um tweaking their settings um adding this file file and then kind of crossing your fingers and hoping that it works. + +00:09:14.140 --> 00:09:19.860 +Every time I've gone through that process, I've kind of thought to myself, this is all just boilerplate. + +00:09:20.140 --> 00:09:25.100 +Like we're just being told to, everybody's being told to add the same settings, make the same modifications. + +00:09:25.880 --> 00:09:27.460 +And so I had that shower + +00:09:27.460 --> 00:09:28.640 +thought of like, automation. + +00:09:29.160 --> 00:09:29.320 +Yeah. + +00:09:29.780 --> 00:09:29.880 +Yeah. + +00:09:30.280 --> 00:09:35.260 +I had that shower thought of like, okay, how could we do this configuration from the Django side? + +00:09:35.560 --> 00:09:35.660 +So + +00:09:35.660 --> 00:09:37.040 +when we're just looking + +00:09:37.040 --> 00:09:41.480 +at the user's computer, you've got a project that's working We know it works. + +00:09:42.680 --> 00:09:50.380 +How can we do all the configuration automatically so that you then just push your automatically, correctly configured project to your hosting platform? + +00:09:51.160 --> 00:09:52.460 +So Jango Simple Deploy does that. + +00:09:52.640 --> 00:09:54.320 +You pip install on Jango Simple Deploy. + +00:09:54.900 --> 00:09:57.720 +You need to have a plugin for the platform that you're using. + +00:09:57.940 --> 00:10:00.240 +So Fly.io is a go-to example. + +00:10:01.860 --> 00:10:04.200 +And then you add Jango Simple Deploy to installed apps. + +00:10:04.670 --> 00:10:07.900 +And then there's two modes for this project to work. + +00:10:08.020 --> 00:10:14.620 +The fully automated mode is easier to talk about because you run one command, bifimanage.py deploy --automate all. + +00:10:14.890 --> 00:10:27.900 +And if you do that and you have that platform CLI installed, it configures your project, makes a git commit, and pushes your project, and your project appears in a new browser window, which feels like magic. + +00:10:28.860 --> 00:10:30.120 +That's kind of magical, actually. + +00:10:30.860 --> 00:10:31.740 +Yeah, yeah. + +00:10:32.180 --> 00:10:32.440 +All right. + +00:10:32.920 --> 00:10:36.580 +So I want to talk about what platforms are supported. + +00:10:36.960 --> 00:10:46.700 +But before we do, maybe we could talk more broadly about just hosting platforms that you've considered, that you see out there, that people publish Django apps to, right? + +00:10:47.040 --> 00:10:49.680 +On one hand, we've got really, really straightforward platforms. + +00:10:50.220 --> 00:10:53.620 +On the other, you can get as complicated as you want, basically. + +00:10:53.820 --> 00:11:03.460 +You play the game of how many AWS services could you combine into one Django app, which, you know, that's kind of like the obfuscated code contest. + +00:11:04.680 --> 00:11:09.040 +How much can you fit under one line that's unlegible but still does the thing or whatever, right? + +00:11:09.960 --> 00:11:12.720 +You know, we started this with a conversation about our backgrounds. + +00:11:13.270 --> 00:11:28.400 +And so kind of weaving this for a little bit back into that background story, I do remember one of my transitions to doing more professional programming work was when I was teaching, I would write programs to address inefficiencies and difficulties we had in our school for our staff. + +00:11:28.880 --> 00:11:36.780 +And so I remember like exactly this, getting something that like automated report card generation for our staff. + +00:11:36.860 --> 00:11:37.980 +And I got it working on my computer. + +00:11:38.020 --> 00:11:41.700 +So like for me, writing report cards was so much easier than for everybody else. + +00:11:42.200 --> 00:11:43.120 +And so I learned Django. + +00:11:43.240 --> 00:11:45.260 +I'm like, cool, I can share this with everybody else. + +00:11:45.820 --> 00:11:48.580 +And I tried to deploy it and it just, I could not do it. + +00:11:49.080 --> 00:11:50.460 +It just fell + +00:11:50.460 --> 00:11:50.700 +apart. + +00:11:51.880 --> 00:11:53.740 +And I sort of figured it out. + +00:11:54.080 --> 00:12:01.480 +So my first deployment platform was a laptop in a closet in my school connected directly to the school network. + +00:12:02.060 --> 00:12:08.500 +So the deployment worked for people on our network, and it was only out of the kindness of our IT staff that would allow me to do that. + +00:12:09.120 --> 00:12:11.240 +Clearly not a solution for most people. + +00:12:12.720 --> 00:12:16.800 +It's really hard to deploy. Let me just deploy my machine to the school. + +00:12:17.020 --> 00:12:17.300 +Yes, + +00:12:17.500 --> 00:12:18.720 +yes. Update. + +00:12:18.920 --> 00:12:19.120 +Let me + +00:12:19.120 --> 00:12:20.040 +go to that closet. + +00:12:21.760 --> 00:12:22.660 +Yeah, yeah, yeah. + +00:12:23.340 --> 00:12:24.480 +I'm going to be in here for a few + +00:12:24.480 --> 00:12:26.000 +hours. I'm programming on the server. + +00:12:26.480 --> 00:12:26.920 +Yeah. + +00:12:27.750 --> 00:12:42.940 +So, okay, Heroku is the first one to talk about because Heroku was one of the first platforms that really automated a lot of the DevOps work and allowed us the classic sales pitch for Heroku was run and get push Heroku main and your project is live. + +00:12:43.920 --> 00:12:49.140 +There's always a bit more to it than that, including that configuration to make your project work on Heroku. + +00:12:49.920 --> 00:12:55.380 +So the platforms I start thinking about are the ones that manage the DevOps aspect for you. + +00:12:56.100 --> 00:13:03.340 +Heroku, Flyto.io, Platform.sh, Python Anywhere is in there. + +00:13:03.840 --> 00:13:09.480 +So to take a step back even further, this project started because I used Heroku for a while. + +00:13:10.070 --> 00:13:17.060 +In Python Crash Course, the first edition, I taught people, I walked people through deploying a Django project to Heroku. + +00:13:17.520 --> 00:13:20.480 +is the only part of the book that felt like I'm not really teaching anything. + +00:13:20.980 --> 00:13:23.300 +I'm just telling people to write certain things and add certain + +00:13:23.300 --> 00:13:23.660 +files. + +00:13:24.180 --> 00:13:30.120 +A bit of explanation about what it does, but mostly it's, you know, make these changes and cross your fingers that you type them all right. + +00:13:31.779 --> 00:13:40.400 +So I wrote a Heroku build pack that did the configuration automatically, but it did it on Heroku servers. + +00:13:41.140 --> 00:13:42.460 +So you'd specify this build pack. + +00:13:42.770 --> 00:13:47.440 +It would load that build pack onto Heroku servers, and then all your configuration changes would happen on their end. + +00:13:48.000 --> 00:13:49.360 +Tell us what is a build pack. + +00:13:50.600 --> 00:13:51.640 +I'm not sure that I can tell you. + +00:13:52.020 --> 00:13:54.700 +It's basically, I said my short + +00:13:54.700 --> 00:13:54.760 +take is... + +00:13:54.760 --> 00:13:58.580 +I'll tell you, I've never built a build pack myself, and I've deployed a lot of Python, so... + +00:13:58.620 --> 00:13:58.880 +Yeah, + +00:14:00.260 --> 00:14:10.460 +my short description is it's a set of programs, body of code that configures your project for running on that specific platform. + +00:14:10.920 --> 00:14:11.360 +Got it, okay. + +00:14:11.820 --> 00:14:17.800 +And so typically it's generic and it's not targeting, it's not trying to do any changes to your project. + +00:14:19.079 --> 00:14:19.960 +We won't go further. + +00:14:20.340 --> 00:14:21.000 +I'll say something + +00:14:21.000 --> 00:14:21.200 +wrong. + +00:14:21.460 --> 00:14:22.140 +Yeah, yeah, yeah, sounds good. + +00:14:22.480 --> 00:14:25.440 +But the takeaway for that is it felt like doing Heroku's work for them. + +00:14:26.000 --> 00:14:30.980 +And so it was nice for anybody trying to push Django to Heroku, but that boundary of like, what's our work? + +00:14:31.140 --> 00:14:31.740 +What's their work? + +00:14:31.960 --> 00:14:33.520 +Are we doing work free for them? + +00:14:34.120 --> 00:14:34.900 +That's an interesting question. + +00:14:35.320 --> 00:14:38.440 +And it really felt like it was skewed towards doing their work. + +00:14:38.900 --> 00:14:39.040 +Right. + +00:14:39.280 --> 00:14:47.560 +Well, I've done it once or twice, deployed an app to Heroku, never for real work, but to, you know, play around with it and sort of check it out. + +00:14:47.720 --> 00:14:51.320 +Because Heroku was definitely a darling of + +00:14:51.320 --> 00:14:52.100 +the + +00:14:52.100 --> 00:14:56.640 +Python space and certainly of the Python tutorials, right? + +00:14:56.860 --> 00:14:59.520 +Because for a long time, they had a free tier. + +00:14:59.940 --> 00:15:00.820 +And so everybody + +00:15:00.820 --> 00:15:08.140 +who needed to deploy something was like, here's how you put it on Heroku to sort of complete the loop and get it out there in the world, right? + +00:15:09.120 --> 00:15:28.600 +Right. And then, yeah, you know, a testament to Heroku is how long it has worked without a lot of attention from them. And so they lost a lot of trust in the community, not just from doing away with their free tier. It's really hard to maintain a free tier. It's just puts a big target on the platform for abuse. + +00:15:29.120 --> 00:15:37.080 +But it's about some reliability issues and communication issues around issues on the platform. + +00:15:37.920 --> 00:15:40.000 +So there was an exodus from Heroku for a while. + +00:15:40.320 --> 00:15:43.920 +And I was working on this automation problem for a while. + +00:15:44.880 --> 00:15:52.200 +And I had that shower thought of like, okay, all that work that I'm doing through the Buildpack on Heroku's end could be done through a management command with NJNGO. + +00:15:52.520 --> 00:15:57.880 +You can do all the changes to your project on your computer if you know what to do. + +00:15:58.260 --> 00:16:03.860 +So internally, Django simple deploy is just a single management command, that deploy command. + +00:16:04.350 --> 00:16:09.880 +And when you run that command, based on the plugin that you have installed, it knows which platform you're targeting. + +00:16:10.270 --> 00:16:15.680 +And then basically, I've just spent hours pouring over the documentation for a few different platforms. + +00:16:16.500 --> 00:16:23.260 +And so we pick up the platform that you're trying to deploy to, make all the changes necessary for a basic initial deployment to that platform. + +00:16:24.160 --> 00:16:26.480 +And then there's two ways you can push your project. + +00:16:26.880 --> 00:16:35.420 +The fully automated mode puts all those changes into a single commit, runs that platform's deploy command, and pops up your project in a new window. + +00:16:35.820 --> 00:16:41.420 +Yeah, and the transport mechanism is Git, right? + +00:16:41.840 --> 00:16:42.780 +Good question. + +00:16:43.220 --> 00:16:44.040 +Not necessarily. + +00:16:44.880 --> 00:16:48.720 +The transport mechanism is whatever that platform uses. + +00:16:49.400 --> 00:16:55.620 +So for Fly, Django simple deploy makes the configuration changes for your project. + +00:16:56.060 --> 00:16:58.600 +and then it calls the fly deploy command. + +00:16:59.110 --> 00:17:03.380 +So I don't even have to know what fly deploy is using behind the scenes. + +00:17:04.100 --> 00:17:08.920 +You have the CLI installed, simple deploy calls that deploy command. + +00:17:09.680 --> 00:17:16.800 +For Heroku, when it gets to that deployment stage, in the automated mode, Jingo simple deploy runs git push Heroku main. + +00:17:17.089 --> 00:17:27.240 +And so it's one of the nice things about this project, and part of the reason it has taken so long to get to 1.0 is quite adaptable to different approaches. + +00:17:27.740 --> 00:17:32.100 +Yeah, that's pretty wild because you're just automating the CLI for that platform, right? + +00:17:32.420 --> 00:17:32.820 +Right. + +00:17:34.030 --> 00:17:39.660 +And Chris is commenting, there should be one obvious way to deploy your Django project to a service. + +00:17:40.360 --> 00:17:41.260 +Yeah, he's right. + +00:17:41.390 --> 00:17:48.640 +The nice thing about this is Django Simple Deploy becomes this abstraction layer between your work and the platform. + +00:17:49.260 --> 00:17:53.400 +So it creates a nice, consistent interface for deploying to a wide range of platforms. + +00:17:54.500 --> 00:18:10.660 +And when a platform inevitably changes their deployment process or configuration requirements, as long as one person makes that update to the Django SEP deploy plugin, everybody else gets to just keep using that consistent interface for how you do your initial deployment, + +00:18:10.890 --> 00:18:11.580 +which is really, + +00:18:11.780 --> 00:18:12.160 +really nice. + +00:18:12.640 --> 00:18:13.860 +Yeah, it is really nice. + +00:18:14.400 --> 00:18:14.660 +You don't + +00:18:14.660 --> 00:18:20.900 +have to worry about all the details and also the ability to move without losing all of your expertise. + +00:18:23.660 --> 00:18:26.860 +This portion of Talk Python to Me is brought to you by Worth Recruiting. + +00:18:27.420 --> 00:18:29.940 +Are you tired of applying for jobs and never hearing back? + +00:18:30.600 --> 00:18:39.100 +Have you been getting the runaround or having trouble making it past the AI resume screeners that act as the new gatekeepers for your next level Python job? + +00:18:39.380 --> 00:18:41.100 +You should reach out to Worth Recruiting. + +00:18:41.760 --> 00:18:46.080 +Worth Recruiting specializes in placing senior level Python developers and data scientists. + +00:18:46.820 --> 00:18:52.820 +They work directly with hiring managers at startups, helping them grow their software engineering and data science teams. + +00:18:53.660 --> 00:18:56.100 +With Worth, it's not just connecting you with the company. + +00:18:56.660 --> 00:19:01.460 +It will guide you through the interview process and help make sure you're ready with their detailed preparation approach. + +00:19:01.840 --> 00:19:06.560 +They can even coach you on salary negotiations and other important decision-making processes. + +00:19:07.360 --> 00:19:12.180 +So if you're ready to see what new opportunities are out there for you, reach out to Worth Recruiting. + +00:19:12.600 --> 00:19:18.320 +Let them be your partner and specialist to find the right Python developer or data scientist position for you. + +00:19:18.780 --> 00:19:23.160 +Fill out their short contact form at talkpython.fm/worth. + +00:19:23.580 --> 00:19:24.360 +It only takes a minute. + +00:19:24.940 --> 00:19:27.360 +That's talkpython.fm/worth. + +00:19:27.800 --> 00:19:29.760 +The link is in your podcast player's show notes. + +00:19:30.280 --> 00:19:32.260 +Thank you to Worth Recruiting for supporting the show. + +00:19:33.820 --> 00:19:41.600 +This is where it gets really fun to talk about this project because, you know, people think when people first hear about this project, they think it's just for beginners. + +00:19:42.320 --> 00:19:44.120 +It's just for people who already know deployment. + +00:19:46.120 --> 00:19:47.660 +But where is I going with that? + +00:19:48.640 --> 00:19:55.660 +Oh, so thinking about writers, authors, myself first, but content creators, tutorial writers, presenters. + +00:19:56.540 --> 00:20:12.940 +If you're wanting to include deployment as part of your process that you're teaching people, then a lot of people have chosen not to do that because as soon as you include deployment, you're now tied to that platform's stability. + +00:20:13.520 --> 00:20:16.920 +As soon as they change their process, you've got to update your materials or it's broken. + +00:20:17.000 --> 00:20:18.920 +and that's not appealing. + +00:20:19.480 --> 00:20:31.720 +So for somebody who uses Django Simply Deploy in their tutorial, again, as long as one of us keeps the plugin updated to work on that platform, your learning resources stays the same. + +00:20:31.900 --> 00:20:32.720 +That is really nice. + +00:20:33.020 --> 00:20:33.300 +And whether + +00:20:33.300 --> 00:20:33.500 +you're + +00:20:33.500 --> 00:20:38.020 +teaching a college class or you're just writing something for your blog, it is really nice. + +00:20:38.880 --> 00:20:40.520 +Yeah, and I think about the Django Girls workshops. + +00:20:41.080 --> 00:20:46.860 +And so part of that, the goal of that conclusion of deploying your project is just to see a live. + +00:20:47.020 --> 00:20:48.540 +Like, why are we doing this if we can't see a line? + +00:20:49.120 --> 00:20:52.440 +You don't necessarily want to spend a whole bunch of time focused on deployment. + +00:20:52.710 --> 00:20:56.120 +And so if you can kind of really crunch that into like, okay, we're going to run these commands. + +00:20:57.040 --> 00:20:58.000 +We have a working deployment. + +00:20:58.580 --> 00:21:00.700 +It's not about hiding deployment from people. + +00:21:01.160 --> 00:21:06.600 +Because one of the coolest parts of this project is that it contains all those configuration changes in a single Git commit. + +00:21:07.520 --> 00:21:11.080 +Is there a verbose mode where you get to show all the little steps it's doing and stuff? + +00:21:11.440 --> 00:21:14.860 +Yeah, it is verbose by default. + +00:21:15.160 --> 00:21:26.780 +So you get to basically it streams and dumps all the platform's output as it's deploying, which is kind of important because if you don't do that, you just see this blank screen for two, three, seven + +00:21:26.780 --> 00:21:27.000 +minutes. + +00:21:27.040 --> 00:21:28.160 +Yeah, it's just sitting there spinning. + +00:21:28.980 --> 00:21:29.260 +Yeah. + +00:21:31.040 --> 00:21:32.720 +But to slow this down, take a step back for a minute. + +00:21:33.100 --> 00:21:37.340 +I've mentioned the automated mode where you include this stash-tash automate all. + +00:21:37.820 --> 00:21:42.440 +That's easiest to talk about because it boils down this deployment process to three steps. + +00:21:43.180 --> 00:21:52.940 +I don't actually recommend that for most people because it makes a Git commit for you, runs a deploy command, and maybe you want to have a little more control about that process. + +00:21:53.860 --> 00:22:01.580 +There's a configuration-only mode, which is every bit as helpful, but it lets you kind of take a little more control. + +00:22:01.880 --> 00:22:10.520 +And so what you do for most platforms, you have to create a resource, basically a deployed project, an empty project on that platform, and then you run deploy. + +00:22:11.960 --> 00:22:20.480 +And Django Simple Deploy recognizes the resource you just created on that platform, configures your project, and then it stops. + +00:22:21.000 --> 00:22:27.180 +And so you now have a configured project, and you can run git diff, and you get to see all the configuration + +00:22:27.180 --> 00:22:28.000 +changes + +00:22:28.000 --> 00:22:29.100 +that were made on your behalf. + +00:22:30.180 --> 00:22:31.320 +So you don't have to just trust it. + +00:22:32.140 --> 00:22:32.880 +Yeah, yeah. + +00:22:33.420 --> 00:22:39.780 +So when I shared this on the Django Discord yesterday, somebody was excited about it, and then I said something about that commit that was made. + +00:22:40.160 --> 00:22:41.660 +And they said, oh, it makes commits for you. + +00:22:41.900 --> 00:22:43.160 +I'm not sure I want to use that. + +00:22:43.510 --> 00:22:43.860 +Is it okay? + +00:22:43.970 --> 00:22:47.640 +You can do the longer mode where you get to review that. + +00:22:48.740 --> 00:22:52.140 +And then you can see all the files that were created for that platform. + +00:22:52.640 --> 00:22:55.600 +You can see the settings changes that were made to make your project work. + +00:22:56.400 --> 00:22:58.560 +And then you can choose to make that commit. + +00:22:58.850 --> 00:23:00.940 +You can choose to push it or revert it. + +00:23:01.380 --> 00:23:05.220 +Even if you use the automated mode, it's still all the configuration is one commit. + +00:23:05.550 --> 00:23:08.600 +And so you can just revert that commit and you're right back to your project. + +00:23:09.120 --> 00:23:10.500 +I'll throw one more piece in there right away. + +00:23:10.960 --> 00:23:15.800 +one of the priorities of the project is that it does not change the local behavior of your project. + +00:23:16.180 --> 00:23:31.800 +And so when I run the test suite, it tests that all the configuration changes are correct for the target platform, but it also runs run server again locally and makes sure that we haven't impacted the local running of your project. + +00:23:32.260 --> 00:23:35.780 +Oh, interesting. It'll run it to make sure it basically starts. Nice. + +00:23:36.200 --> 00:23:44.300 +Yeah, you know, this automated versus config sort of deal, it reminds me a little bit of how maybe DBAs feel about ORMs. + +00:23:45.300 --> 00:23:50.160 +You know, like they're like, oh, you can't just let it run arbitrary queries against my database. + +00:23:50.600 --> 00:23:52.420 +Like, how do we know it's doing the right thing? + +00:23:52.540 --> 00:23:54.840 +I'm sure you feel that way for a while. + +00:23:55.660 --> 00:23:58.580 +But eventually, all right, we've reviewed them. + +00:23:58.920 --> 00:23:59.660 +They're fine, right? + +00:23:59.840 --> 00:24:02.640 +So maybe this config thing you do a couple of times, you're like, + +00:24:02.740 --> 00:24:03.260 +yeah, that's + +00:24:03.260 --> 00:24:03.980 +what I would do anyway. + +00:24:04.720 --> 00:24:05.540 +Just run it, right? + +00:24:05.940 --> 00:24:06.580 +Because what + +00:24:06.580 --> 00:24:07.100 +else are you going to do? + +00:24:07.240 --> 00:24:13.600 +You're probably going to create a script that does the things you expect it to do anyway for yourself as a bash script or something, right? + +00:24:13.660 --> 00:24:15.200 +And so might as well be a Python script. + +00:24:15.560 --> 00:24:15.840 +Yeah. + +00:24:15.980 --> 00:24:20.440 +I mean, it's funny hearing you talk about this because this project started as a pile of bash scripts. + +00:24:20.900 --> 00:24:21.340 +And so, + +00:24:21.540 --> 00:24:22.560 +yeah. + +00:24:22.760 --> 00:24:24.380 +I got a pile of bash scripts on my server. + +00:24:24.460 --> 00:24:24.940 +They do things. + +00:24:25.580 --> 00:24:25.660 +Yeah. + +00:24:25.900 --> 00:24:32.820 +One of the pre-1.0 tasks was a long issue that was just about converting all these shell scripts to Python. + +00:24:33.560 --> 00:24:36.660 +And mostly that was about creating, making this work cross-platform. + +00:24:37.140 --> 00:24:39.980 +So it's basically taken me like four years to get this to 1.0. + +00:24:40.290 --> 00:24:45.120 +And it's that 80-20 principle turned into like 595 principle. + +00:24:49.380 --> 00:24:58.220 +So another reason, so a good thing to kind of move into, another reason it took so long to get to 1.0 was originally this was all one library. + +00:24:58.640 --> 00:24:59.960 +So it was just Django simple deploy. + +00:25:00.440 --> 00:25:05.100 +And originally you would run managed.py deploy --platform by.io. + +00:25:06.020 --> 00:25:10.660 +And so the entire project contained all the code for all the different platforms. + +00:25:12.300 --> 00:25:17.100 +And that felt better than the original project just targeting Heroku because you have choice. + +00:25:17.500 --> 00:25:19.180 +But it felt like a nightmare for maintenance. + +00:25:19.880 --> 00:25:26.680 +Anytime any project changes, it's going to need an update and it's just going to grow exponentially. + +00:25:27.300 --> 00:25:29.900 +Yeah, it becomes combinatorially complex in + +00:25:29.900 --> 00:25:31.280 +terms of dependencies, which + +00:25:31.280 --> 00:25:33.940 +is not a good order of magnitude. + +00:25:35.480 --> 00:25:36.220 +It's not good. + +00:25:36.840 --> 00:25:37.060 +Yeah. + +00:25:37.550 --> 00:25:45.840 +So I had some people kind of very kindheartedly nudging me towards just doing a 1.0 release to get over that fear of a 1.0 release. + +00:25:46.760 --> 00:25:54.220 +But it was always about having – I had a vision that the mature version of this project should be plug-in based. + +00:25:54.820 --> 00:25:57.820 +So the core project, Django Simple Deploy, is small. + +00:25:58.620 --> 00:25:59.520 +It's platform agnostic. + +00:25:59.960 --> 00:26:05.500 +All it does is inspect your project, inspect your system, and then hand out the rest of the work to a plugin. + +00:26:05.810 --> 00:26:11.460 +And so currently there's a plugin targeting apply.io, plugin for platform.sh, platform for Heroku. + +00:26:12.220 --> 00:26:18.920 +I have a proof of concept plugin for CodeRed, and I'm working on a proof of concept for DigitalOcean. + +00:26:20.860 --> 00:26:23.000 +Yeah, so many cool things to talk about with this. + +00:26:23.540 --> 00:26:23.620 +Yeah, + +00:26:23.880 --> 00:26:23.980 +yeah. + +00:26:24.200 --> 00:26:32.380 +Before we go, I do want to maybe just go through some of these hosts and let you talk about maybe why they're included. + +00:26:32.840 --> 00:26:35.840 +Which ones are, there's stuff out there, obviously, that's not included. + +00:26:35.920 --> 00:26:36.760 +Why is it not included? + +00:26:37.220 --> 00:26:42.360 +So Heroku, obvious, especially when you started, when they had the free tier and stuff like that. + +00:26:43.220 --> 00:26:45.480 +I kind of want to get them, I know people are angry at them. + +00:26:45.780 --> 00:26:46.440 +I understand that. + +00:26:46.500 --> 00:26:59.860 +But also, I kind of want to give them a little bit of a pass, like offering free hosting to the world where people just put up malware and other badness and use the trust of your domain to sort of leverage that, it's challenging. + +00:27:00.200 --> 00:27:01.220 +And I don't know. + +00:27:01.890 --> 00:27:04.480 +I understand people are frustrated with them, but that's tough. + +00:27:05.340 --> 00:27:12.200 +So Heroku, platform as a service, you push your stuff up, you say, here's my code, here's how you start it, run it for me, please. + +00:27:12.660 --> 00:27:13.520 +There's not a lot of Linux. + +00:27:13.960 --> 00:27:15.820 +There's not a lot of shell, right? + +00:27:15.920 --> 00:27:17.220 +They kind of take care of that for you, yes? + +00:27:17.360 --> 00:27:17.800 +They do. + +00:27:18.040 --> 00:27:19.240 +And there's still, yeah. + +00:27:20.220 --> 00:27:27.140 +I've had conversations with people who, with founders of a number of these platforms over the years, because it's always interesting to talk about this. + +00:27:28.130 --> 00:27:31.380 +And platforms would love to see an easier way to deploy Jango projects. + +00:27:32.740 --> 00:27:38.900 +One of those founders said something really insightful at one point, and I appreciated the honesty, is that we're all trying to be second best for beginners. + +00:27:39.880 --> 00:27:45.200 +Because if you draw that crowd, you're drawing more support than benefit in the short term. + +00:27:45.950 --> 00:27:49.100 +And so, yeah, for a long time, Foroku paid that cost. + +00:27:49.580 --> 00:27:51.440 +they were willing to be the go-to for everybody. + +00:27:51.960 --> 00:27:54.800 +So there's still a lot of projects running long-term on Heroku. + +00:27:55.120 --> 00:27:57.200 +I know some important Python ones that are there. + +00:27:57.600 --> 00:27:57.700 +Yeah. + +00:27:57.890 --> 00:27:58.040 +Yeah. + +00:27:58.560 --> 00:27:58.680 +Okay. + +00:27:58.900 --> 00:28:02.680 +And then we got fly.io and platform.sh. + +00:28:03.260 --> 00:28:04.360 +What's the story of these? + +00:28:05.300 --> 00:28:06.480 +Honestly, I haven't used either of them. + +00:28:06.720 --> 00:28:07.360 +I've heard of them. + +00:28:07.360 --> 00:28:07.900 +I haven't used them. + +00:28:08.120 --> 00:28:14.280 +So the selling point for fly.io is they're an edge-based platform. + +00:28:14.840 --> 00:28:15.040 +And so + +00:28:15.040 --> 00:28:17.220 +they take your, + +00:28:17.620 --> 00:28:24.920 +if I'm saying this right, They take your project and they push your application code to a bunch of different nodes. + +00:28:25.430 --> 00:28:26.180 +And so your code + +00:28:26.180 --> 00:28:26.300 +is... + +00:28:26.320 --> 00:28:27.400 +Kind of like a CDN, right? + +00:28:28.260 --> 00:28:30.180 +Yeah, CDN for your application code. + +00:28:30.430 --> 00:28:30.760 +And so your + +00:28:30.760 --> 00:28:31.440 +code is right + +00:28:31.440 --> 00:28:32.780 +close to your end users. + +00:28:33.430 --> 00:28:38.700 +And so I don't need it, but they've built it well enough that I can just push a project there and it works. + +00:28:39.280 --> 00:28:43.140 +And I'm a big fan of, you know, if your project does what you need it to, is it a good question? + +00:28:43.240 --> 00:28:43.760 +Is it good enough? + +00:28:43.930 --> 00:28:46.000 +And so it's been good enough for me for a lot. + +00:28:46.320 --> 00:28:49.340 +And I know it's targeting some more complex use cases. + +00:28:50.000 --> 00:28:50.400 +I + +00:28:50.400 --> 00:28:51.100 +have so many questions. + +00:28:52.600 --> 00:28:53.280 +It's so interesting. + +00:28:53.740 --> 00:28:56.880 +Well, the first thing that comes to mind is what about databases, right? + +00:28:57.220 --> 00:29:05.240 +If your app is running, yeah, you've got to have some way to correlate and centralize and make the currency of the database and all that work, yeah. + +00:29:05.480 --> 00:29:08.780 +They've had some fantastic people working for them, and they have some fantastic blog posts. + +00:29:09.160 --> 00:29:15.200 +So they've done really interesting cutting-edge work around distributed SQLite. + +00:29:16.080 --> 00:29:20.700 +databases. And so, yeah, they are quite familiar with that question and they + +00:29:20.700 --> 00:29:22.020 +have answered + +00:29:22.020 --> 00:29:24.140 +it well and they are answering it well. + +00:29:24.320 --> 00:29:31.540 +If you can answer that question well, then you can get some stuff that flies, you know? I mean, I guess, hence the name. I didn't even do that on purpose. + +00:29:32.240 --> 00:29:45.700 +I've been using bunny.net, which is a really nice CDN platform, but just for static content, not for application code that talks to the database. + +00:29:46.020 --> 00:29:59.960 +And I understand how you would replicate the database, but the challenge is if you have chatty rights, all of a sudden you're pushing your database away from your app, which has its own sort of latency and stuff. + +00:30:00.100 --> 00:30:01.140 +But what an interesting idea. + +00:30:01.460 --> 00:30:03.620 +Okay, and Platform.sh? + +00:30:03.960 --> 00:30:12.340 +Platform.sh started as a PHP, I believe, started as a PHP-only platform, but then expanded to support other languages. + +00:30:12.900 --> 00:30:16.280 +And so they have been sponsors of DjangoCon for a long time. + +00:30:17.260 --> 00:30:20.040 +And I really enjoy working with their platform. + +00:30:21.260 --> 00:30:22.120 +Deployment is pretty straightforward. + +00:30:23.380 --> 00:30:32.080 +And then, you know, one of the things I look for in these platform as a service is, so once you push it, I'm very happy to have them managing the DevOps work. + +00:30:32.360 --> 00:30:35.860 +I also want to be able to like get into the server and run commands and see what's going on. + +00:30:36.160 --> 00:30:37.940 +And so they make it easy to do that. + +00:30:38.400 --> 00:30:46.240 +Yeah, if things are not starting or they're crashing or something, sometimes just go, just 30 seconds at a glance at the log file or something. + +00:30:46.480 --> 00:30:49.360 +Oh, yeah, or maybe why is that file? + +00:30:49.550 --> 00:30:50.440 +That file is actually missing. + +00:30:50.690 --> 00:30:51.240 +Oh, that's weird. + +00:30:51.360 --> 00:30:52.320 +Let me see if I can just get that. + +00:30:52.320 --> 00:30:55.420 +And then you're back to like, okay, back to platform as a service, and you're good, you know? + +00:30:55.720 --> 00:31:02.260 +Yeah, and that's, you know, platform as a service certainly works for people who know nothing about, you know, hopping into a server and running commands. + +00:31:03.200 --> 00:31:05.680 +But they're also for, like, people who know how to do that and don't want to do that. + +00:31:05.980 --> 00:31:21.100 +And so a service like platform.sh that makes it easy to do both is also, I see it as an onboarding for people who have something worth deploying, don't know DevOps yet, but letting them hop in and start to do that is pretty interesting. + +00:31:21.440 --> 00:31:22.360 +Yeah, it's very interesting. + +00:31:22.760 --> 00:31:32.500 +Okay, so I can run Python manage.py deploy --automate all, and that will take my app and put it into whatever destination you pick, right, of those three. + +00:31:32.640 --> 00:31:36.140 +For these platforms as a service, a lot of times you have to have a database. + +00:31:36.750 --> 00:31:43.700 +Sometimes that's a managed Postgres or whatever you might have data that has to import initially. + +00:31:44.070 --> 00:31:48.560 +You know, you're like, well, here's all the counties in the United States and we have to query them for stuff. + +00:31:48.610 --> 00:31:50.000 +So we preload that at first. + +00:31:50.140 --> 00:31:53.040 +How does all that get connected and happen, right? + +00:31:53.200 --> 00:31:58.740 +If I had like no database and I just have a pure Django app that just runs Python, easy. + +00:31:58.800 --> 00:32:03.900 +I see how this works, but there's settings and stuff here that vary by production versus dev. + +00:32:04.220 --> 00:32:05.200 +Yeah, fun question. + +00:32:06.440 --> 00:32:10.180 +So I've done most of the development work on Django Simple Deploy. + +00:32:10.700 --> 00:32:13.840 +I've actually run sprints at several PyCons and several Django Cons. + +00:32:14.540 --> 00:32:18.540 +And this project has brought together like 8 to 15 people at each sprint. + +00:32:19.220 --> 00:32:19.980 +It draws some interest. + +00:32:20.480 --> 00:32:23.720 +And so the project would not be where it is without those contributions. + +00:32:24.340 --> 00:32:33.660 +A lot of those people have made some contributions and not done a whole lot outside of the conferences, but that whole group of people has been available to just bounce ideas off of. + +00:32:34.250 --> 00:32:44.040 +And so a lot of the pre-1.0 work was also like me getting rid of all the baggage that just kind of only made sense to me, or you really need to have years of contact with the project to make sense of it. + +00:32:44.700 --> 00:32:48.780 +And so now it's at a place where the structure is pretty consistent. + +00:32:49.180 --> 00:32:57.560 +And so one of the things I'm looking forward to is somebody else who knows Fly.io managing the Fly.io plugin more than me. + +00:32:57.840 --> 00:33:01.120 +And so they don't need to go deal with Chango Simple Deploy. + +00:33:01.620 --> 00:33:05.420 +They can focus on the parts of the script that do the things you were just talking about. + +00:33:06.080 --> 00:33:09.160 +So all the configuration has been largely my choice. + +00:33:09.660 --> 00:33:16.160 +You know, okay, on Heroku, we'll choose this Postgres instance because it's cheaper, because it lets people get started. + +00:33:17.340 --> 00:33:28.080 +It's possible for somebody to either take the Heroku plugin and add a few more CLI flags to give you a bit more choice, or just write your own Heroku plugin. + +00:33:28.350 --> 00:33:33.240 +And so right now the plugin for Heroku is dsd-heroku. + +00:33:33.640 --> 00:33:42.260 +Somebody else could do like DSD Heroku-large, and it's meant for scaling large or deploying large-scale Heroku projects. + +00:33:43.620 --> 00:33:45.580 +And so that's not a, I don't want to do that work. + +00:33:45.850 --> 00:33:46.680 +They should write their own. + +00:33:46.880 --> 00:33:50.420 +That's if you have a use case that you know well, you want to automate. + +00:33:51.480 --> 00:34:00.600 +Simple Deploy just becomes this, again, abstraction layer for, you know, you have a consistent interface for running that deployment from within Django. + +00:34:00.900 --> 00:34:04.480 +Yeah, there's a nice plugin model, which really opens things up. + +00:34:04.670 --> 00:34:14.280 +I suppose the short answer is probably, well, the Django ORM will create the database structure for you if you could just tell it where the database is, right? + +00:34:14.580 --> 00:34:14.700 +Right. + +00:34:15.200 --> 00:34:24.460 +And I know certain apps I've built, if they have data that has to be preloaded, I'll just have a little check like, hey, has this table got any data in it? + +00:34:24.590 --> 00:34:27.540 +If not, run the load up the database the first time. + +00:34:27.980 --> 00:34:29.940 +Otherwise, assume it's all good and keep going. + +00:34:30.639 --> 00:34:31.679 +that pretty much handles it? + +00:34:31.980 --> 00:34:34.899 +I think, you know, it's kind of two directions I'll take this. + +00:34:35.379 --> 00:34:40.300 +You know, a short list of the projects I've done over the years, they tend to be smaller, but they tend to be impactful. + +00:34:40.820 --> 00:34:44.899 +So one project, automated report cards for students. + +00:34:45.460 --> 00:34:52.179 +I used to work with struggling students, students who were plenty smart, but life had happened and they had gotten off track with school. + +00:34:52.530 --> 00:34:57.760 +And so they just had holes in their, in their graduation requirements. + +00:34:58.440 --> 00:35:02.280 +And so those students wanted to graduate, but they didn't know where to focus their efforts. + +00:35:02.670 --> 00:35:07.540 +And so looking at a transcript was really hard because it's just a bunch of letters and numbers. + +00:35:08.660 --> 00:35:13.020 +So I wrote a program that automated a generation of visual transcripts. + +00:35:13.280 --> 00:35:17.060 +So I'd read their text-based transcript and then give them basically a chart, a bunch of gauges. + +00:35:17.540 --> 00:35:21.320 +You just look at it in 30 seconds, know which classes you needed to focus on. + +00:35:22.880 --> 00:35:25.260 +And it also went from deficiencies to strengths. + +00:35:25.800 --> 00:35:29.760 +So traditionally, we talked to students who are struggling and saying, you need to do this, you need to do that. + +00:35:30.400 --> 00:35:34.400 +You could see where they needed to do their work, but you could also see where they had been shining. + +00:35:34.690 --> 00:35:35.640 +You could see where the strengths were. + +00:35:36.500 --> 00:35:38.180 +And so I wrote a project. + +00:35:38.270 --> 00:35:39.240 +I worked with whale researchers. + +00:35:39.859 --> 00:35:41.220 +And people don't know this. + +00:35:41.860 --> 00:35:46.500 +Sperm whales have learned to pick the fish off of – I'll take a step back. + +00:35:47.060 --> 00:35:57.600 +People catch halibut, fish for halibut by putting literally miles long cables on the ocean floor or lines with a hook every six to 10 feet. + +00:35:57.900 --> 00:36:01.380 +And so you put a thousand hooks on the ocean floor, leave it overnight or pull it. + +00:36:01.620 --> 00:36:02.720 +And that's how you catch halibut. + +00:36:04.380 --> 00:36:12.860 +And sperm whales have learned to listen for the sound of the transmission changes when somebody puts their boat into gear to pull those hooks. + +00:36:13.400 --> 00:36:16.120 +They swim towards that sound and they just pick the fish off those lines. + +00:36:16.620 --> 00:36:16.860 +Wow. + +00:36:17.200 --> 00:36:26.160 +And so these researchers were starting to put transponders on boats that would allow anybody who saw a sperm whale to report that. + +00:36:26.280 --> 00:36:30.340 +And then you could kind of time your fishing sets to be away from the sperm whales. + +00:36:31.140 --> 00:36:35.840 +And so I wrote a project that just like these researchers didn't know how to get their data from one place to another. + +00:36:36.140 --> 00:36:43.260 +And so I just wrote a small Django project that listened for that data and gave them a visible place where they could go to it and share it. + +00:36:45.460 --> 00:36:53.420 +And one other, oh, I lived in Southeast Alaska, and we were having increasing landslides, more frequent landslides, and more impactful. + +00:36:54.520 --> 00:37:02.280 +And so I wrote a small project that examines real-time stream gauge data and turns that into a measure of landslide risk. + +00:37:03.480 --> 00:37:05.600 +And they gave a talk about that last year at PyCon. + +00:37:06.390 --> 00:37:07.640 +All those projects are small. + +00:37:07.760 --> 00:37:09.500 +They're never going to scale to millions of users. + +00:37:10.040 --> 00:37:10.780 +They all need a database. + +00:37:11.390 --> 00:37:14.900 +And so they all fit this model of something like I personally would build. + +00:37:15.200 --> 00:37:18.360 +And if I didn't know deployment, those would not have ended up being useful. + +00:37:18.940 --> 00:37:24.160 +And so this tool takes all those people out there right now who are getting the skills to build something that works locally. + +00:37:24.350 --> 00:37:29.900 +And they don't have to go through that three weeks to three months of learning how to deploy things. + +00:37:30.840 --> 00:37:35.600 +So I appreciate you bringing up the questions of what happens with these larger scale projects. + +00:37:36.130 --> 00:37:42.460 +But there's also a whole class before you even get there of projects that would make the world better if they could be easily deployed. + +00:37:43.540 --> 00:37:50.460 +Such a challenge because if you go down the, there's so many paths that people get led down that are not good paths. + +00:37:50.810 --> 00:37:52.540 +You know, it's sort of the architectural equivalent. + +00:37:52.630 --> 00:37:58.480 +I guess even just the DevOps equivalent of you're not Microsoft, you're not Google, you're not Netflix. + +00:37:59.120 --> 00:38:02.480 +You probably don't need the chaos monkey running around your whale research project. + +00:38:03.080 --> 00:38:03.680 +Right, right. + +00:38:04.000 --> 00:38:11.400 +But people read that and they're like, well, I saw a really awesome conference talk and people loved it about how they had like geo-replication of their database and stuff. + +00:38:11.400 --> 00:38:11.660 +You're like, + +00:38:11.740 --> 00:38:12.060 +yes. + +00:38:12.780 --> 00:38:13.020 +Interesting. + +00:38:13.480 --> 00:38:13.900 +not for you. + +00:38:14.120 --> 00:38:19.820 +So the other direction that's interesting to finish out this discussion, so I'm targeting a VPS right now. + +00:38:20.140 --> 00:38:24.800 +I have deployed a Django project to DigitalOcean just on the bare VPS. + +00:38:25.620 --> 00:38:32.060 +For people less familiar with that virtual private server, it's basically a brand new virtual Linux machine that you can push your project to. + +00:38:33.640 --> 00:38:53.020 +And so this is more complicated because rather than just like making a few configuration changes to your project and then running a platform's deploy command, you have to do that whole update the server, install Django, set up a Git push if you want to do that approach, make the right port changes and whatnot, expose that. + +00:38:54.480 --> 00:38:57.440 +So it's more work for me to get that proof of concept done. + +00:38:57.880 --> 00:39:05.040 +But once it's done, the work I'm doing to write a plugin for DigitalOcean would work just as well for Hetzner, just as well for Linode. + +00:39:05.940 --> 00:39:10.440 +And then it opens up to all the different approaches that people take to deployment. + +00:39:11.120 --> 00:39:14.080 +One of the interesting questions about Django Simple Deploy is what are the boundaries? + +00:39:14.490 --> 00:39:15.260 +So what is simple? + +00:39:16.540 --> 00:39:19.500 +And I don't know the answer to that yet. + +00:39:19.730 --> 00:39:21.120 +It shouldn't become tube loaded. + +00:39:21.470 --> 00:39:25.700 +It shouldn't re-implement the entire CLI for a platform. + +00:39:26.940 --> 00:39:28.960 +But the bounds are pretty large. + +00:39:29.480 --> 00:39:36.280 +So you could have a DigitalOcean that targets NGINX Benicorn. + +00:39:36.620 --> 00:39:39.640 +You could have one that targets using the Doku approach. + +00:39:40.180 --> 00:39:42.900 +You could have one that targets external databases. + +00:39:43.700 --> 00:39:49.940 +There's really, it's a pretty open platform for automating deployment processes. + +00:39:51.120 --> 00:39:51.920 +Yeah, absolutely. + +00:39:52.220 --> 00:39:52.520 +Lots of people. + +00:39:53.080 --> 00:40:02.400 +I mean, once you knock out one of those virtual servers, it's really, really close to seeing, well, anything that can run a Linux VM pretty much can do it. + +00:40:02.640 --> 00:40:03.680 +And you were talking about Nginx. + +00:40:03.920 --> 00:40:06.240 +You know, you might, I don't know, have you considered Caddy? + +00:40:06.600 --> 00:40:07.260 +CaddyServer.com? + +00:40:07.860 --> 00:40:11.180 +As I was saying Nginx, I was like, oh, wait, there's another one that I should be saying. + +00:40:11.600 --> 00:40:17.980 +Well, I use Nginx for all of my stuff because it's gotten over the years, gotten more and more complicated. + +00:40:18.620 --> 00:40:26.960 +However, you've got to run, you've got to independently manage Let's Encrypt, and there's this sort of scaling up process. + +00:40:27.260 --> 00:40:34.060 +First, you've got to put the non-encrypted version up, so that cert bot and all the Let's Encrypt stuff can find it. + +00:40:34.060 --> 00:40:36.760 +Then you create the SSL certificate. + +00:40:36.840 --> 00:40:45.600 +Then you change the configuration again, because it won't run saying it's SSL until it actually has this, like there's this scaffolding step that's always messy. + +00:40:45.740 --> 00:40:56.800 +And if I think like easy getting started for people, Caddy, which manages the security and all that and has a real simple configuration language, it seems like it might be a good option. + +00:40:56.970 --> 00:40:59.220 +And it has 62,000 GitHub stars. + +00:40:59.440 --> 00:41:02.360 +It's, I think a person or two uses it, you know? + +00:41:02.540 --> 00:41:03.080 +That's kind of neat. + +00:41:03.300 --> 00:41:11.900 +Yeah, last week, before I started the DigitalOcean plugin, I asked on Mastodon how people are deploying to VPS these days. + +00:41:13.200 --> 00:41:14.620 +And Cady was one of the things that came up. + +00:41:14.670 --> 00:41:17.540 +It was super interesting to see the range of approaches that people are using. + +00:41:18.580 --> 00:41:27.580 +So, you know, as I go through this, I'm kind of asking the question for basic initial deployment for somebody new to this platform, what are some sane choices? + +00:41:28.640 --> 00:41:31.980 +So Cady or Nginx, Postgres or SQLite. + +00:41:32.470 --> 00:41:33.380 +So I'll make those choices. + +00:41:34.120 --> 00:41:40.100 +And one of the open questions that I don't have clear answers to is, you know, where are the boundaries around those plugins? + +00:41:41.260 --> 00:41:47.600 +So I'll certainly implement a --DB flag that lets you choose like Postgres or SQLite. + +00:41:48.370 --> 00:41:52.740 +But I don't want to do a flag for like --approach, daku, whatnot. + +00:41:53.220 --> 00:42:00.020 +At some point, it makes more sense to have other people write other plugins and just foster that plugin ecosystem. + +00:42:00.220 --> 00:42:00.740 +so + +00:42:00.740 --> 00:43:02.620 +another consideration and kim out in the audience was hinting at this before and it's certainly the way that i'm doing things these days is docker so let me just give you an example like yeah you can go all in and you could run on like the doc like eks and all these kinds of things you can go way way deep on it or you can just do stuff as simple as like hey i want i want the front end web server to be caddy i could install caddy on the server using apt or whatever but then you those kind of update more slowly, you have less control over it. You could just as equally well run a catty Docker image on that machine and have basically a zero touch experience on the server itself, other than just having a Docker thing there, right? And you can do whatever you want to that Docker image. You could even have a catty Django simple deployed one based off their original one or, you know, things like that where you've got it exactly configured. What's the story with Docker? And how have you been thinking about that? Because once you start to go to virtual or you start to get into some of these processes and some of these ideas? + +00:43:03.000 --> 00:43:04.340 +Yeah, great question. + +00:43:04.980 --> 00:43:06.280 +And it certainly has come up multiple times. + +00:43:06.880 --> 00:43:13.940 +My main experience with Docker is working with Fly because Fly is a Docker-based approach. + +00:43:15.080 --> 00:43:18.740 +I have not written my own Docker files, but I have learned about them. + +00:43:19.540 --> 00:43:27.420 +And so short answer, Docker is an approach to deployment, and there's nothing stopping anybody from writing a plugin that has a Docker-based approach. + +00:43:28.480 --> 00:43:32.160 +this kind of gets back to the question of like what's next + +00:43:33.060 --> 00:43:33.360 +yeah + +00:43:34.500 --> 00:44:08.500 +I'm going to finish the plugin for a proof of concept plugin for DigitalOcean I'll take it to the 1.0 version of that approach so people can reliably deploy to DigitalOcean with a particular articulated approach it'll probably be catty choice of SQLite or Progres Whereas, and then I'm probably like, my best approach is probably to back off of writing more plugins because I could spend the next few years just enjoyably exploring all these platforms and approaches. + +00:44:09.540 --> 00:44:24.140 +But I really should back off, just continue to let people know the project exists, make sure these set of plugins that I've started continue to work, and then just continue to tighten up the internals so that it's easier and easier for other people to write plugins. + +00:44:24.640 --> 00:44:31.160 +I can already tell from the way you're talking, like you could, I have a DSD plugin template repository. + +00:44:31.640 --> 00:44:33.840 +It's in the Django Simple Deploy organization. + +00:44:34.480 --> 00:44:37.480 +And so if you want to write a plugin, you don't start from scratch. + +00:44:37.840 --> 00:44:43.040 +You download that repository for the DSD plugin template. + +00:44:43.340 --> 00:44:45.680 +You don't clone it because there's no upstream for it. + +00:44:46.020 --> 00:44:52.460 +Download it, install it alongside Simple Deploy in development mode, and you can run tests right away. + +00:44:52.940 --> 00:44:55.140 +So you run an integration test and it just verifies it. + +00:44:55.620 --> 00:44:56.460 +It gives you a couple of questions. + +00:44:56.620 --> 00:44:57.340 +What's the name of your platform? + +00:44:57.680 --> 00:44:59.720 +What's the command you want people to use? + +00:45:00.860 --> 00:45:01.940 +Do you support automate all? + +00:45:02.500 --> 00:45:07.880 +And then it writes a new plugin for you with all the plumbing there. + +00:45:08.280 --> 00:45:13.120 +You run tests, verify that it works, and then you start writing your platform-specific code. + +00:45:13.410 --> 00:45:17.240 +And so there's a bit of a learning curve for developers where you got to learn, like, what... + +00:45:17.960 --> 00:45:21.520 +You know, Django's node deploy has a module called plugin utils. + +00:45:22.060 --> 00:45:26.420 +And so that's how you write output in a way that's consistent with the rest of the project, feeds into its logging. + +00:45:27.420 --> 00:45:31.580 +There's a run quick command for small changes you're making to the project. + +00:45:32.040 --> 00:45:37.320 +There's a run slow command for things like deploy to make sure we're capturing output, streaming it, whatnot. + +00:45:37.900 --> 00:45:47.360 +So you learn those utilities, but you pretty quickly get to a place where you're just focusing on your expertise or opinion about how to approach deployment for a particular platform. + +00:45:47.760 --> 00:45:50.960 +Yeah. Okay. I'm sure there's a ton out there. + +00:45:51.120 --> 00:45:53.680 +People can be like, well, what about my platform, right? + +00:45:53.840 --> 00:45:59.720 +You know, I'm sure there's some folks who are experts at wherever they're deploying their code that could maybe come and create a plugin, right? + +00:46:00.000 --> 00:46:00.260 +Absolutely. + +00:46:00.760 --> 00:46:08.140 +And somebody tried to write in support for Google. + +00:46:08.840 --> 00:46:09.240 +What was it? + +00:46:09.460 --> 00:46:09.900 +Google Cloud? + +00:46:10.260 --> 00:46:11.000 +Yeah, Google. + +00:46:11.360 --> 00:46:11.760 +Yeah, GCP. + +00:46:12.060 --> 00:46:12.520 +Google Cloud. + +00:46:13.160 --> 00:46:13.660 +A couple years + +00:46:13.660 --> 00:46:13.940 +ago. + +00:46:15.180 --> 00:46:17.000 +But that was back when there was no plugin system. + +00:46:17.480 --> 00:46:23.180 +And so it got bogged down in just like all the back and forth between core and that platform. + +00:46:24.020 --> 00:46:29.840 +These days, that person could just download that plugin template and start focusing on the platform. + +00:46:30.160 --> 00:46:30.640 +Yeah, awesome. + +00:46:31.040 --> 00:46:33.940 +I'm sure that there's people out there thinking, hey, we could do this for Azure. + +00:46:34.240 --> 00:46:35.080 +Hey, we could do this for + +00:46:35.080 --> 00:46:35.880 +AWS as + +00:46:35.880 --> 00:46:36.440 +well, right? + +00:46:36.720 --> 00:46:45.560 +You know, I'm really blown away about how many people choose the hyperscale clouds, given how complicated and expensive they are. + +00:46:45.860 --> 00:46:48.260 +There was some stat in one of the recent surveys. + +00:46:48.510 --> 00:46:53.620 +I don't know if it was the ESF chip brains one or just a more general developer one. + +00:46:54.100 --> 00:46:54.740 +Anyway, it doesn't matter. + +00:46:54.990 --> 00:47:02.140 +One of the bigger surveys, 75, 78% of the people used either Azure, AWS, or GCP. + +00:47:02.660 --> 00:47:09.000 +And I look at those, I'm like, geez, I don't think 75% of the apps out there need the stuff that those are offering. + +00:47:09.300 --> 00:47:11.800 +And it's so, so much more expensive. + +00:47:11.940 --> 00:47:18.000 +I don't think people fully understand how much more expensive it is to get like a VM on one versus the other. + +00:47:18.360 --> 00:47:23.180 +I did some, I think, you know, it always depends on which one you pick or something. + +00:47:23.440 --> 00:47:35.960 +But for a $20 Hetzner server, it's, I think, $60 on DigitalOcean, $200 something dollars on AWS, and I think pushing up towards $300 for Azure. + +00:47:36.580 --> 00:47:39.980 +And what you get when you go up to these prices, you don't get a better experience. + +00:47:40.140 --> 00:47:45.980 +you get a worse one because all the complexity, all the stuff that you got to deal with, it's not like, well, just here's your server, go with it. + +00:47:46.060 --> 00:47:46.540 +Like, oh, no, no. + +00:47:46.660 --> 00:47:48.300 +There's all these other things you got to bring in. + +00:47:48.680 --> 00:47:52.220 +And it just, it super surprises me that that's the case. + +00:47:52.330 --> 00:47:53.580 +And so, yeah, I don't know. + +00:47:54.020 --> 00:47:54.300 +Yeah. + +00:47:54.460 --> 00:47:57.940 +I built working support for Azure early on. + +00:47:58.280 --> 00:48:03.200 +My goal, you know, I worked for Vork originally, then I made it work for one other platform, which was Fly. + +00:48:03.520 --> 00:48:04.200 +Like, okay, cool. + +00:48:04.300 --> 00:48:05.340 +This isn't tied to one platform. + +00:48:06.020 --> 00:48:07.500 +And then the rule of three, I did a platform in a stage. + +00:48:07.690 --> 00:48:09.980 +Like, okay, now I've gotten all the generalities out. + +00:48:11.200 --> 00:48:21.260 +moving over to VPS as opposed to platform as a service opens up a few new things, but I can put those in internally without making breaking changes in the project. + +00:48:22.160 --> 00:48:33.600 +Yeah. And Azure does have a platform as a service offering, for example. I'm not, I think, I'm sure AWS does as well, although I haven't tried to try to use it. Yeah. So, I mean, they're analogies. I don't know. + +00:48:34.460 --> 00:48:37.180 +Well, I also focused on Heroku. + +00:48:37.200 --> 00:48:40.720 +One of the things for Heroku is it's one of the fastest deployments I've used. + +00:48:41.620 --> 00:48:42.720 +It kind of got a dialed. + +00:48:44.120 --> 00:48:48.360 +Azure was taking like 10 or 15 minutes, so it was hard to do development work against that. + +00:48:49.480 --> 00:48:51.780 +Kim posted this interesting question or comment. + +00:48:52.040 --> 00:48:54.640 +The various deployment mechanisms presumably all need credentials. + +00:48:55.260 --> 00:48:57.080 +Are these provided as NV variables? + +00:48:57.780 --> 00:49:00.080 +Many beginners perhaps don't know what has to do with these. + +00:49:00.540 --> 00:49:01.340 +This project may help. + +00:49:01.780 --> 00:49:02.480 +I agree completely. + +00:49:03.440 --> 00:49:10.040 +So this project is like I'm approaching VPS credentials around SSH as ENV variables. + +00:49:10.720 --> 00:49:11.680 +That's what I've used in the past. + +00:49:12.900 --> 00:49:19.700 +ENV variables being files that have basically key values that you don't put into Git, but you create on the platform, right? + +00:49:19.720 --> 00:49:20.240 +Something like that? + +00:49:20.460 --> 00:49:20.720 +Yeah. + +00:49:20.920 --> 00:49:30.820 +And so the quick start guide will say like export these two environment variables, you know, your IP address and your host password, maybe a username if you've already created a non-root user. + +00:49:31.540 --> 00:49:38.580 +But it goes through and it basically, if you don't give it a username, it tries the default Django user username and it goes through root and whatnot. + +00:49:41.140 --> 00:49:46.220 +It's a little humbling to make these plugins because it shows you exactly where my current knowledge is at. + +00:49:47.120 --> 00:49:48.620 +But it's also a starting place that works. + +00:49:49.120 --> 00:49:52.560 +And so if somebody looks at this and says like, hey, you really shouldn't be using the NB variables. + +00:49:52.940 --> 00:49:55.180 +Here's the current best way to deal with credentials. + +00:49:55.700 --> 00:50:23.860 +we can write that into the plugin and as kim is pointing out it steers all beginners all people from that point forward um into that better usage and it also comes back to that idea that this isn't just for beginners think about all the people out there who are using the inv variables because that's what they learned and maybe if there's something better and the latest version of the plugin steers them towards that as well i've often found there are things that are good for beginners that end up being really helpful for everybody else as well um and this project feels like one of those. + +00:50:24.280 --> 00:50:28.540 +That's very cool. Certainly making your life easier. Everybody wants that, right? Yeah. + +00:50:28.740 --> 00:50:48.260 +Automated a lot of things. Okay. Well, I mean, I'll throw one more thought out there. Like as you're working on the server, the virtual server side of things, I think there's a lot of opportunity to leverage Docker to say like, if you could build an image that is 90% the way there, you could just say, run this Docker image, run this Docker container based on this image. + +00:50:48.640 --> 00:50:53.720 +And what, it doesn't even matter what server it's on. I think there's, I think there's some interesting stuff you could do with docker here but + +00:50:53.720 --> 00:50:55.720 +oh absolutely yes i'll + +00:50:55.720 --> 00:51:11.440 +let you go down that path but i i certainly have enjoyed it um yeah so what's next what do you have to you this is by the way i don't know if we called it out that way this is kind of the 1.0 you mentioned it in passing but this is the 1.0 milestone sort of thing right + +00:51:11.440 --> 00:51:21.960 +yeah the 1.0 for this is really about like people should be able to trust that it's reliable from this point forward it does work works on all three OSs for the officially supported plugins. + +00:51:23.260 --> 00:51:35.820 +I'm going to try to keep a list of plugins that I'm aware of that are being worked on and sort out that whole, you know, how do you manage an ecosystem? But, you know, I've also thought of this as like a gift to the Django community. + +00:51:36.500 --> 00:51:39.760 +I wanted to release 1.0 on Christmas. I couldn't quite get there. + +00:51:41.340 --> 00:51:46.060 +You know, if you listen to Django chat, when the word deployment comes up, there's always this Corona. + +00:51:46.100 --> 00:51:46.440 +of oh + +00:51:46.440 --> 00:51:52.000 +um and i've been there for 20 years excited as well over he loves it yes um + +00:51:52.000 --> 00:52:22.560 +but so this problem has been in jango in the jango world for 20 years how do we deal with the fact that it's hard to deploy jango harder than php harder than some other things and i really think this project kind of answers that question or goes a long way towards entering it um and there's a there are people in the django community who have been kind of cheerleading this project for a while um because if they've seen it i've announced it in public i've kind of done the development in public and at django cons whatnot + +00:52:22.560 --> 00:52:25.340 +yeah so awesome i'm gonna + +00:52:25.340 --> 00:52:33.440 +i'm gonna step back from creating more plugins i'm gonna continue to refine the internals and i'm gonna go back to my own like actual django projects you're + +00:52:33.440 --> 00:52:44.940 +gonna use it rather than build it yes yes you know that's the way to do it you know Well, I think one of the bits of magic of Python and why it's so popular is it's very welcoming to beginners, right? + +00:52:45.740 --> 00:52:50.840 +People coming from science, like we talked about, or other things, they kind of get sucked in and they're like, huh, it's nice here. + +00:52:50.960 --> 00:52:51.340 +Maybe I'll stay. + +00:52:51.840 --> 00:52:56.840 +And, you know, these kind of things sort of tell that same story for web developers, right? + +00:52:57.160 --> 00:53:01.020 +There might be a point where people outgrow this or they don't, but they might. + +00:53:01.360 --> 00:53:13.280 +But it might have gotten them that year's worth of experience running a server or running a platform as a service app and dealing with all of that so that they're then ready to take whatever next step that needs to be taken, if it needs to. + +00:53:13.540 --> 00:53:18.200 +Yeah, it really smooths out that learning process because you just described it. + +00:53:18.740 --> 00:53:20.180 +Python's welcoming people to learn Python. + +00:53:20.880 --> 00:53:22.580 +They say, oh, I want a web front end for this. + +00:53:23.160 --> 00:53:29.860 +Going back to my father not releasing his projects, he was a software developer in the 70s and 80s. + +00:53:30.680 --> 00:53:38.300 +And so for him, deployment was, I need to make my project so good, it can be printed to like 10,000 physical copies. + +00:53:39.980 --> 00:53:43.380 +So it's no fault to him for having those unreleased projects. + +00:53:44.600 --> 00:53:48.680 +So the pathway these days is people learn Python, say, okay, I'm solving my problem with Python. + +00:53:48.960 --> 00:53:51.360 +I want a web front end so other people can access it. + +00:53:51.680 --> 00:53:53.340 +They learn Django because they already know Python. + +00:53:53.940 --> 00:53:54.620 +That's all smooth. + +00:53:55.140 --> 00:53:57.800 +Django is a bit hard, but you can learn it if you know Python. + +00:53:58.520 --> 00:54:03.000 +And then they get to deployment and it's this entirely different class of things to learn. + +00:54:03.440 --> 00:54:04.860 +And that's where that cliff comes in. + +00:54:05.220 --> 00:54:09.320 +And so simple deploy tends to smooth out that. + +00:54:09.540 --> 00:54:10.680 +It basically removes that cliff. + +00:54:11.030 --> 00:54:13.280 +And so you go right from where you are. + +00:54:13.660 --> 00:54:15.320 +I have a project working on my system. + +00:54:15.840 --> 00:54:17.320 +Now it's working on a remote server. + +00:54:17.760 --> 00:54:27.300 +I don't fully understand it, but it's much easier to poke around at the working deployment than poke around at something that you're trying to do that's broken and people end up giving up. + +00:54:28.080 --> 00:54:29.620 +I'm excited about like, yeah, + +00:54:30.060 --> 00:54:32.220 +I'm excited about like opening that pathway for people. + +00:54:32.500 --> 00:54:33.160 +Yeah, absolutely. + +00:54:33.400 --> 00:54:33.660 +That's awesome. + +00:54:33.960 --> 00:54:34.400 +Nice work. + +00:54:34.480 --> 00:54:35.340 +And thanks for doing it. + +00:54:35.400 --> 00:54:52.500 +There's already so much to learn to build a web app to even go from, I know Python to a web app because CSS databases, you know, HTML, JavaScript, there's already enough that you don't need to throw Linux and CDNs and all that at people at the same time, right? + +00:54:52.580 --> 00:54:56.200 +Let them go across the chasm and they'll figure it out on the other side. + +00:54:56.480 --> 00:54:56.620 +All right. + +00:54:57.040 --> 00:54:58.300 +Well, thanks for this project. + +00:54:58.920 --> 00:54:59.880 +Maybe final call to action. + +00:55:00.060 --> 00:55:00.860 +People want to get started with it. + +00:55:00.880 --> 00:55:01.300 +What do you tell them? + +00:55:02.020 --> 00:55:02.860 +Go take a look at the docs. + +00:55:03.160 --> 00:55:06.080 +There's friendly quick start docs, show you how it's used. + +00:55:06.240 --> 00:55:10.840 +And there's pretty comprehensive contributing docs if you're curious to write a plugin or how about otherwise? + +00:55:11.500 --> 00:55:13.460 +Yeah, I guess let's close it out with this. + +00:55:13.780 --> 00:55:20.200 +If you got a platform out there, if you're somebody like fly.io, but a different, consider creating one of these plugins. + +00:55:20.440 --> 00:55:21.900 +It blows my mind, Eric. + +00:55:22.080 --> 00:55:25.020 +I don't know how you feel, but there'll be some major, major service. + +00:55:25.640 --> 00:55:30.820 +And you'll go there and they're like, well, you can use our JavaScript API or you can just write it from scratch. + +00:55:31.520 --> 00:55:31.940 +Like how + +00:55:31.940 --> 00:55:37.760 +much effort is it for one of these companies to spend a week, create a package for their thing and make it go? + +00:55:37.780 --> 00:55:48.560 +And I think, you know, a sort of a similar call to action, like, hey, if you want to make it real easy for people to just switch to your platform, you know, consider following the steps that you talked about to build a plugin for this, right? + +00:55:48.940 --> 00:55:49.040 +Yeah. + +00:55:49.340 --> 00:55:49.400 +Yeah. + +00:55:49.500 --> 00:55:52.640 +And for anybody getting, anybody interested in writing a plugin, feel free to reach out. + +00:55:52.800 --> 00:55:55.660 +I'm happy to make sure that works first time people do it. + +00:55:55.940 --> 00:55:56.420 +Yeah, absolutely. + +00:55:56.760 --> 00:55:56.920 +All right. + +00:55:57.240 --> 00:55:58.100 +Thanks for being here again. + +00:55:58.400 --> 00:55:58.660 +Talk to you + +00:55:58.660 --> 00:55:58.780 +later. + +00:55:58.820 --> 00:55:59.120 +Thank you, Michael. + +00:55:59.680 --> 00:56:00.080 +Bye. + +00:56:00.640 --> 00:56:00.820 +Bye. + +00:56:01.600 --> 00:56:04.320 +This has been another episode of Talk Python to Me. + +00:56:05.060 --> 00:56:06.020 +Thank you to our sponsors. + +00:56:06.480 --> 00:56:07.720 +Be sure to check out what they're offering. + +00:56:07.880 --> 00:56:09.140 +It really helps support the show. + +00:56:10.100 --> 00:56:12.580 +This episode is sponsored by Worth Recruiting. + +00:56:13.180 --> 00:56:17.560 +Worth Recruiting specializes in placing senior level Python developers and data scientists. + +00:56:18.200 --> 00:56:22.960 +Let Worth help you find your next Python opportunity at talkpython.fm/worth. + +00:56:23.720 --> 00:56:24.600 +Want to level up your Python? + +00:56:25.060 --> 00:56:28.700 +We have one of the largest catalogs of Python video courses over at Talk Python. + +00:56:29.180 --> 00:56:33.860 +Our content ranges from true beginners to deeply advanced topics like memory and async. + +00:56:34.300 --> 00:56:36.480 +And best of all, there's not a subscription in sight. + +00:56:37.000 --> 00:56:39.400 +Check it out for yourself at training.talkpython.fm. + +00:56:40.120 --> 00:56:44.280 +Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +00:56:44.720 --> 00:56:45.600 +We should be right at the top. + +00:56:46.000 --> 00:56:54.960 +You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +00:56:55.640 --> 00:56:57.880 +We're live streaming most of our recordings these days. + +00:56:58.240 --> 00:57:05.720 +If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +00:57:06.740 --> 00:57:07.860 +This is your host, Michael Kennedy. + +00:57:08.280 --> 00:57:09.120 +Thanks so much for listening. + +00:57:09.280 --> 00:57:10.280 +I really appreciate it. + +00:57:10.620 --> 00:57:12.220 +Now get out there and write some Python code. + diff --git a/transcripts/501-marimo-reactive-notebooks-for-python.txt b/transcripts/501-marimo-reactive-notebooks-for-python.txt new file mode 100644 index 0000000..69c8958 --- /dev/null +++ b/transcripts/501-marimo-reactive-notebooks-for-python.txt @@ -0,0 +1,1532 @@ +00:00:00 Have you ever spent an afternoon wrestling with a Jupyter notebook, hoping that you ran all the cells in just the right order, only to realize your outputs were completely out of sync? + +00:00:08 Today's guest has a fresh take on solving that exact problem. + +00:00:12 Akshay Agrawal is here to introduce Marimo, a reactive Python notebook that ensures your code and outputs always stay in lockstep. + +00:00:20 And that's just the start. + +00:00:21 We'll also dig into Axie's background at Google Brain and Stanford, what it's like to work on cutting-edge AI, and how Marimo is uniting the best of data science exploration and real software engineering. + +00:00:33 This is Talk Python to Me, episode 501, recorded Thursday, February 27th, 2025. + +00:00:41 Are you ready for your host, please? + +00:00:43 You're listening to Michael Kennedy on Talk Python to Me. + +00:00:46 Live from Portland, Oregon, and this segment was made with Python. + +00:00:53 Welcome to Talk Python to Me, a weekly podcast on Python. + +00:00:56 This is your host, Michael Kennedy. + +00:00:58 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, both accounts over at fosstodon.org, and keep up with the show and listen to over nine years of episodes at talkpython.fm. + +00:01:12 If you want to be part of our live episodes, you can find the live streams over on YouTube. + +00:01:16 subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. + +00:01:23 This episode is sponsored by Worth Recruiting. + +00:01:26 Worth Recruiting specializes in placing senior level Python developers and data scientists. + +00:01:31 Let Worth help you find your next Python opportunity at talkpython.fm/Worth. + +00:01:36 Akshay, welcome to Talk Python to Me. Awesome to have you here. + +00:01:39 Thanks, Michael. Really happy to be here. + +00:01:41 Yeah, it's data science hour, isn't it? + +00:01:44 Yeah, I guess it is. + +00:01:46 In a sense, but also in a sense, kind of bringing data science a little closer to traditional, formal computer science or more. + +00:01:55 I don't know. + +00:01:55 I don't know what to think about computer science. + +00:01:57 Software engineering. + +00:01:58 Software engineering. + +00:01:59 Yeah, that's our goal. + +00:02:00 Blending data science with sort of the rigor reproducibility of software engineering. + +00:02:05 Yeah. + +00:02:05 And I think people are going to find it pretty interesting. + +00:02:08 It's definitely gained a lot of traction lately. + +00:02:10 So it's going to be really fun to dive into Marimo. + +00:02:13 Before we do, though, who are you? + +00:02:15 What are you doing here? Tell people about yourself. + +00:02:18 Sure. So I'm Akshay. I'm one of the co-founders and developers of Marimo. + +00:02:24 I started working on Marimo a few years ago, I guess, a little over two years ago, right after I finished a PhD at Stanford where I was working on machine learning research. + +00:02:35 And before that, I was at Google Brain where I worked on TensorFlow. + +00:02:39 And I guess for the past several years, I've been wearing different hats, It's either a computer systems hat where I build open source software and domain specific languages for people who do machine learning or I just do machine learning. + +00:02:51 And for the past few years, it's been the first hat. + +00:02:54 So working on the Marimo notebook, which is our attempt to blend the best parts of interactive computing with the reusability and reproducibility of regular Python software. + +00:03:05 Well, that's very cool. + +00:03:06 You've been right at the cutting edge of this whole machine learning thing, I guess, right? + +00:03:11 If you go back a handful of years, it's different today than it was five years ago. + +00:03:16 Oh, yeah. + +00:03:16 It's very different. + +00:03:17 I was at Google Brain 2017 to 2018. + +00:03:21 So I think we had Transformers, but we didn't really appreciate all that they could do. + +00:03:27 Yeah. + +00:03:29 But it was a really exciting time to be at Google Brain where it was pure research, just like one, that you could just work on anything that you wanted to work on. + +00:03:37 And I was an engineer there on a, actually on a small sub team inside of TensorFlow where we were working on TensorFlow too, which is sort of making TensorFlow more like PyTorch crudely. + +00:03:47 And that was a lot of fun, but yeah, got to go to like a bunch of talks by researchers and it was a special time for sure. + +00:03:54 Yeah, I bet it was. + +00:03:55 Well, you know, that's such an interesting background. + +00:03:58 I think maybe give people a sense of what is Google Brain and all that and maybe just Transformers, right? + +00:04:04 It's the foundation of so much surprising and unexpected tech these days. + +00:04:12 Yeah, yeah, I'm happy to. + +00:04:14 So Google Brain, what is Google Brain? + +00:04:16 So I guess technically Google Brain no longer exists as it did back then. + +00:04:20 Now it's just DeepMind with DeepMind and Google Brain merged. + +00:04:23 But back then Google Brain was, so there was, I guess, two researchers under the alphabet umbrella, Google Brain and DeepMind. + +00:04:31 And Google Brain was a place where it was really special. + +00:04:34 So there were folks working on all kinds of research projects from you, like using transformers for code generation, but also like systems research projects. + +00:04:42 Like I was there when the JAX machine learning library, which is an alternative to TensorFlow and PyTorch, was still being in the early days of development. + +00:04:51 And I was able to meet with like Roy Frostig, Matt Johnson. + +00:04:56 And that was really cool sort of coming up to speed on machine learning systems in that era. + +00:05:03 I was only there for a year because I decided I wanted to go back and do a PhD myself in machine learning research where I used actually TensorFlow and PyTorch to work on new sort of like machine learning modules. + +00:05:16 Did you feel bad when you used PyTorch? + +00:05:18 You're like, oh, I'm cheating. + +00:05:20 No, no. + +00:05:21 I mean, a little, I guess. + +00:05:23 But, you know, it's the thing like everyone else used PyTorch. + +00:05:27 And we had just started working on TensorFlow too when I joined, whereas PyTorch had already been mature. + +00:05:31 So it was the responsible thing to do. + +00:05:36 And Transformers, what are these Transformer things? + +00:05:39 They came out of Google, right? + +00:05:41 Yeah, yeah, they did. + +00:05:42 Attention is all you need. + +00:05:45 These Transformer things, I guess, are the backbone of all these large language models that we're seeing today. + +00:05:51 And they proved to have like an incredible capacity for modeling, I guess, in a generative way, all kinds of text. + +00:05:59 And in a way that, honestly, if I had to admit, surprised me. + +00:06:03 At the time in 27 to 18, we were doing good work in machine translation and stuff. + +00:06:08 But even back then, there was researchers who were working on, at Google Brain, code generation, writing a bunch of unit tests, stuff that people do regularly today, actually, with large language models. + +00:06:20 I put on my systems engineer hat back then, and I was a little skeptical, to be honest. + +00:06:24 And I've just totally proved wrong, and I'm happy to have been proved wrong. + +00:06:28 So it's been phenomenal seeing sort of that, the success of that particular model. + +00:06:34 And yeah, having quite a small part in it. + +00:06:37 Yeah, I was, I've been a long time skeptic of machine learning in the sense that for so long, it was, it's kind of like fusion. + +00:06:43 It's going to be amazing, but it's 10, 20, 30 years out and maybe, you know, and it was all almost like knowledge, knowledge-based type things. + +00:06:55 And just, I mean, it was interesting. + +00:06:57 it looked kind of relevant and possibly useful in some of the cases, but not like it is today. + +00:07:02 Like today, people who don't even care about technology are blown away by this stuff, and they use it all day long, and it's gone so much farther and so much faster than I expected it to. + +00:07:13 I'm all about it these days. + +00:07:15 I still write most of my code, but any time there's something tricky instead of going to Google and hunting through Stack Overflow, let's see what chat says about it or something like that. + +00:07:27 And then maybe we'll go to Google if we don't get a good answer, you know? + +00:07:29 Yeah. + +00:07:30 No, no, I'm the same way. + +00:07:31 I spent more time than I care to admit last night migrating from Vim to NeoVim. + +00:07:37 And I know I'm late to the party, but perplexity and other tools just made it a lot easier to start from scratch installation without bringing over my VimRC. + +00:07:49 And you can ask it detailed questions, right? + +00:07:51 For example, I needed some way to reset the initial offset in DaVinci Resolve timelines. + +00:07:56 It was, okay, well, here's your four options, and these are the way you can do it, and these are the tradeoffs. + +00:08:00 It's not just you might get an, you know, here is the switch, but you get an analysis. + +00:08:05 And I'm pretty impressive. + +00:08:08 So the reason I'm going into this is I want to ask you, having been on the inside at the early days, what is your perspective of seeing this trajectory or trajectory is the way I should point it. + +00:08:20 Upwards, not logarithmic. + +00:08:22 Yeah. + +00:08:23 I guess what I will say is I think one thing that's like really changed, like, so back when I was at Brain, even the models were advanced, like, I felt like in order to use machine learning well, you really did need to be somewhat trained in machine learning. + +00:08:39 You know, it was important to understand what a loss function, like basic things, like what a + +00:08:42 loss function was, what's training, what's validation. + +00:08:46 These days, like, you don't need to, like, at all, actually. + +00:08:49 I think especially to use LLMs, right? + +00:08:52 Like, you just need to know inputs and outputs. + +00:08:54 It's like a function call that sort of has fuzzy outputs. + +00:08:57 You know, it's a mathematical function in quotes. + +00:09:00 Yeah. + +00:09:01 So I think that's one thing that's totally... + +00:09:04 I guess it kind of shows that artificial intelligence is transitioning more from, I guess, like a science to a technology, and that tool that people can just readily use without + +00:09:13 really needing to understand how it works. + +00:09:18 As for the trajectory, I've been bad at making predictions in the past. + +00:09:22 I think I will be bad at predicting anything right now. + +00:09:26 Surely, these models are very good for coding. + +00:09:31 for massaging marketing copy and like you know i do like the random tasks where like i have a bio i wrote in for first person and someone asked for it in third person and i'm too lazy to change and i just give it to a chat model um yeah but if you ask for a trajectory of agi or asi which i don't even remember what the s stands for these days but i i'm not sure i can predict anything on those timelines. + +00:09:56 Yeah. I hear you. I'm, I'm, I'm with you on the projects prediction, but I would say that the adoption and effectiveness of it has far outpaced what I would have guessed five years ago. + +00:10:08 Yeah. I mean, even local models, like it's kind of exciting, like what you can just run on your Mac book. + +00:10:13 I also didn't really anticipate that. yeah. + +00:10:16 All right. I don't want to make this the AI show cause it's not about AI, + +00:10:19 but your background is so interesting. So when you were working initially on transformers did you perceive you as a group not you individually you guys perceive the need for the scale of the computation like you know to train gpt01 i don't know how much compute was involved but probably more than cities use in terms of energy sometimes you know what i mean it was a lot and how much compute were you all using back then versus now i guess is what i'm getting that so + +00:10:50 i guess yeah and i think you kind of said it but just just to clarify or to make it clear that i personally didn't work on transformers i was working on tensorflow so that yeah + +00:10:59 the yeah the i guess embedded dsl for for training machine learning models that was used within google and still used today um i can't give you specific numbers to be honest because i was pretty deep in the systems part of working on tensorflow and not paying too much attention for like standing up distributed clusters. + +00:11:21 I did things like adding support to TensorFlow for creating a function library. + +00:11:27 TensorFlow is like a Dataflow graph engine, and it didn't have functions, so I added support for functions, partitioning functions over devices, but other teams used that work to go and train the big models. + +00:11:38 I know it was a lot of compute, but a lot versus a lot, all caps versus just capital case. + +00:11:46 Yeah, I'm sure there's a delta. + +00:11:48 I just, I don't know how much. + +00:11:49 Yeah. + +00:11:50 Well, today they're all talking about restarting nuclear reactors and plugging them straight to the data center. + +00:11:55 So it's a weird time. + +00:11:57 Let's go back. + +00:11:59 Maybe a little bit before then, 2012 or so, I believe is maybe the right timeframe. + +00:12:05 And let's talk about notebooks. + +00:12:07 And, you know, around then, Jupyter notebooks burst on the scene. + +00:12:12 And I think they fundamentally changed the ecosystem of Python and basically scientific work in general, right? + +00:12:22 By changing Python, I mean, there are so many people in Python who are here because they got pulled into a notebook. + +00:12:28 They wrote a little bit of code. + +00:12:30 They got incredible results. + +00:12:32 They were never, ever a programmer, but now they're the maintainer of an open source project or something like that. + +00:12:36 You know what I mean? + +00:12:38 They're just like, oh, this is kind of neat. + +00:12:40 oh I'll create a biology library or whatever and then all the all of a sudden you know they're they're deep in it and so jupiter I think was really the start of that maybe there was some precursors that I'm not familiar with + +00:12:54 yeah jupiter was essentially the start of it especially for python like I think + +00:12:58 the idea of computational yeah notebooks has been around but uh yeah the ipython repl I guess the ipython project with the ipython repl I think was early 2000s and um And Jupyter, I think you're right, like 2012-ish, as he's burst onto the scene with putting the IPython REPL into a web browser with multiple REPL cells, and all of a sudden you could do more than what you could do in the console and also made it a lot more accessible. + +00:13:25 Yeah. + +00:13:26 Going back to that sixth cell you entered on your REPL and editing it, that is not a way to draw beginners in. + +00:13:35 No, definitely not. + +00:13:37 Up arrow key six times, no. + +00:13:38 Yeah, 2012, coincidentally, is when I started college as an undergrad. + +00:13:44 So I had been using, I kind of grew up, as long as I used Python, I think I probably was using Jupyter Notebooks. + +00:13:52 And I even started using Google CoLab before it was made public, because when I was interning at Google, and I think the year I started working at Google full-time is when they made it public to the world. + +00:14:03 So, yeah, notebooks have sort of shaped the way that I think and interact with code quite a bit. + +00:14:09 Yeah, absolutely. + +00:14:11 So maybe you could give us some perspective. + +00:14:14 It sounds like you were there, like, right at the right time. + +00:14:16 You know, what was the goal and sort of the job of Jupyter Notebooks and those kinds of things then? + +00:14:23 And how has that changed now? + +00:14:25 And maybe how does it still solve that problem or not solve it so much? + +00:14:29 Yeah, I think, so stepping back. + +00:14:32 So you mentioned like how many people who might not have traditionally, you know, worked with code sort of came into it because of Jupyter Notebooks. + +00:14:41 And I think that's like really, I think you're onto something there. + +00:14:43 I think that's correct. + +00:14:44 And I think the special thing about Notebooks is that they like mix code and visuals and like narrative text in like this interactive programming environment. + +00:14:55 And that interactivity in particular, I think is really important for anyone who works with data. + +00:15:01 It's where you got to like first like, you know, run some queries against your data to see the general shape of it, you know, run some transformations to see like, you know, basically like explore your data, like hold it in your hands. + +00:15:14 Right. And like, you know, REPLs before and then Jupyter Notebooks in particular lets you do that in a way that you just can't easily do in a script because it holds the variables in memory. + +00:15:23 And so I think the role that it played was that it enabled data scientists or computational scientists, biologists, astrophysicists, all these different types of people who work with data. + +00:15:34 It let them rapidly experiment with code and models and importantly, get some kind of like artifact out of it. + +00:15:45 And in particular, this was like a static HTML type artifact, right, where you have some text documenting what experiments you're running. + +00:15:53 you have Python code, and then you have like Matplotlib plots and other artifacts. + +00:15:59 And I think for the sciences in particular early on, that was super important and transformative. + +00:16:07 And we were talking about other computational notebook environments. + +00:16:10 And it's interesting to mention this because things like this existed sort of before, like Mathematica's workbooks, I think they + +00:16:17 called them. + +00:16:17 I might have done that wrong. + +00:16:19 But I think just like the open source nature of Python and the accessibility of it just like made it way easier to get started. + +00:16:26 And also the fact that it was like browser based, right, so that like you can easily share like these, the HTML artifacts that come out, I think was really transformative. + +00:16:37 And insofar as like what they're used for today, they're used for a lot. + +00:16:41 It's kind of remarkable how much they're used. + +00:16:44 Like I remember seeing a statistic for Google CoLab alone, which is just one particular hosted version of a Jupyter notebook. + +00:16:53 And it was two years ago, and they said something about like having 10 million monthly active users. + +00:16:58 And so the number of people who use Jupyter on a monthly basis is surely larger than that. + +00:17:04 Oh, yeah. + +00:17:05 And they're used for the sciences, but they're also used for like things where you want, like the traditional rigor and reproducibility of software engineering. + +00:17:14 So they're increasingly used for like data pipelines and like ETL jobs. + +00:17:19 And like, so anyone who's in Databricks, for example, runs like, they call them workflows, Databricks workflows, through essentially Jupyter Notebooks. + +00:17:29 They're used for training machine learning models. + +00:17:31 Like I've trained many models in a Jupyter Notebook. + +00:17:35 I've developed algorithms in Jupyter Notebooks. + +00:17:37 I've produced scientific figures that go straight into my paper from a Jupyter notebook. + +00:17:42 And the reason people do this is because, again, the interactivity and the visuals that come out is very liberating compared to using a script. + +00:17:51 I think there's also a developing understanding that happens from them. + +00:17:56 Because when you're writing just a straight program, you're like, okay, this step, this step, this step, this step, and then we get the answer. + +00:18:02 Whereas notebooks, it's like step one, step two. + +00:18:05 Let me look at it. + +00:18:06 oh, maybe that's different. + +00:18:07 Let me go back and change. + +00:18:08 It's more iterative from an exploratory perspective. + +00:18:12 Yeah, I think that's exactly right. + +00:18:15 Because you have that, when you're writing a software system, you kind of know the state of the system. + +00:18:20 You know what you want to change it to. + +00:18:22 But when you're working with data, there's that unknown, what will my algorithm do to my data? + +00:18:26 You just have to run it to find out. + +00:18:27 Yeah, absolutely. + +00:18:28 And graphs and tables and iterative stuff like that is really nice. + +00:18:33 So why create something instead of Jupyter? + +00:18:37 Why is Jupyter not enough for the world? + +00:18:40 Yeah. + +00:18:41 I think there's a... + +00:18:42 So just, you know, you're the creator of Marimo, along with the rest of the team and so on. + +00:18:48 But yeah, it's like, why create this thing when Jupyter exists? + +00:18:53 Yeah. + +00:18:53 So I think there's quite a few reasons. + +00:18:58 So Jupyter notebooks are like, I guess, It's more like Jupyter Notebooks powered by the IPython kernel, just to put a finger on it. + +00:19:07 Right, because it could be different. + +00:19:08 It could be like a C++ kernel or.NET kernel or whatever. + +00:19:12 Yeah, exactly. + +00:19:13 But this particular form of notebooks where you get a page and then in the front end shows you a bunch of cells that you execute one at a time imperatively, even though you have a sequence of cells, it still is essentially a REPL. + +00:19:27 And like it's the onus falls on the developer, the scientist, whoever's using it to like run each cell on their own and like maintain the state of their kernel. + +00:19:38 So like you run a cell, say you have three cells, you know, you decide to go rerun the second cell. + +00:19:43 You have to remember to rerun whatever cell depends on it. + +00:19:47 Maybe it's the third cell. + +00:19:48 Maybe it's the first cell because you wrote things out of order. + +00:19:50 And so you can easily get into a state with Jupyter Notebooks where your code on the page doesn't match the outputs that were + +00:19:57 created. + +00:19:58 Yeah. So if you were to like basically go to the menu option, say rerun all cells, you would get different answers. + +00:20:03 Exactly. And this is like it's actually been studied and like it happens a kind of a shockingly large amount of times that like. + +00:20:11 So there's one study in 2019 by Pimentel, I'm going to probably pronounce his name wrong, but by I think four professors. + +00:20:20 And they studied a bunch of notebooks on GitHub. + +00:20:22 And they found that only like a quarter of the notebooks that were on GitHub, they were able to run it all. + +00:20:28 And then like either 4% of those or 4% of all of them, only 4% of them like when they ran recreated the same results that were serialized in the notebook. + +00:20:38 Meaning like maybe people ran sales out of order when they originally created the notebook and then committed it. + +00:20:45 Or maybe they didn't like create a requirements.txt that faithfully captured the package environment they used. + +00:20:53 So there's like this recognition that Jupyter Notebooks suffer from a reproducibility crisis, which is rather unfortunate for a tool that is used for science and data science and data engineering, like things where reproducibility is paramount. + +00:21:07 And so that was one issue. + +00:21:08 Reproducibility was like one main thing that sort of we wanted to solve for with Marima notebooks. + +00:21:13 And the other thing I think that we alluded to earlier is that Jupyter notebooks are stored as like a JSON file where like the outputs, like the plots, et cetera, are serialized as these binary strings. + +00:21:25 And like the, you know, the one downside of that is that you can't really use these things like you can use like traditional software, at least not without jumping through extra hoops. + +00:21:33 So like you make a small change to your Jupyter notebook, you get a gigantic git diff because, you know, the binary representation of some object changed by a large amount. + +00:21:43 You want to reuse the Python code in that Jupyter notebook when you have to run it through like a Jupytext or some other thing to strip it out. + +00:21:51 And then so you oftentimes you get this like situation where people just like end up copy pasting a bunch of things across a bunch of notebooks and it quickly gets really messy. + +00:22:00 And that was another thing we wanted to solve for, making notebooks like Git-friendly and interoperable with other Python tooling like Git, pytest, etc. + +00:22:10 Yeah, there's some things that will run them kind of function module-like, but yeah, it's not really the intended use case, is it? + +00:22:18 Yeah, it's not, but people want to, which is, I think, like the, it's funny because like the intended use case, I think, of like traditional notebooks like Jupyter was like interactive REPL, explore data, produce like simple narrative text. + +00:22:33 But then like they just got used for so much more than that in situations where reproducibility, reusability is like really important. + +00:22:41 And I think that that's what we're trying to solve for. + +00:22:45 This portion of Talk Python to Me is brought to you by Worth Recruiting. + +00:22:49 Are you tired of applying for jobs and never hearing back? + +00:22:53 Have you been getting the runaround or having trouble making it past the AI resume screeners that act as the new gatekeepers for your next level Python job? + +00:23:01 You should reach out to Worth Recruiting. + +00:23:04 Worth Recruiting specializes in placing senior level Python developers and data scientists. + +00:23:09 They work directly with hiring managers at startups, helping them grow their software engineering and data science teams. + +00:23:16 with Worth it's not just connecting you with the company it will guide you through the interview process and help make sure you're ready with their detailed preparation approach they can even coach you on salary negotiations and other important decision making processes so if you're ready to see what new opportunities are out there for you reach out to Worth Recruiting let them be your partner and specialist to find the right Python developer or data scientist position for you fill out their short contact form at talkpython.fm/worth. It only takes a minute. That's talkpython.fm/worth. + +00:23:50 The link is in your podcast player's show notes. Thank you to Worth Recruiting for supporting the show. I was just wondering out loud, you know, that study. I know it was not your study, so I'm not asking you for answers. Just, just what do you think? If it's that low of a percentage of reproducibility because people fell down on their like rigorous software engineering, for example, like no requirements file or hard-coded pass instead of relative pass or whatever. + +00:24:16 If it's that number for notebooks, I wonder about just plain Python files that are also meant to act to solve like science problems and stuff that just didn't happen to be in notebooks. + +00:24:27 I wonder if they're any better. + +00:24:28 Yeah, it's a good question. + +00:24:31 I think they will be better. + +00:24:33 I can just give an anecdote to why I think they would be better. + +00:24:36 And so one example, and this happened to me many times, But like one particular example is like when I was working on my PhD thesis, which was about vector embeddings, it was me and a couple of co-authors. + +00:24:48 And, you know, both of us did our, like created the examples for this embedding algorithm in like Jupyter Notebooks. + +00:24:57 And these notebooks produced a bunch of plots, et cetera. + +00:25:00 And so at the end of the, once the thesis was written, I was putting up all our code on GitHub. + +00:25:04 And so I asked one of my co-authors, like, hey, like, can you share your notebooks with me? + +00:25:09 And this co-author is amazing. + +00:25:10 He's really smart, but not necessarily trained as a software engineer. + +00:25:14 And he shares his Jupyter notebook with me. + +00:25:16 And I run it. + +00:25:17 It doesn't work at all. + +00:25:19 And the reason it doesn't work at all is that there's like multiple cells. + +00:25:22 And there's like this branching path where like it's clear that like he ran like, you know, you see the execution order serialized in the notebook. + +00:25:29 And it's not like one, two, three, four. + +00:25:31 It's like one, two, seven, four. + +00:25:33 It's like these cells were ran in this path dependent way. + +00:25:36 And then also it's like there's like three cells in the middle. + +00:25:39 He ran just one of them. + +00:25:40 I'm not exactly sure which one. + +00:25:42 And like, whereas if you had a Python script, there's only one way to, I mean, sure, you can have + +00:25:46 data files. + +00:25:46 That's a good point. + +00:25:47 You do not really get a choice. + +00:25:49 It runs top to bottom. + +00:25:51 Exactly. + +00:25:51 Depending on the data. + +00:25:52 Okay. + +00:25:53 So yeah, that is something that's really, it's just very counter to the concept of reproducibility is that you + +00:26:00 can run them in any order and there's not, you know, I feel like Jupyter Notebooks should almost have like a red bar to the cross, like an out of order bar. + +00:26:10 warning across the top if it's not top to bottom. + +00:26:14 It doesn't have to be 1, 2, 3. + +00:26:15 It could be 1, 5, 6, 9. + +00:26:17 But it should be a monotonic increase in function as you do the numbers top + +00:26:22 to bottom. + +00:26:24 Otherwise, it should be like a big warning or like a weird, like some kind of indicator, like, hey, you're in the danger zone. + +00:26:29 You know what I mean? + +00:26:30 Yeah, and there's actually another thing that actually has happened to me way more often than I should care to admit while working with Jupyter Notebooks is that it's not just the order. + +00:26:40 you run the cells it's also like what happens if you delete a cell because when you delete a cell you no longer so say a cell declares a variable x and you deleted that cell you're like i don't want x anymore but you may forget that just by deleting the cell um you're not removing the variables from memory and so like x still exists and so then you're running other cells that depend on x everything's fine uh but then you come back and you run your notebook and everything's broken and you don't You have no idea. + +00:27:07 The first time this happened to me, it took me forever to debug. + +00:27:11 I can imagine that is so rough. + +00:27:13 Yeah. + +00:27:13 Later on, you realize, okay, this is a pattern that happens. + +00:27:17 But it's like, yeah, so that's another thing that we also wanted to solve for in Marimo notebooks. + +00:27:23 Awesome. + +00:27:23 All right. + +00:27:23 Well, tell us about Marimo. + +00:27:26 Okay. + +00:27:27 So let's see. + +00:27:28 So Marimo is, it's an open source notebook for Python. + +00:27:31 The main thing that's different between Marimo and Jupyter is that Marimo is reactive. + +00:27:38 And what that means is that unlike a Jupyter notebook where you can run cells in any arbitrary order you like, in Marimo, when you run one cell, it knows what other cells need to be run in order to make sure that your code and outputs are synchronized. + +00:27:55 So it's kind of like Excel, right? + +00:27:56 So you have two cells. + +00:27:58 One declares X. + +00:27:59 Another one references X. + +00:28:00 you run the cell that declares X or defines X, all other cells that reference X will automatically run or they'll be marked as stale. + +00:28:09 You can configure it if you're scared of automatic execution. + +00:28:12 But the point is that it manages your variables for you so that you don't have to worry about outputs becoming decohered from your code. + +00:28:22 And similarly, if you delete a cell, the variable will be scrubbed from memory and then cells that are dependent on it will either be marked as stale a big warning or automatically run and invalidate it. + +00:28:34 So that reactivity provides some amount of reproducibility. + +00:28:38 And then it also, some of the key features, and then we can dive into each of them, is an additional reactivity. + +00:28:44 Marima notebooks are stored as pure Python files, which makes them easy to version with Git. + +00:28:50 And then we've also made it easy to run Marima notebooks as Python scripts and then share them as interactive web apps with little UI elements. + +00:28:58 Yeah, I'm excited to talk about that. + +00:29:00 Some cool stuff there. + +00:29:01 Okay, so when you create a variable, if you say like x equals one, it's not just a pi long pointer in memory, right? + +00:29:09 It's something that kind of looks at the read and write that get in the set of it and creates like a relationship where it says if you were to push a change into it, it says, okay, I have been changed. + +00:29:21 Who else is interested in reading from me? + +00:29:23 And it can sort of work that out, right? + +00:29:25 How does that work? + +00:29:25 Actually, it's not really implemented in that way. + +00:29:29 Okay. + +00:29:30 So, okay. + +00:29:31 So let's see. + +00:29:32 Yeah, this is interesting to dive into because there's like two ways that you can like get at reactivity. + +00:29:37 Like so in Marima, what we actually do is we do static analysis. + +00:29:41 So we look for definitions and references on like of global variables. + +00:29:46 And then so we just make a graph out of that. + +00:29:48 So for every cell, we look at the, I guess, the loads in the ASTs and then like + +00:29:53 the assignments. + +00:29:57 And so we can see statically who declares what and who reads what, but who is a cell. + +00:30:04 And so that makes it like performant and also predictable of like how your notebook is going to run. + +00:30:10 A alternative, I think what you were getting at was like runtime tracing. + +00:30:14 More like a JavaScript front end sort of deal, like a view binding model binding type thing or something like that. + +00:30:20 Yeah. So yeah, we, everything we do is static, is done with static analysis. + +00:30:27 this there was another project called ipyflow which was like a reactive kernel for jupiter it's still around um they took like the runtime tracing approach of like checking on every assignment like okay + +00:30:39 okay there's a read and now where is that where is that object in memory and then running cells that depend on it um i think in practice you know i was talking to steven mackie the author of that article of that extension and then also like based on some work that i saw at Google TensorFlow, that's really hard to get 100% right. + +00:30:58 It's basically impossible to get 100% right. + +00:31:00 So you will miss some low references and definitions. + +00:31:06 And so there's this weird usability cliff as a user where when you run a cell, you're not sure what else will run. + +00:31:12 Whereas if you do it based on static analysis, you can give guarantees on what will run and what won't. + +00:31:18 Yeah, I guess the static analysis is a good choice because there's only so many cells. + +00:31:23 You don't have to track it down to like, well, within the DOM and all the JavaScript objects, we're linking all these together and they can do whatever they want from all these different angles. + +00:31:32 It's just like, when this cell runs, what other cells do we need to think about? + +00:31:35 So it's kind of like, what does this create? + +00:31:38 What does it change? + +00:31:39 And then push that along, right? + +00:31:40 So it's a little more constrained. + +00:31:42 Exactly, yeah. + +00:31:43 So basically, yeah, that's exactly right. + +00:31:45 And so it's a data flow graph where the data flowing on the edges is the variables across cells. + +00:31:51 Yeah, I don't know. + +00:31:52 I think I've been thinking about Dataflow Grass for a long time, ever since TensorFlow, doing my PhD too when I worked on CVXPY. + +00:31:58 So it's just a thing that I enjoy thinking about and working on. + +00:32:02 Yeah, well, what about your experience working on TensorFlow and at Google and at Stanford and stuff that sort of influenced this project? + +00:32:11 Yeah, a couple of things. + +00:32:14 So definitely a big thing that influenced this project was working as a scientist at Stanford where like I and my colleagues use Jupyter Notebooks like on almost a daily basis because they want to see our data while we worked on it. + +00:32:29 And so really valued the iterative programming environment, really didn't like all the bugs that we kept on running into, which are + +00:32:37 like kind of our fault, right? + +00:32:38 Because of like, oh, we forget to run a cell, et cetera. + +00:32:42 Didn't like that it was not easy to reuse the code in a notebook in just other Python modules. + +00:32:49 Didn't like that I couldn't share my artifacts in a meaningful way with my PhD advisor who can't run Python notebooks. + +00:32:55 I couldn't make a little app to showcase my research project. + +00:32:59 It's so interesting how PhD advisors and just professors in general, how they're either embracing or their inability to embrace or their prejudices for or against the technology so influence science, the people in the research teams, everything. + +00:33:15 It's like, well, we'd like to use this, but the principal author really doesn't like it, doesn't want to run it, so we're not using that. + +00:33:21 Or, you know, just these little edge cases. + +00:33:24 Like, I was made to learn Fortran in college. + +00:33:26 I'm still sour about it. + +00:33:29 That's funny. + +00:33:30 Yeah, no, it's true. + +00:33:31 Yeah, it does have a big effect. + +00:33:33 And then at Google, I guess the things that influenced – so we used Notebooks 2 there. + +00:33:39 Since I was doing the systems engineering work at Google, I personally didn't use them too much, although I did use like Google Colab for like training, like creating training courses and stuff for, for like other engineers and for demos. + +00:33:50 But I guess the part of Google that influenced it was just like thinking about data flow graphs for a + +00:33:55 year. And you know, I, we were, there was like a couple of parable projects, like one that was using runtime tracing for like figuring out like how to make a DAG. And it was just, I don't know, kind of traumatic just because you like you miss things and the user experience is just like it kind of falls over. + +00:34:13 And so I guess the part from there just made me not want to use runtime tracing, + +00:34:19 made me embrace static analysis for this. + +00:34:22 Yeah. + +00:34:22 Yeah, it makes a lot of sense. + +00:34:24 All right. + +00:34:25 Well, let's talk some features here. + +00:34:27 What are the, I guess probably the premier feature is the reactivity, in which case it doesn't matter which cell you run, they're never going to get out of date, right? + +00:34:37 Yeah, that's correct. + +00:34:38 So the reactivity, in other words, the data flow graph behind it is the premier feature. + +00:34:45 And it lets you reason about your code locally, right, which is really nice. + +00:34:48 Like you can just look at your one cell. + +00:34:49 You don't really have to worry about, you know, you know what variables it reads and what it defines. + +00:34:54 And you don't have to worry about, oh, but what cells do I need to run before this? + +00:34:58 So not only does it like make sure your code and outputs are in sync, it also like makes it really like fun and useful. + +00:35:08 to really rapidly experiment with ideas. + +00:35:11 As long as your cells aren't too expensive, if you enable automatic execution, you can change a parameter, run a particular cell, and it'll run only what's required to update subsequent outputs. + +00:35:27 And then another consequence of reactivity that I think is really neat and our users really like is it makes it far easier to use interactive UI widgets than it is in like a Jupyter notebook. + +00:35:40 And the way that this works is that, so Marimo is like both a notebook and also a library. + +00:35:47 So you can import Marimo as Mo into your notebook. + +00:35:50 And when you do this, you get access to a bunch of things, including a bunch of different UI widgets, ranging from the simple ones like sliders, drop-down menus, to like cooler ones like interactive selectable charts and data frame transformers that automatically generate the Python code needed for your transformation. + +00:36:12 And the way that reactivity comes into here is that because we know when you create a UI element, as long as you bind it to a global variable, just say my slider equals mo.ui.slider, then when you interact with the slider anywhere on the screen, well, we can just say, okay, that slider is bound to X. + +00:36:31 We just need to run all other cells that depend on X. + +00:36:33 all of a sudden you have really nice interactivity, interactive elements controlling your code execution without ever having to write a callback. + +00:36:42 And so I think people find that really liberating too. + +00:36:44 So you don't really have to hit backspace, change a character value for your variable, shift, enter, shift, enter, shift, enter. + +00:36:52 Just change a number and everything automatically recalculates. + +00:36:56 Yeah. + +00:36:56 In this dependency flow, is it possible to have a cyclical graph? + +00:37:02 Like if I did this in Jupyter, I could go down to the third cell and define X and then go back up and then use X. + +00:37:09 If I run it in the right order, it won't know that there's a problem. + +00:37:13 Imagine it's only possible to go one way, right? + +00:37:16 You can't get into a weird cycle. + +00:37:19 Yeah. + +00:37:19 So we do give people escape patches. + +00:37:22 We ask them, please don't use it unless you really know what you're doing and you really want to do this. + +00:37:27 Because in most cases we find, especially for like if your goal is to just like work with data data scientist, data engineer scientist. We feel you don't need that cycles. You don't need cycles. + +00:37:40 But if you're making an app, and I guess I should mention Marimo lets you run any notebook as well a notebook, but also from the command line, you can serve your notebook as a web app, like similar to Streamlit if you've seen that. In those cases, sometimes you do want state basically, right? Like cyclic references. So by default, Marimo actually checks for cycles. If you have a cycle across cells, politely tells you, hey, that's not really cool in Marimo. + +00:38:07 Please break your cycle. + +00:38:08 Here's some suggestions how to do that. + +00:38:11 But if you're really insistent, then you can go to our docs and learn about the state object that we have that does let you get into some runtime-based cell execution. + +00:38:20 You can update a state object and kind of in a React from 10D way, it'll run dependence of that state. + +00:38:26 Okay, interesting. + +00:38:27 Yeah, I guess that makes sense for apps. + +00:38:28 You definitely need some sort of global state there. + +00:38:32 Yeah, I think we've seen that sometimes our users will reach for state in call VIX just because that's what they're used to. + +00:38:41 And that's what they maybe were required to use in Jupyter, et cetera. + +00:38:45 So we're trying to tell them, hey, in most cases, you actually don't need this. + +00:38:49 Yeah, very cool. + +00:38:50 So when you create a Jupyter notebook, you don't really create Python files. + +00:38:56 you create the notebook files, and those notebook files are JSON with both embedded cell execution bits, the code, but also the answers that go below. + +00:39:05 That's why if you go to GitHub and you look at a notebook, you can instantly see what could be super expensive computation, but pictures and stuff there because it's residual in the artifact, right? + +00:39:16 Yeah. + +00:39:17 If you guys do Python files, do you have a way to save that state in a sort of presentation style or something like that? + +00:39:26 You know what I mean? + +00:39:26 Is there like an associated state file? + +00:39:29 Yeah, so that's a really good question. + +00:39:31 And the answer is yes, but that is the biggest trade-off, I think, of our file format. + +00:39:35 So like a surprising number of people actually have come to us and told us, I didn't expect this, but they tell us the reason we decided to try Marima Notebooks is because you say it is versionable with Git. + +00:39:47 Like that's like, they're like, that's the one reason I, especially like people in software and industry. + +00:39:52 And then they stay for all the other things. + +00:39:55 So that's what the pure Python file format allows, in addition to modularity, running it as a script. + +00:40:02 For seeing outputs, yeah, they're not saved in the Python file. + +00:40:05 So we have a couple of ways to get around that. + +00:40:09 So one, you can export any Marimo notebook to an IPython notebook file, actually. + +00:40:14 So you can set up this, we have a feature, automatic snapshotting. + +00:40:18 You can automatically snapshot your notebook to like a parallel IPIMB file that's stored in a subdirectory of your notebook folder. + +00:40:28 And so you can push that up or share that if you like. + +00:40:32 When working locally, though, we actually have this cool feature that my co-finder Miles recently built is that we actually – so when you're working, you're working with your notebook, the outputs are actually saved. + +00:40:45 The representation of the outputs are saved in this subdirectory, underscore underscore marimo. + +00:40:51 So that the next time you open the notebook, it just picks up those outputs and then like loads them into the browser so that you can see where you left off. + +00:41:01 So I see. So kind of like PyCache. + +00:41:04 Yeah. + +00:41:04 I would actually assume given the name, but I haven't actually looked into PyCache too much. + +00:41:09 Well, I mean, just in the sense that it's saved like right there in your project using that directory. + +00:41:14 Oh, yes, yes. You have a PyCache folder. Exactly. + +00:41:16 Yeah, yeah, yeah. Exactly. + +00:41:18 And we have one other feature that another contributor, his name is Dylan, has been developing, which I think is really cool, which is along those same lines, but based on Nick's style caching. + +00:41:32 So it'll actually save the Python objects themselves using a variety of different protocols. + +00:41:40 And because we have the DAG, he can actually guarantee consistency of the cache. + +00:41:47 But that means not only are your outputs automatically loaded, also the variables are loaded, so you can literally pick up where you left off. + +00:41:54 Yeah, it sounds like some sweet SQLite could be in action there instead of just flat files or whatever. + +00:42:01 Yeah. + +00:42:02 Yeah, very cool. + +00:42:02 So you talked about when the reactivity fires, it can sometimes be expensive to recompute the cells. + +00:42:10 Sometimes they're super simple. + +00:42:12 Sometimes they're trained in the machine learning model for two days. + +00:42:16 Yeah. + +00:42:17 Are there caching mechanisms? + +00:42:19 You know, in a Python script, we've got functools.lrucache, and then there's other things that are kind of cool, in-process caches, like PyMocha is kind of like the functool stuff, but way more flexible and so on. + +00:42:33 And you could put those onto functions that then would have really interesting caching characteristics, like, hey, if you're going to rerun it, but I've run it before with this other value, here's the same answer back, right? + +00:42:43 Is there a way to set up that kind of caching or performance memoization type stuff? + +00:42:50 Yeah, definitely. + +00:42:51 And that same contributor, Dylan. + +00:42:53 So he's implemented these and is continuing to build it out. + +00:42:57 But we have, I guess, two options. + +00:43:02 One is in-memory cache, which is basically the API is the same as functools.cache, but is designed to work in an iterative programming environment. + +00:43:10 If you use functools.cache naively, if a cell defining a function reruns, your cache gets busted. + +00:43:18 So modoc.cache is a little smarter, and using the DAG can know whether or not it needs to bust the cache. + +00:43:25 But to the user, it feels the same. + +00:43:28 And then we also have a persistent cache, which is the one that I was alluding to when I mentioned you can automatically pick up where you left off by loading objects from disk. + +00:43:37 And so you can put an entire cell in a context manager with MoDub persistent cache. + +00:43:44 It'll sort of do the right thing. + +00:43:46 We've been talking about experimenting with letting users opt into global cell-wide caching, just automatically memorize cells or just have a UI element + +00:43:55 to do something like that. + +00:43:57 But we haven't quite done that yet. + +00:43:59 Yeah, I mean, it sounds real tricky for Jupyter. + +00:44:01 But because you all know the inputs and the outputs, It's almost as if you could sort of drive a hidden LRU cache equivalent at the cell level. + +00:44:11 You know what I mean? + +00:44:12 That's exactly right. + +00:44:13 Yeah. + +00:44:14 I think, yeah, because we know the inputs and outputs, that's kind of how everything flows from a project. + +00:44:19 You can think about projects like if you turn a notebook into a data flow graph, what can you do? + +00:44:24 And yeah, caching is, smart caching is definitely one of them. + +00:44:27 Yeah. + +00:44:27 I mean, it's not a, I make a sense like, well, you know, the inputs, outputs, so it's fine. + +00:44:32 But, you know, you could have the same data frame, but you could have added a column to the data frame, and the LRU cache goes, is it the same pointer? + +00:44:39 It is. + +00:44:40 We're good to go. + +00:44:40 It's like, well, yes. + +00:44:41 However, it's not the same. + +00:44:44 You know, you almost got to, like, hash it or do something funky. + +00:44:47 Yeah, it's not perfect. + +00:44:48 Automatic sounds hard. + +00:44:49 No. + +00:44:50 Yeah, yeah, exactly. + +00:44:51 That's why we haven't quite enabled it. + +00:44:54 Yeah, it is hard. + +00:44:56 Yeah. + +00:44:56 Side effects, too, network requests, all these things. + +00:44:59 All right. + +00:45:00 I have three more topics I want to cover before we run out of time here. + +00:45:04 We may cover more, but three are required. + +00:45:07 First one, because this is the one I'm going to forget most likely. + +00:45:10 You were at Google, and then by way of going through a PhD, you now are not at Google creating this project, which is open source on GitHub. + +00:45:20 And I don't see a pricing page at the top, but I do see it for enterprises. + +00:45:24 Like, how is this sustaining itself? + +00:45:27 Like, what's the business model? + +00:45:29 So when we first started right after my PhD, so that was early 2022, we were actually lucky enough to get funding from a national lab associated with Stanford. + +00:45:42 So the Stanford Linear Accelerator. + +00:45:44 So I was talking with some scientists there and we were talking about, I knew I wanted to make this thing. + +00:45:50 And they were like, what are you up to? + +00:45:51 And I was like, oh, well, I want to make this like notebook thing that like, you know, fixes all notebooks inspired by Pluto JL. + +00:45:57 And they were like, that's awesome. + +00:45:59 We're scientists. + +00:45:59 We use notebooks every day and we're getting a little tired of reproducibility issues. + +00:46:03 We want to make apps. + +00:46:04 So they're like, we'll fund you to do this. + +00:46:06 And so we... + +00:46:07 Awesome. + +00:46:07 So they kind of wrote you into one of their larger grants, something like that. + +00:46:11 We got like a subcontract that like was covered by one of their grants. + +00:46:14 Yeah. + +00:46:15 So that wasn't enough for me and my co-founder, Miles, to work on it for two years full time alone. + +00:46:21 And like, I think it really let us polish the product and develop it. + +00:46:27 And then mid last year, in July. One of our users is like one of our probably biggest power users actually is Anthony Goldblum, who is the founder and former CEO of Kaggle. So + +00:46:40 really into data science. So he's been one of our biggest power users. And at a certain point he last year became, I guess, passionate enough about our project that he reached out to us and asked us, hey, like, can this venture fund I'm part of, can we invest? + +00:46:58 And so that was July of last year. + +00:47:00 And so we raised money from them as well as from a bunch of prominent angel investors like Jeff Dean from Google, Clem from Hugging Face, Lucas B. + +00:47:09 Walt from Weights & Bias and others. + +00:47:12 So now our team is funded primarily through the venture funding. + +00:47:18 Right now our business model is build open source software based on our venture + +00:47:22 funding. + +00:47:23 We do have plans for commercialization, but we're just not at a point where it makes sense to work on that right now. + +00:47:28 But what I can promise is that Marima will always be open source Apache 2.0 license. + +00:47:33 We never plan to sell the notebook itself, but instead plan to work on complementary infrastructure. + +00:47:39 Yeah, I can already think of two good ones. + +00:47:43 Cool. + +00:47:44 Yeah. + +00:47:45 Awesome. + +00:47:45 Okay. The reason I ask is, you know, people always just want to know either + +00:47:49 how, what is kind of the success story of open source, but sustainable, you know, as a full-time thing, or if they're buying into it, how likely is that going to stay open source or like what's the catch, you know, that kind of stuff? + +00:48:06 Yeah, no, it's a good question. + +00:48:08 I mean, honestly, I think when we raised money, it actually assuaged some of our users because they were like, oh, Miles and Akshay are really cool. + +00:48:18 They're building this really cool stuff, but how are they paying rent? + +00:48:22 And so when we told them, yeah, we raised a few million dollars from X Ventures, they were like, okay, that's great. + +00:48:28 You deserve it. + +00:48:29 Please continue building. + +00:48:31 Nice. + +00:48:32 Yeah. + +00:48:33 Yeah, that's awesome. + +00:48:34 All right. + +00:48:35 Let's see, which one are we going to talk about next? + +00:48:37 Not this one yet. + +00:48:39 We'll talk about this because there's a nice comment from Amir out in the audience. + +00:48:42 I love the new AI feature in Mario. + +00:48:44 So you have an AI capability. + +00:48:47 And then the other thing I want to talk about is, so we save some time for it, is publishing or running your Reactive Notebook as an app. + +00:48:55 But let's talk AI first because we started the show that way. + +00:48:59 I don't want to start to round it out that way, you know? + +00:49:02 So we have like a few different ways that AI is, I guess, integrated into the project. + +00:49:08 So one thing that we're trying really hard to do is like build like a modern editor, like designed specifically for working with data. + +00:49:15 And I think these days modern means one of the requirements is you have AI stuff built in. + +00:49:20 So you can like generate code with AI. + +00:49:22 You can like even tell the, you can like tag like data frames and like tables that you have in memory and like give them as context, give their schemas as context to your assistant. + +00:49:32 in like a cursor-like way. + +00:49:35 We're also like experimenting with a new service right now like marimo.amp.ai where you can go type a little prompt like generate me a notebook that plots a 3D quadratic surface with Matplotlib because I always forget how to do that. + +00:49:51 And then it'll do its best to in one shot create that notebook for you and then you can play around with it and then you can either download it locally or share it out. + +00:50:04 Yeah, very cool. + +00:50:05 I'll put a link to the online AI feature that people can play with. + +00:50:09 It looks really, really nice and it worked super quick when I asked it to go. + +00:50:12 Now, one thing I want to talk about really quick now that I'm looking at this is you've got the code, but it's below the presentation of the result of the code. + +00:50:22 Yeah. + +00:50:23 So that is a stylistic choice, which is configurable because people told us, Many of them told us we don't like this. + +00:50:30 Please just put the output below. + +00:50:31 But other people like it. + +00:50:33 Okay, you love it. + +00:50:34 Okay, yeah. + +00:50:34 And so this is... + +00:50:35 Okay, well, what is the point? + +00:50:36 Is the point for me to see the code or the answer? + +00:50:39 The point is to see the graph and the tables and if I care, I'll look at the code. + +00:50:44 So the way that I... + +00:50:46 So I think I mentioned Pluto JL really briefly. + +00:50:49 So Pluto JL is one of our biggest inspirations. + +00:50:51 It's a Julia project, a reactive notebook. + +00:50:54 Now, a lot of the great ideas that I think are great about Marimo came, honestly, very straight from pluto and one thing that fawns like the creator of pluto's you know he likes to say is he said code is the code of a cell is a caption for its output and that's the that's the way you think about it and when you think about it that way it makes a lot of sense um but i think like pluto and then observable also both have outputs on top i don't know it's like i feel like if you it's like one of those heuristics you look at a notebook output on top it's a reactive notebook otherwise it's imperative + +00:51:23 i gotcha gotcha yeah i can see why you might not like it but i like it so i think + +00:51:30 it's pretty cool yeah and then this um marmo.app slash ai once you create one of these you have a shareable link that you can hand off to other people right yeah + +00:51:40 so you can open a new tab which i think will so actually this all of this actually interestingly enough is running in our WebAssembly playground um + +00:51:48 yeah so there's that url right there which you can just share You can also do like the little, there's a hamburger menu, which you can click on to get a permalink that's shorter. + +00:51:58 But yeah, honestly, we just built this generate with AI feature like a little over a week ago. + +00:52:05 And just curious to see what people use it for. + +00:52:08 There's a lot more we could invest here. + +00:52:10 I mean, you know, everyone talks about agents. + +00:52:12 You can think about, oh, data science agent. + +00:52:14 But for now, we're just trying to one shot thing and see how it lands with folks. + +00:52:19 Yeah, sure. + +00:52:20 Let's start with that. + +00:52:20 Well, it looks really great. + +00:52:21 And the UI is super nice. + +00:52:23 So well done. + +00:52:25 Appreciate it. + +00:52:25 I'll pass that on to Miles. + +00:52:26 He built this as a weeknight project. + +00:52:30 Sometimes you just get inspired. + +00:52:32 Just like, you know what? + +00:52:33 I'm doing it. + +00:52:34 I'm just taking two days off the regular work and I'm just doing this. + +00:52:38 You know what I mean? + +00:52:38 Yeah. + +00:52:39 Yeah, definitely. + +00:52:40 Yeah. + +00:52:41 All right. + +00:52:41 So the final thing I want to talk about, which you kind of hinted at a little bit there with the WebAssembly stuff, is I want to run my app. + +00:52:50 So tell us about, as I'm fumbling around to find a place to show you. + +00:52:55 Anyway, tell us about running the app. + +00:52:58 Yeah, so there's a few different ways. + +00:53:02 The sort of traditional way, if you have a client-server architecture, so you have your notebook + +00:53:09 file, notebook.py. + +00:53:12 It has some UI elements, et cetera. + +00:53:14 It has some code. + +00:53:15 You get an app, but just by default, you hide all the code. + +00:53:18 Now you have text, outputs, UI elements. + +00:53:20 You can think of it as an app. + +00:53:21 So if you type marimorun, notebook.py, at the command line, it'll start a web server that's serving your notebook in a read-only session that you can connect to, et cetera. + +00:53:36 Can you do the widgets? + +00:53:40 Like if it's read-only, can I slide the widgets to see it do stuff? + +00:53:44 You know what I mean? + +00:53:44 Yeah, yeah, yeah. + +00:53:45 So you can slide the widgets, see it do stuff. + +00:53:47 You just can't change the underlying Python code. + +00:53:49 Like you can't like, got it. + +00:53:51 But yes, that's true. + +00:53:53 That is real. + +00:53:53 Exact RM dash RF. + +00:53:55 Okay. + +00:53:55 Let's go. + +00:53:56 Yeah. + +00:53:56 Yeah. + +00:53:57 So that is not allowed. + +00:53:58 And so that's the traditional way client server. + +00:54:01 but, last year we actually, I think it was last year. + +00:54:05 we, we added support for, WebAssembly through the PyDype project, which is a port of CPython to, to Wasm slash Unscriptum. + +00:54:14 And so now you can actually take any Marima notebook. + +00:54:17 And as long as you satisfy some constraints, which are documented on our website, you can export it as like static HTML and some assets and just throw it up on GitHub pages or server it wherever you like. + +00:54:28 And that's also what our online playground is powered by. + +00:54:31 And we found this to be just like a really easy way to share notebooks, easy for folks in industry, easy for educators, and also just incredibly satisfying. + +00:54:39 In our docs, we have tons of little Marima notebooks iframed into various API pages, etc., that just let you actually interact with the code as opposed to just reading it statically. + +00:54:52 Yeah, that's nice. + +00:54:53 That's super cool. + +00:54:54 So it's pretty low effort, it sounds like. + +00:54:57 And it's a lot of work to run a server and maintain it, especially if you're going to put it up there so other people can interact with it. + +00:55:04 And you've got to worry about abuse and all that kind of stuff. + +00:55:07 If you can ship it as a static website, people are just abusing themselves if they mess with it anyway. + +00:55:12 Exactly. + +00:55:13 I had to give huge props to the PyDive maintainers. + +00:55:19 They are amazing, really responsive, and just building this labor of love. + +00:55:24 They are really pushing the needle on the accessibility of Python. + +00:55:32 Yeah, that's awesome. + +00:55:33 Have you done anything with PyScript? + +00:55:35 I haven't done anything with it. + +00:55:37 I know of them. + +00:55:37 So they also use Pyodide underneath the hood. + +00:55:40 Yeah, they let you pick between Pyodide for more data science-y stuff and + +00:55:43 MicroPython for more faster stuff. + +00:55:47 I am pretty not educated about MicroPython. + +00:55:51 I read about it, but I'd be curious on your take also on choosing one versus the end. + +00:55:56 Yeah, well, I think it's brilliant if you want a front-end framework equivalent. + +00:56:01 If you want PyView or PyReact or whatever, You don't need all the extras. + +00:56:07 You kind of need the Python language and a little bit of extras and the ability to call APIs, right? + +00:56:12 And so MicroPython is a super cut-down version that's built to run on like little tiny chips and self-contained systems on a chip type things. + +00:56:22 So it's just way lighter. + +00:56:23 It's 100K versus 10 megs or something like that. + +00:56:26 And if it solves the problem, right, if it gives you enough Python, I guess is the way to think of it, then it's a really cool option for front-end things. + +00:56:35 I don't know that it works for you guys, right? + +00:56:37 Because you want as much compatibility with like Matplotlib and Seaborn and all these things. + +00:56:43 But if your goal is like, I want to replace the JavaScript language with the Python language, and I know I'm in a browser, so I'm willing to make concessions, I think it's a pretty good option. + +00:56:52 Yeah, that makes sense. + +00:56:53 Yeah, you're right. + +00:56:53 We want max compatibility. + +00:56:55 Like we have one demo where like you can load sklearn, take PCA of these like images of numerical digits, put it in all Terraplot, select them, get it back as a data frame. + +00:57:05 That's all running in the browser. + +00:57:06 And, like, it's magical. + +00:57:08 It's on your home screen. + +00:57:09 People can watch the little embeddings explore. + +00:57:12 It's super cool. + +00:57:13 It's very cool. + +00:57:14 It's honestly magical what PyDat has been able to enable. + +00:57:18 Yeah. + +00:57:19 I imagine it's just getting started. + +00:57:21 I think so. + +00:57:22 Yeah. + +00:57:22 Yeah. + +00:57:23 They're constantly shipping. + +00:57:24 They just added support for Wasm GC. + +00:57:27 So performance is increasing. + +00:57:29 Expose the bug in WebKit. + +00:57:31 along the way. + +00:57:33 Wow, okay. + +00:57:34 That's good. + +00:57:35 Yeah. + +00:57:36 Cool. + +00:57:37 All right, well, we're getting to the end of our time here. + +00:57:40 So, you know, people are interested in this. + +00:57:42 They might want to try it for themselves. + +00:57:44 Maybe they're part of a science team or a data science team at a company. + +00:57:49 What do you tell them if they're interested in Marmo and they want to check it out? + +00:57:53 The easiest way, I think, is to just start running it locally. + +00:57:57 And so our sort of source of truth is our GitHub repo. + +00:58:01 So github.com, marimoteam.com, or if it's easy to remember, our homepage is marimo.io, which links to our GitHub. + +00:58:09 But then you can install from pip or UV or whatever your favorite package manager is and just sort of go to town. + +00:58:15 If you really just don't have the ability to do that for whatever reason, you can go to marimo.new, which will create a blank Marimo notebook powered by WebAssembly in your browser. + +00:58:25 It has links to various Marimo tutorials, as an example, notebooks. + +00:58:31 We didn't even get to talk about the UV integration, which may be another time, but that's another way that Marimo makes notebooks reproducible down to the packages. + +00:58:39 Oh, yeah, okay. + +00:58:41 UV is fantastic. + +00:58:42 I'm a huge fan of UV and Charlie Marsh and team. + +00:58:45 Yeah, but GitHub, Marimo.io, and docs.marimo.io is what I would recommend. + +00:58:51 Thank you so much for being on the show. + +00:58:52 Congratulations on the project. + +00:58:55 It's really come a long way, it sounds like, so it looks great. + +00:58:58 Thanks, Michael. I appreciate it. + +00:58:59 It was a lot of fun to chat. + +00:59:00 Yeah, you bet. Bye. + +00:59:02 Bye. + +00:59:03 This has been another episode of Talk Python to Me. + +00:59:06 Thank you to our sponsors. + +00:59:08 Be sure to check out what they're offering. + +00:59:09 It really helps support the show. + +00:59:11 This episode is sponsored by Worth Recruiting. + +00:59:14 Worth Recruiting specializes in placing senior-level Python developers and data scientists. + +00:59:19 Let Worth help you find your next Python opportunity at talkpython.fm/worth. + +00:59:25 Want to level up your Python? + +00:59:26 We have one of the largest catalogs of Python video courses over at Talk Python. + +00:59:30 Our content ranges from true beginners to deeply advanced topics like memory and async. + +00:59:35 And best of all, there's not a subscription in sight. + +00:59:38 Check it out for yourself at training.talkpython.fm. + +00:59:41 Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +00:59:46 We should be right at the top. + +00:59:47 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +00:59:57 We're live streaming most of our recordings these days. + +00:59:59 If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:00:08 This is your host, Michael Kennedy. + +01:00:09 Thanks so much for listening. + +01:00:10 I really appreciate it. + +01:00:12 Now get out there and write some Python code. + diff --git a/transcripts/501-marimo-reactive-notebooks-for-python.vtt b/transcripts/501-marimo-reactive-notebooks-for-python.vtt new file mode 100644 index 0000000..4fd34b6 --- /dev/null +++ b/transcripts/501-marimo-reactive-notebooks-for-python.vtt @@ -0,0 +1,2633 @@ +WEBVTT + +00:00:00.020 --> 00:00:08.320 +Have you ever spent an afternoon wrestling with a Jupyter notebook, hoping that you ran all the cells in just the right order, only to realize your outputs were completely out of sync? + +00:00:08.900 --> 00:00:11.800 +Today's guest has a fresh take on solving that exact problem. + +00:00:12.380 --> 00:00:20.040 +Akshay Agrawal is here to introduce Marimo, a reactive Python notebook that ensures your code and outputs always stay in lockstep. + +00:00:20.640 --> 00:00:21.840 +And that's just the start. + +00:00:21.900 --> 00:00:32.700 +We'll also dig into Axie's background at Google Brain and Stanford, what it's like to work on cutting-edge AI, and how Marimo is uniting the best of data science exploration and real software engineering. + +00:00:33.580 --> 00:00:39.780 +This is Talk Python to Me, episode 501, recorded Thursday, February 27th, 2025. + +00:00:41.040 --> 00:00:41.220 +Are + +00:00:41.220 --> 00:00:42.640 +you ready for your host, please? + +00:00:43.560 --> 00:00:46.400 +You're listening to Michael Kennedy on Talk Python to Me. + +00:00:46.940 --> 00:00:50.020 +Live from Portland, Oregon, and this segment was made with Python. + +00:00:53.300 --> 00:00:56.140 +Welcome to Talk Python to Me, a weekly podcast on Python. + +00:00:56.840 --> 00:00:58.300 +This is your host, Michael Kennedy. + +00:00:58.740 --> 00:01:11.640 +Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, both accounts over at fosstodon.org, and keep up with the show and listen to over nine years of episodes at talkpython.fm. + +00:01:12.240 --> 00:01:16.100 +If you want to be part of our live episodes, you can find the live streams over on YouTube. + +00:01:16.540 --> 00:01:22.380 +subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. + +00:01:23.000 --> 00:01:25.520 +This episode is sponsored by Worth Recruiting. + +00:01:26.120 --> 00:01:30.500 +Worth Recruiting specializes in placing senior level Python developers and data scientists. + +00:01:31.140 --> 00:01:35.880 +Let Worth help you find your next Python opportunity at talkpython.fm/Worth. + +00:01:36.640 --> 00:01:39.100 +Akshay, welcome to Talk Python to Me. Awesome to have you here. + +00:01:39.400 --> 00:01:41.000 +Thanks, Michael. Really happy to be here. + +00:01:41.280 --> 00:01:43.500 +Yeah, it's data science hour, isn't it? + +00:01:44.640 --> 00:01:45.840 +Yeah, I guess it is. + +00:01:46.320 --> 00:01:54.380 +In a sense, but also in a sense, kind of bringing data science a little closer to traditional, formal computer science or more. + +00:01:55.080 --> 00:01:55.400 +I don't know. + +00:01:55.660 --> 00:01:56.920 +I don't know what to think about computer science. + +00:01:57.120 --> 00:01:57.660 +Software engineering. + +00:01:58.200 --> 00:01:58.780 +Software engineering. + +00:01:59.040 --> 00:01:59.920 +Yeah, that's our goal. + +00:02:00.100 --> 00:02:04.640 +Blending data science with sort of the rigor reproducibility of software engineering. + +00:02:05.120 --> 00:02:05.320 +Yeah. + +00:02:05.700 --> 00:02:07.840 +And I think people are going to find it pretty interesting. + +00:02:08.100 --> 00:02:10.300 +It's definitely gained a lot of traction lately. + +00:02:10.560 --> 00:02:13.420 +So it's going to be really fun to dive into Marimo. + +00:02:13.940 --> 00:02:15.780 +Before we do, though, who are you? + +00:02:15.840 --> 00:02:17.360 +What are you doing here? Tell people about yourself. + +00:02:18.100 --> 00:02:23.880 +Sure. So I'm Akshay. I'm one of the co-founders and developers of Marimo. + +00:02:24.960 --> 00:02:34.000 +I started working on Marimo a few years ago, I guess, a little over two years ago, right after I finished a PhD at Stanford where I was working on machine learning research. + +00:02:35.080 --> 00:02:38.040 +And before that, I was at Google Brain where I worked on TensorFlow. + +00:02:39.400 --> 00:02:51.300 +And I guess for the past several years, I've been wearing different hats, It's either a computer systems hat where I build open source software and domain specific languages for people who do machine learning or I just do machine learning. + +00:02:51.830 --> 00:02:52.120 +And for the + +00:02:52.120 --> 00:02:52.740 +past few years, + +00:02:52.820 --> 00:02:54.520 +it's been the first hat. + +00:02:54.610 --> 00:03:05.260 +So working on the Marimo notebook, which is our attempt to blend the best parts of interactive computing with the reusability and reproducibility of regular Python software. + +00:03:05.550 --> 00:03:06.180 +Well, that's very cool. + +00:03:06.680 --> 00:03:11.120 +You've been right at the cutting edge of this whole machine learning thing, I guess, right? + +00:03:11.220 --> 00:03:15.860 +If you go back a handful of years, it's different today than it was five years ago. + +00:03:16.240 --> 00:03:16.680 +Oh, yeah. + +00:03:16.900 --> 00:03:17.640 +It's very different. + +00:03:17.930 --> 00:03:20.700 +I was at Google Brain 2017 to 2018. + +00:03:21.680 --> 00:03:27.020 +So I think we had Transformers, but we didn't really appreciate all that they could do. + +00:03:27.720 --> 00:03:27.880 +Yeah. + +00:03:29.180 --> 00:03:37.680 +But it was a really exciting time to be at Google Brain where it was pure research, just like one, that you could just work on anything that you wanted to work on. + +00:03:37.800 --> 00:03:47.520 +And I was an engineer there on a, actually on a small sub team inside of TensorFlow where we were working on TensorFlow too, which is sort of making TensorFlow more like PyTorch crudely. + +00:03:47.920 --> 00:03:54.260 +And that was a lot of fun, but yeah, got to go to like a bunch of talks by researchers and it was a special time for sure. + +00:03:54.440 --> 00:03:55.360 +Yeah, I bet it was. + +00:03:55.700 --> 00:03:58.360 +Well, you know, that's such an interesting background. + +00:03:58.520 --> 00:04:04.680 +I think maybe give people a sense of what is Google Brain and all that and maybe just Transformers, right? + +00:04:04.780 --> 00:04:11.840 +It's the foundation of so much surprising and unexpected tech these days. + +00:04:12.260 --> 00:04:13.860 +Yeah, yeah, I'm happy to. + +00:04:14.600 --> 00:04:16.299 +So Google Brain, what is Google Brain? + +00:04:16.400 --> 00:04:20.200 +So I guess technically Google Brain no longer exists as it did back then. + +00:04:20.380 --> 00:04:22.800 +Now it's just DeepMind with DeepMind and Google Brain merged. + +00:04:23.300 --> 00:04:30.140 +But back then Google Brain was, so there was, I guess, two researchers under the alphabet umbrella, Google Brain and DeepMind. + +00:04:31.140 --> 00:04:33.900 +And Google Brain was a place where it was really special. + +00:04:34.020 --> 00:04:42.780 +So there were folks working on all kinds of research projects from you, like using transformers for code generation, but also like systems research projects. + +00:04:42.880 --> 00:04:43.980 +Like I was there when + +00:04:43.980 --> 00:04:45.460 +the JAX + +00:04:45.460 --> 00:04:51.600 +machine learning library, which is an alternative to TensorFlow and PyTorch, was still being in the early days of development. + +00:04:51.780 --> 00:04:56.140 +And I was able to meet with like Roy Frostig, Matt Johnson. + +00:04:56.800 --> 00:05:02.400 +And that was really cool sort of coming up to speed on machine learning systems in that era. + +00:05:03.460 --> 00:05:16.000 +I was only there for a year because I decided I wanted to go back and do a PhD myself in machine learning research where I used actually TensorFlow and PyTorch to work on new sort of like machine learning modules. + +00:05:16.960 --> 00:05:18.540 +Did you feel bad when you used PyTorch? + +00:05:18.640 --> 00:05:19.600 +You're like, oh, I'm cheating. + +00:05:20.340 --> 00:05:21.100 +No, no. + +00:05:21.330 --> 00:05:23.040 +I mean, a little, I guess. + +00:05:23.220 --> 00:05:26.780 +But, you know, it's the thing like everyone else used PyTorch. + +00:05:27.140 --> 00:05:28.680 +And we had just started working + +00:05:28.680 --> 00:05:29.220 +on TensorFlow + +00:05:29.220 --> 00:05:31.760 +too when I joined, whereas PyTorch had already been mature. + +00:05:31.940 --> 00:05:34.240 +So it was the responsible thing to do. + +00:05:36.780 --> 00:05:36.940 +And + +00:05:36.940 --> 00:05:37.500 +Transformers, + +00:05:38.140 --> 00:05:39.020 +what are these Transformer things? + +00:05:39.800 --> 00:05:40.800 +They came out of Google, right? + +00:05:41.320 --> 00:05:42.200 +Yeah, yeah, they did. + +00:05:42.420 --> 00:05:43.600 +Attention is all you need. + +00:05:45.280 --> 00:05:50.600 +These Transformer things, I guess, are the backbone of all these large language models that we're seeing today. + +00:05:51.120 --> 00:05:59.160 +And they proved to have like an incredible capacity for modeling, I guess, in a generative way, all kinds of text. + +00:05:59.260 --> 00:06:02.860 +And in a way that, honestly, if I had to admit, surprised me. + +00:06:03.160 --> 00:06:07.780 +At the time in 27 to 18, we were doing good work in machine translation and stuff. + +00:06:08.560 --> 00:06:18.680 +But even back then, there was researchers who were working on, at Google Brain, code generation, writing a bunch of unit tests, stuff that people do regularly today, actually, with large language models. + +00:06:20.280 --> 00:06:24.120 +I put on my systems engineer hat back then, and I was a little skeptical, to be honest. + +00:06:24.660 --> 00:06:27.400 +And I've just totally proved wrong, and I'm happy to have been proved wrong. + +00:06:28.220 --> 00:06:33.140 +So it's been phenomenal seeing sort of that, the success of that particular model. + +00:06:34.540 --> 00:06:35.320 +And yeah, having + +00:06:35.320 --> 00:06:36.240 +quite a small + +00:06:36.240 --> 00:06:36.960 +part in it. + +00:06:37.100 --> 00:06:43.300 +Yeah, I was, I've been a long time skeptic of machine learning in the sense that for so long, it was, it's kind of like fusion. + +00:06:43.860 --> 00:06:54.960 +It's going to be amazing, but it's 10, 20, 30 years out and maybe, you know, and it was all almost like knowledge, knowledge-based type things. + +00:06:55.140 --> 00:06:55.400 +And just, + +00:06:55.840 --> 00:06:56.460 +I + +00:06:56.460 --> 00:06:57.420 +mean, it was interesting. + +00:06:57.620 --> 00:07:01.880 +it looked kind of relevant and possibly useful in some of the cases, but not like it is today. + +00:07:02.260 --> 00:07:13.240 +Like today, people who don't even care about technology are blown away by this stuff, and they use it all day long, and it's gone so much farther and so much faster than I expected it to. + +00:07:13.900 --> 00:07:15.280 +I'm all about it these days. + +00:07:15.510 --> 00:07:26.060 +I still write most of my code, but any time there's something tricky instead of going to Google and hunting through Stack Overflow, let's see what chat says about it or something like that. + +00:07:27.160 --> 00:07:29.620 +And then maybe we'll go to Google if we don't get a good answer, you + +00:07:29.620 --> 00:07:29.700 +know? + +00:07:29.800 --> 00:07:29.920 +Yeah. + +00:07:30.340 --> 00:07:31.320 +No, no, I'm the same way. + +00:07:31.380 --> 00:07:37.340 +I spent more time than I care to admit last night migrating from Vim to NeoVim. + +00:07:37.360 --> 00:07:47.800 +And I know I'm late to the party, but perplexity and other tools just made it a lot easier to start from scratch installation without bringing over my VimRC. + +00:07:49.020 --> 00:07:49.240 +And you + +00:07:49.240 --> 00:07:50.980 +can ask it detailed questions, right? + +00:07:51.260 --> 00:07:52.540 +For example, I + +00:07:52.540 --> 00:07:56.780 +needed some way to reset the initial offset in DaVinci Resolve timelines. + +00:07:56.960 --> 00:08:00.820 +It was, okay, well, here's your four options, and these are the way you can do it, and these are the tradeoffs. + +00:08:00.960 --> 00:08:05.700 +It's not just you might get an, you know, here is the switch, but you get an analysis. + +00:08:05.800 --> 00:08:08.160 +And I'm pretty impressive. + +00:08:08.420 --> 00:08:19.800 +So the reason I'm going into this is I want to ask you, having been on the inside at the early days, what is your perspective of seeing this trajectory or trajectory is the way I should point it. + +00:08:20.560 --> 00:08:22.140 +Upwards, not logarithmic. + +00:08:22.160 --> 00:08:22.440 +Yeah. + +00:08:23.260 --> 00:08:39.000 +I guess what I will say is I think one thing that's like really changed, like, so back when I was at Brain, even the models were advanced, like, I felt like in order to use machine learning well, you really did need to be somewhat trained in machine learning. + +00:08:39.099 --> 00:08:42.979 +You know, it was important to understand what a loss function, like basic things, like what a + +00:08:42.979 --> 00:08:43.800 +loss function was, + +00:08:43.880 --> 00:08:45.620 +what's training, what's validation. + +00:08:46.480 --> 00:08:49.700 +These days, like, you don't need to, like, at all, actually. + +00:08:49.860 --> 00:08:52.040 +I think especially to use LLMs, right? + +00:08:52.120 --> 00:08:54.660 +Like, you just need to know inputs and outputs. + +00:08:54.820 --> 00:08:56.920 +It's like a function call that sort of has fuzzy outputs. + +00:08:57.600 --> 00:08:57.840 +You know, + +00:08:57.900 --> 00:08:58.840 +it's a mathematical + +00:08:58.840 --> 00:08:59.860 +function in quotes. + +00:09:00.760 --> 00:09:00.900 +Yeah. + +00:09:01.000 --> 00:09:01.320 +So + +00:09:01.320 --> 00:09:04.520 +I think that's one thing that's totally... + +00:09:04.740 --> 00:09:13.760 +I guess it kind of shows that artificial intelligence is transitioning more from, I guess, like a science to a technology, and that tool that people can just readily use without + +00:09:13.760 --> 00:09:14.260 +really needing + +00:09:14.260 --> 00:09:16.460 +to understand how it works. + +00:09:18.200 --> 00:09:22.540 +As for the trajectory, I've been bad at making predictions in the past. + +00:09:22.880 --> 00:09:25.580 +I think I will be bad at predicting anything right now. + +00:09:26.860 --> 00:09:30.400 +Surely, these models are very good for coding. + +00:09:31.040 --> 00:09:55.700 +for massaging marketing copy and like you know i do like the random tasks where like i have a bio i wrote in for first person and someone asked for it in third person and i'm too lazy to change and i just give it to a chat model um yeah but if you ask for a trajectory of agi or asi which i don't even remember what the s stands for these days but i i'm not sure i can predict anything on those timelines. + +00:09:56.260 --> 00:10:07.760 +Yeah. I hear you. I'm, I'm, I'm with you on the projects prediction, but I would say that the adoption and effectiveness of it has far outpaced what I would have guessed five years ago. + +00:10:08.200 --> 00:10:13.260 +Yeah. I mean, even local models, like it's kind of exciting, like what you can just run on your Mac book. + +00:10:13.540 --> 00:10:13.940 +I also + +00:10:13.940 --> 00:10:16.360 +didn't really anticipate that. yeah. + +00:10:16.840 --> 00:10:18.860 +All right. I don't want to make this the AI show cause it's not about + +00:10:18.860 --> 00:10:19.200 +AI, + +00:10:19.440 --> 00:10:50.840 +but your background is so interesting. So when you were working initially on transformers did you perceive you as a group not you individually you guys perceive the need for the scale of the computation like you know to train gpt01 i don't know how much compute was involved but probably more than cities use in terms of energy sometimes you know what i mean it was a lot and how much compute were you all using back then versus now i guess is what i'm getting that so + +00:10:50.840 --> 00:10:59.620 +i guess yeah and i think you kind of said it but just just to clarify or to make it clear that i personally didn't work on transformers i was working on tensorflow so that yeah + +00:10:59.620 --> 00:11:00.640 +the yeah the + +00:11:00.640 --> 00:11:20.140 +i guess embedded dsl for for training machine learning models that was used within google and still used today um i can't give you specific numbers to be honest because i was pretty deep in the systems part of working on tensorflow and not paying too much attention for like standing up distributed clusters. + +00:11:21.340 --> 00:11:26.540 +I did things like adding support to TensorFlow for creating a function library. + +00:11:27.280 --> 00:11:37.980 +TensorFlow is like a Dataflow graph engine, and it didn't have functions, so I added support for functions, partitioning functions over devices, but other teams used that work to go and train the big models. + +00:11:38.540 --> 00:11:43.360 +I know it was a lot of compute, but a lot versus a lot, all + +00:11:43.360 --> 00:11:45.540 +caps versus just capital case. + +00:11:46.760 --> 00:11:46.820 +Yeah, + +00:11:47.160 --> 00:11:47.420 +I'm sure + +00:11:47.420 --> 00:11:47.900 +there's a delta. + +00:11:48.090 --> 00:11:49.500 +I just, I don't know how much. + +00:11:49.720 --> 00:11:49.900 +Yeah. + +00:11:50.230 --> 00:11:55.280 +Well, today they're all talking about restarting nuclear reactors and plugging them straight to the data center. + +00:11:55.480 --> 00:11:56.560 +So it's a weird time. + +00:11:57.920 --> 00:11:58.560 +Let's go back. + +00:11:59.740 --> 00:12:05.240 +Maybe a little bit before then, 2012 or so, I believe is maybe the right timeframe. + +00:12:05.490 --> 00:12:06.760 +And let's talk about notebooks. + +00:12:07.660 --> 00:12:11.700 +And, you know, around then, Jupyter notebooks burst on the scene. + +00:12:12.100 --> 00:12:20.140 +And I think they fundamentally changed the ecosystem of Python and basically scientific work in general, right? + +00:12:22.140 --> 00:12:28.400 +By changing Python, I mean, there are so many people in Python who are here because they got pulled into a notebook. + +00:12:28.920 --> 00:12:30.080 +They wrote a little bit of code. + +00:12:30.410 --> 00:12:31.480 +They got incredible results. + +00:12:32.000 --> 00:12:36.600 +They were never, ever a programmer, but now they're the maintainer of an open source project or something like that. + +00:12:36.980 --> 00:12:37.600 +You know what I mean? + +00:12:38.200 --> 00:12:40.320 +They're just like, oh, this is kind of neat. + +00:12:40.620 --> 00:12:54.240 +oh I'll create a biology library or whatever and then all the all of a sudden you know they're they're deep in it and so jupiter I think was really the start of that maybe there was some precursors that I'm not familiar with + +00:12:54.240 --> 00:12:58.580 +yeah jupiter was essentially the start of it especially for python like I think + +00:12:58.580 --> 00:12:59.620 +the idea of computational + +00:12:59.620 --> 00:13:24.220 +yeah notebooks has been around but uh yeah the ipython repl I guess the ipython project with the ipython repl I think was early 2000s and um And Jupyter, I think you're right, like 2012-ish, as he's burst onto the scene with putting the IPython REPL into a web browser with multiple REPL cells, and all of a sudden you could do more than what you could do in the console and also made it a lot more accessible. + +00:13:25.150 --> 00:13:25.300 +Yeah. + +00:13:26.280 --> 00:13:34.200 +Going back to that sixth cell you entered on your REPL and editing it, that is not a way to draw beginners in. + +00:13:35.420 --> 00:13:36.520 +No, definitely not. + +00:13:37.040 --> 00:13:38.420 +Up arrow key six times, no. + +00:13:38.840 --> 00:13:44.340 +Yeah, 2012, coincidentally, is when I started college as an undergrad. + +00:13:44.980 --> 00:13:51.780 +So I had been using, I kind of grew up, as long as I used Python, I think I probably was using Jupyter Notebooks. + +00:13:52.460 --> 00:14:03.100 +And I even started using Google CoLab before it was made public, because when I was interning at Google, and I think the year I started working at Google full-time is when they made it public to the world. + +00:14:03.500 --> 00:14:08.940 +So, yeah, notebooks have sort of shaped the way that I think and interact with code quite a bit. + +00:14:09.440 --> 00:14:10.140 +Yeah, absolutely. + +00:14:11.300 --> 00:14:14.100 +So maybe you could give us some perspective. + +00:14:14.230 --> 00:14:16.540 +It sounds like you were there, like, right at the right time. + +00:14:16.810 --> 00:14:23.120 +You know, what was the goal and sort of the job of Jupyter Notebooks and those kinds of things then? + +00:14:23.780 --> 00:14:25.560 +And how has that changed now? + +00:14:25.700 --> 00:14:29.020 +And maybe how does it still solve that problem or not solve it so much? + +00:14:29.360 --> 00:14:32.780 +Yeah, I think, so stepping back. + +00:14:32.980 --> 00:14:41.300 +So you mentioned like how many people who might not have traditionally, you know, worked with code sort of came into it because of Jupyter Notebooks. + +00:14:41.300 --> 00:14:43.900 +And I think that's like really, I think you're onto something there. + +00:14:43.900 --> 00:14:44.520 +I think that's correct. + +00:14:44.710 --> 00:14:55.100 +And I think the special thing about Notebooks is that they like mix code and visuals and like narrative text in like this interactive programming environment. + +00:14:55.560 --> 00:15:00.520 +And that interactivity in particular, I think is really important for anyone who works with data. + +00:15:01.160 --> 00:15:13.780 +It's where you got to like first like, you know, run some queries against your data to see the general shape of it, you know, run some transformations to see like, you know, basically like explore your data, like hold it in your hands. + +00:15:14.030 --> 00:15:23.020 +Right. And like, you know, REPLs before and then Jupyter Notebooks in particular lets you do that in a way that you just can't easily do in a script because it holds the variables in memory. + +00:15:23.660 --> 00:15:33.640 +And so I think the role that it played was that it enabled data scientists or computational scientists, biologists, astrophysicists, all these different types of people who work with data. + +00:15:34.980 --> 00:15:45.120 +It let them rapidly experiment with code and models and importantly, get some kind of like artifact out of it. + +00:15:45.180 --> 00:15:52.940 +And in particular, this was like a static HTML type artifact, right, where you have some text documenting what experiments you're running. + +00:15:53.380 --> 00:15:58.300 +you have Python code, and then you have like Matplotlib plots and other artifacts. + +00:15:59.660 --> 00:16:06.860 +And I think for the sciences in particular early on, that was super important and transformative. + +00:16:07.380 --> 00:16:09.960 +And we were talking about other computational notebook environments. + +00:16:10.110 --> 00:16:17.000 +And it's interesting to mention this because things like this existed sort of before, like Mathematica's workbooks, I think they + +00:16:17.000 --> 00:16:17.380 +called them. + +00:16:17.410 --> 00:16:17.760 +I might have + +00:16:17.760 --> 00:16:18.420 +done that wrong. + +00:16:19.220 --> 00:16:25.760 +But I think just like the open source nature of Python and the accessibility of it just like made it way easier to get started. + +00:16:26.140 --> 00:16:36.760 +And also the fact that it was like browser based, right, so that like you can easily share like these, the HTML artifacts that come out, I think was really transformative. + +00:16:37.620 --> 00:16:40.880 +And insofar as like what they're used for today, they're used for a lot. + +00:16:41.080 --> 00:16:44.480 +It's kind of remarkable how much they're used. + +00:16:44.640 --> 00:16:53.080 +Like I remember seeing a statistic for Google CoLab alone, which is just one particular hosted version of a Jupyter notebook. + +00:16:53.840 --> 00:16:57.800 +And it was two years ago, and they said something about like having 10 million monthly active users. + +00:16:58.600 --> 00:17:03.660 +And so the number of people who use Jupyter on a monthly basis is surely larger than that. + +00:17:04.310 --> 00:17:04.680 +Oh, yeah. + +00:17:05.400 --> 00:17:14.420 +And they're used for the sciences, but they're also used for like things where you want, like the traditional rigor and reproducibility of software engineering. + +00:17:14.620 --> 00:17:19.819 +So they're increasingly used for like data pipelines and like ETL jobs. + +00:17:19.819 --> 00:17:28.360 +And like, so anyone who's in Databricks, for example, runs like, they call them workflows, Databricks workflows, through essentially Jupyter Notebooks. + +00:17:29.740 --> 00:17:31.480 +They're used for training machine learning models. + +00:17:31.820 --> 00:17:34.620 +Like I've trained many models in a Jupyter Notebook. + +00:17:35.020 --> 00:17:36.880 +I've developed algorithms in Jupyter Notebooks. + +00:17:37.160 --> 00:17:41.420 +I've produced scientific figures that go straight into my paper from a Jupyter notebook. + +00:17:42.880 --> 00:17:50.900 +And the reason people do this is because, again, the interactivity and the visuals that come out is very liberating compared to using a script. + +00:17:51.500 --> 00:17:56.400 +I think there's also a developing understanding that happens from them. + +00:17:56.600 --> 00:18:01.680 +Because when you're writing just a straight program, you're like, okay, this step, this step, this step, this step, and then we get the answer. + +00:18:02.180 --> 00:18:04.920 +Whereas notebooks, it's like step one, step two. + +00:18:05.180 --> 00:18:06.000 +Let me look at it. + +00:18:06.080 --> 00:18:07.100 +oh, maybe that's different. + +00:18:07.330 --> 00:18:08.140 +Let me go back and change. + +00:18:08.600 --> 00:18:11.780 +It's more iterative from an exploratory perspective. + +00:18:12.340 --> 00:18:13.800 +Yeah, I think that's exactly right. + +00:18:15.040 --> 00:18:20.280 +Because you have that, when you're writing a software system, you kind of know the state of the system. + +00:18:20.450 --> 00:18:22.300 +You know what you want to change it to. + +00:18:22.500 --> 00:18:26.140 +But when you're working with data, there's that unknown, what will my algorithm do to my data? + +00:18:26.460 --> 00:18:26.540 +You + +00:18:26.540 --> 00:18:26.840 +just have + +00:18:26.840 --> 00:18:27.100 +to run + +00:18:27.100 --> 00:18:27.680 +it to find out. + +00:18:27.960 --> 00:18:28.520 +Yeah, absolutely. + +00:18:28.810 --> 00:18:32.440 +And graphs and tables and iterative stuff like that is really nice. + +00:18:33.040 --> 00:18:37.040 +So why create something instead of Jupyter? + +00:18:37.900 --> 00:18:39.800 +Why is Jupyter not enough for the world? + +00:18:40.240 --> 00:18:40.580 +Yeah. + +00:18:41.750 --> 00:18:42.520 +I think there's a... + +00:18:42.780 --> 00:18:48.640 +So just, you know, you're the creator of Marimo, along with the rest of the team and so on. + +00:18:48.980 --> 00:18:52.720 +But yeah, it's like, why create this thing when Jupyter exists? + +00:18:53.320 --> 00:18:53.540 +Yeah. + +00:18:53.630 --> 00:18:57.000 +So I think there's quite a few reasons. + +00:18:58.179 --> 00:19:06.660 +So Jupyter notebooks are like, I guess, It's more like Jupyter Notebooks powered by the IPython kernel, just to put a finger on it. + +00:19:07.600 --> 00:19:08.520 +Right, because it could be different. + +00:19:08.520 --> 00:19:12.280 +It could be like a C++ kernel or.NET kernel or whatever. + +00:19:12.530 --> 00:19:13.400 +Yeah, exactly. + +00:19:13.770 --> 00:19:27.080 +But this particular form of notebooks where you get a page and then in the front end shows you a bunch of cells that you execute one at a time imperatively, even though you have a sequence of cells, it still is essentially a REPL. + +00:19:27.260 --> 00:19:38.020 +And like it's the onus falls on the developer, the scientist, whoever's using it to like run each cell on their own and like maintain the state of their kernel. + +00:19:38.260 --> 00:19:43.220 +So like you run a cell, say you have three cells, you know, you decide to go rerun the second cell. + +00:19:43.640 --> 00:19:47.100 +You have to remember to rerun whatever cell depends on it. + +00:19:47.220 --> 00:19:48.000 +Maybe it's the third cell. + +00:19:48.120 --> 00:19:50.020 +Maybe it's the first cell because you wrote things out of order. + +00:19:50.420 --> 00:19:57.040 +And so you can easily get into a state with Jupyter Notebooks where your code on the page doesn't match the outputs that were + +00:19:57.040 --> 00:19:57.640 +created. + +00:19:58.000 --> 00:20:03.240 +Yeah. So if you were to like basically go to the menu option, say rerun all cells, you would get different answers. + +00:20:03.820 --> 00:20:11.320 +Exactly. And this is like it's actually been studied and like it happens a kind of a shockingly large amount of times that like. + +00:20:11.440 --> 00:20:19.560 +So there's one study in 2019 by Pimentel, I'm going to probably pronounce his name wrong, but by I think four professors. + +00:20:20.260 --> 00:20:21.740 +And they studied a bunch of notebooks on GitHub. + +00:20:22.070 --> 00:20:27.980 +And they found that only like a quarter of the notebooks that were on GitHub, they were able to run it all. + +00:20:28.330 --> 00:20:38.060 +And then like either 4% of those or 4% of all of them, only 4% of them like when they ran recreated the same results that were serialized in the notebook. + +00:20:38.680 --> 00:20:44.720 +Meaning like maybe people ran sales out of order when they originally created the notebook and then committed it. + +00:20:45.260 --> 00:20:52.140 +Or maybe they didn't like create a requirements.txt that faithfully captured the package environment they used. + +00:20:53.040 --> 00:21:06.680 +So there's like this recognition that Jupyter Notebooks suffer from a reproducibility crisis, which is rather unfortunate for a tool that is used for science and data science and data engineering, like things where reproducibility is paramount. + +00:21:07.120 --> 00:21:08.100 +And so that was one issue. + +00:21:08.580 --> 00:21:13.220 +Reproducibility was like one main thing that sort of we wanted to solve for with Marima notebooks. + +00:21:13.960 --> 00:21:24.560 +And the other thing I think that we alluded to earlier is that Jupyter notebooks are stored as like a JSON file where like the outputs, like the plots, et cetera, are serialized as these binary strings. + +00:21:25.420 --> 00:21:33.700 +And like the, you know, the one downside of that is that you can't really use these things like you can use like traditional software, at least not without jumping through extra hoops. + +00:21:33.920 --> 00:21:42.620 +So like you make a small change to your Jupyter notebook, you get a gigantic git diff because, you know, the binary representation of some object changed by a large amount. + +00:21:43.400 --> 00:21:51.120 +You want to reuse the Python code in that Jupyter notebook when you have to run it through like a Jupytext or some other thing to strip it out. + +00:21:51.190 --> 00:22:00.360 +And then so you oftentimes you get this like situation where people just like end up copy pasting a bunch of things across a bunch of notebooks and it quickly gets really messy. + +00:22:00.440 --> 00:22:09.880 +And that was another thing we wanted to solve for, making notebooks like Git-friendly and interoperable with other Python tooling like Git, pytest, etc. + +00:22:10.280 --> 00:22:18.580 +Yeah, there's some things that will run them kind of function module-like, but yeah, it's not really the intended use case, is it? + +00:22:18.780 --> 00:22:33.180 +Yeah, it's not, but people want to, which is, I think, like the, it's funny because like the intended use case, I think, of like traditional notebooks like Jupyter was like interactive REPL, explore data, produce like simple narrative text. + +00:22:33.580 --> 00:22:41.580 +But then like they just got used for so much more than that in situations where reproducibility, reusability is like really important. + +00:22:41.880 --> 00:22:44.480 +And I think that that's what we're trying to solve for. + +00:22:45.900 --> 00:22:49.320 +This portion of Talk Python to Me is brought to you by Worth Recruiting. + +00:22:49.840 --> 00:22:52.400 +Are you tired of applying for jobs and never hearing back? + +00:22:53.060 --> 00:23:01.560 +Have you been getting the runaround or having trouble making it past the AI resume screeners that act as the new gatekeepers for your next level Python job? + +00:23:01.840 --> 00:23:03.560 +You should reach out to Worth Recruiting. + +00:23:04.260 --> 00:23:08.540 +Worth Recruiting specializes in placing senior level Python developers and data scientists. + +00:23:09.300 --> 00:23:15.260 +They work directly with hiring managers at startups, helping them grow their software engineering and data science teams. + +00:23:16.060 --> 00:23:49.820 +with Worth it's not just connecting you with the company it will guide you through the interview process and help make sure you're ready with their detailed preparation approach they can even coach you on salary negotiations and other important decision making processes so if you're ready to see what new opportunities are out there for you reach out to Worth Recruiting let them be your partner and specialist to find the right Python developer or data scientist position for you fill out their short contact form at talkpython.fm/worth. It only takes a minute. That's talkpython.fm/worth. + +00:23:50.240 --> 00:24:15.660 +The link is in your podcast player's show notes. Thank you to Worth Recruiting for supporting the show. I was just wondering out loud, you know, that study. I know it was not your study, so I'm not asking you for answers. Just, just what do you think? If it's that low of a percentage of reproducibility because people fell down on their like rigorous software engineering, for example, like no requirements file or hard-coded pass instead of relative pass or whatever. + +00:24:16.050 --> 00:24:27.080 +If it's that number for notebooks, I wonder about just plain Python files that are also meant to act to solve like science problems and stuff that just didn't happen to be in notebooks. + +00:24:27.360 --> 00:24:28.420 +I wonder if they're any better. + +00:24:28.900 --> 00:24:29.160 +Yeah, + +00:24:29.360 --> 00:24:30.640 +it's a good question. + +00:24:31.480 --> 00:24:32.720 +I think they will be better. + +00:24:33.110 --> 00:24:36.380 +I can just give an anecdote to why I think they would be better. + +00:24:36.710 --> 00:24:48.160 +And so one example, and this happened to me many times, But like one particular example is like when I was working on my PhD thesis, which was about vector embeddings, it was me and a couple of co-authors. + +00:24:48.720 --> 00:24:57.060 +And, you know, both of us did our, like created the examples for this embedding algorithm in like Jupyter Notebooks. + +00:24:57.130 --> 00:24:59.240 +And these notebooks produced a bunch of plots, et cetera. + +00:25:00.100 --> 00:25:04.500 +And so at the end of the, once the thesis was written, I was putting up all our code on GitHub. + +00:25:04.710 --> 00:25:09.080 +And so I asked one of my co-authors, like, hey, like, can you share your notebooks with me? + +00:25:09.310 --> 00:25:10.340 +And this co-author is amazing. + +00:25:10.520 --> 00:25:13.660 +He's really smart, but not necessarily trained as a software engineer. + +00:25:14.320 --> 00:25:16.380 +And he shares his Jupyter notebook with me. + +00:25:16.860 --> 00:25:17.500 +And I run it. + +00:25:17.940 --> 00:25:19.160 +It doesn't work at all. + +00:25:19.220 --> 00:25:22.240 +And the reason it doesn't work at all is that there's like multiple cells. + +00:25:22.320 --> 00:25:28.780 +And there's like this branching path where like it's clear that like he ran like, you know, you see the execution order serialized in the notebook. + +00:25:29.300 --> 00:25:29.760 +And it's not like + +00:25:29.760 --> 00:25:29.940 +one, + +00:25:30.120 --> 00:25:30.880 +two, three, four. + +00:25:31.020 --> 00:25:32.840 +It's like one, two, seven, four. + +00:25:33.340 --> 00:25:33.600 +It's like + +00:25:33.600 --> 00:25:34.520 +these cells were + +00:25:34.520 --> 00:25:35.960 +ran in this path dependent way. + +00:25:36.260 --> 00:25:38.980 +And then also it's like there's like three cells in the middle. + +00:25:39.420 --> 00:25:40.360 +He ran just one of them. + +00:25:40.600 --> 00:25:41.960 +I'm not exactly sure which one. + +00:25:42.560 --> 00:25:46.420 +And like, whereas if you had a Python script, there's only one way to, I mean, sure, you can have + +00:25:46.420 --> 00:25:46.760 +data files. + +00:25:46.760 --> 00:25:47.280 +That's a good point. + +00:25:47.620 --> 00:25:48.980 +You do not really get a choice. + +00:25:49.660 --> 00:25:50.100 +It runs + +00:25:50.100 --> 00:25:50.600 +top to bottom. + +00:25:51.140 --> 00:25:51.380 +Exactly. + +00:25:51.500 --> 00:25:51.700 +Depending + +00:25:51.700 --> 00:25:52.380 +on the data. + +00:25:52.530 --> 00:25:52.620 +Okay. + +00:25:53.140 --> 00:25:53.880 +So yeah, + +00:25:53.950 --> 00:26:00.320 +that is something that's really, it's just very counter to the concept of reproducibility is that you + +00:26:00.320 --> 00:26:00.480 +can + +00:26:00.480 --> 00:26:10.240 +run them in any order and there's not, you know, I feel like Jupyter Notebooks should almost have like a red bar to the cross, like an out of order bar. + +00:26:10.700 --> 00:26:13.400 +warning across the top if it's not top to bottom. + +00:26:14.010 --> 00:26:15.060 +It doesn't have to be 1, 2, 3. + +00:26:15.090 --> 00:26:17.240 +It could be 1, 5, 6, 9. + +00:26:17.560 --> 00:26:21.560 +But it should be a monotonic increase in function as you + +00:26:21.560 --> 00:26:22.580 +do the numbers top + +00:26:22.580 --> 00:26:23.000 +to bottom. + +00:26:24.300 --> 00:26:29.100 +Otherwise, it should be like a big warning or like a weird, like some kind of indicator, like, hey, you're in the danger zone. + +00:26:29.230 --> 00:26:29.760 +You know what I mean? + +00:26:30.080 --> 00:26:40.220 +Yeah, and there's actually another thing that actually has happened to me way more often than I should care to admit while working with Jupyter Notebooks is that it's not just the order. + +00:26:40.280 --> 00:27:07.680 +you run the cells it's also like what happens if you delete a cell because when you delete a cell you no longer so say a cell declares a variable x and you deleted that cell you're like i don't want x anymore but you may forget that just by deleting the cell um you're not removing the variables from memory and so like x still exists and so then you're running other cells that depend on x everything's fine uh but then you come back and you run your notebook and everything's broken and you don't You have no idea. + +00:27:07.820 --> 00:27:10.140 +The first time this happened to me, it took me forever to debug. + +00:27:11.039 --> 00:27:11.440 +I + +00:27:11.440 --> 00:27:12.060 +can imagine + +00:27:12.060 --> 00:27:12.820 +that is so rough. + +00:27:13.380 --> 00:27:13.520 +Yeah. + +00:27:13.980 --> 00:27:17.020 +Later on, you realize, okay, this is a pattern that happens. + +00:27:17.260 --> 00:27:22.440 +But it's like, yeah, so that's another thing that we also wanted to solve for in Marimo notebooks. + +00:27:23.200 --> 00:27:23.440 +Awesome. + +00:27:23.860 --> 00:27:23.960 +All right. + +00:27:23.960 --> 00:27:24.780 +Well, tell us about Marimo. + +00:27:26.560 --> 00:27:26.720 +Okay. + +00:27:27.080 --> 00:27:27.740 +So let's see. + +00:27:28.040 --> 00:27:31.060 +So Marimo is, it's an open source notebook for Python. + +00:27:31.840 --> 00:27:37.780 +The main thing that's different between Marimo and Jupyter is that Marimo is reactive. + +00:27:38.550 --> 00:27:55.000 +And what that means is that unlike a Jupyter notebook where you can run cells in any arbitrary order you like, in Marimo, when you run one cell, it knows what other cells need to be run in order to make sure that your code and outputs are synchronized. + +00:27:55.480 --> 00:27:56.780 +So it's kind of like Excel, right? + +00:27:56.780 --> 00:27:58.020 +So you have two cells. + +00:27:58.320 --> 00:27:59.060 +One declares X. + +00:27:59.440 --> 00:28:00.460 +Another one references X. + +00:28:00.800 --> 00:28:09.200 +you run the cell that declares X or defines X, all other cells that reference X will automatically run or they'll be marked as stale. + +00:28:09.320 --> 00:28:12.520 +You can configure it if you're scared of automatic execution. + +00:28:12.900 --> 00:28:18.020 +But the point is that it manages your variables for you so + +00:28:18.020 --> 00:28:18.480 +that you + +00:28:18.480 --> 00:28:22.560 +don't have to worry about outputs becoming decohered from your code. + +00:28:22.760 --> 00:28:33.460 +And similarly, if you delete a cell, the variable will be scrubbed from memory and then cells that are dependent on it will either be marked as stale a big warning or automatically run and invalidate it. + +00:28:34.120 --> 00:28:37.560 +So that reactivity provides some amount of reproducibility. + +00:28:38.720 --> 00:28:44.460 +And then it also, some of the key features, and then we can dive into each of them, is an additional reactivity. + +00:28:44.980 --> 00:28:49.360 +Marima notebooks are stored as pure Python files, which makes them easy to version with Git. + +00:28:50.300 --> 00:28:57.540 +And then we've also made it easy to run Marima notebooks as Python scripts and then share them as interactive web apps with little UI elements. + +00:28:58.300 --> 00:28:59.820 +Yeah, I'm excited to talk about that. + +00:29:00.040 --> 00:29:00.780 +Some cool stuff there. + +00:29:01.340 --> 00:29:09.300 +Okay, so when you create a variable, if you say like x equals one, it's not just a pi long pointer in memory, right? + +00:29:09.460 --> 00:29:21.060 +It's something that kind of looks at the read and write that get in the set of it and creates like a relationship where it says if you were to push a change into it, it says, okay, I have been changed. + +00:29:21.400 --> 00:29:23.060 +Who else is interested in reading from me? + +00:29:23.120 --> 00:29:24.760 +And it can sort of work that out, right? + +00:29:25.040 --> 00:29:25.400 +How does that work? + +00:29:25.580 --> 00:29:26.100 +Actually, + +00:29:26.240 --> 00:29:28.940 +it's not really implemented in that way. + +00:29:29.820 --> 00:29:29.900 +Okay. + +00:29:30.500 --> 00:29:31.260 +So, okay. + +00:29:31.460 --> 00:29:32.040 +So let's see. + +00:29:32.300 --> 00:29:37.400 +Yeah, this is interesting to dive into because there's like two ways that you can like get at reactivity. + +00:29:37.560 --> 00:29:40.900 +Like so in Marima, what we actually do is we do static analysis. + +00:29:41.560 --> 00:29:41.740 +So we + +00:29:41.740 --> 00:29:43.080 +look for + +00:29:43.080 --> 00:29:46.080 +definitions and references on like of global variables. + +00:29:46.620 --> 00:29:48.320 +And then so we just make a graph out of that. + +00:29:48.320 --> 00:29:53.400 +So for every cell, we look at the, I guess, the loads in the ASTs and then like + +00:29:53.400 --> 00:29:56.820 +the assignments. + +00:29:57.780 --> 00:30:03.080 +And so we can see statically who declares what and who reads what, but who is a cell. + +00:30:04.600 --> 00:30:09.680 +And so that makes it like performant and also predictable of like how your notebook is going to run. + +00:30:10.980 --> 00:30:13.900 +A alternative, I think what you were getting at was like runtime tracing. + +00:30:14.540 --> 00:30:20.060 +More like a JavaScript front end sort of deal, like a view binding model binding type thing or something like that. + +00:30:20.420 --> 00:30:27.020 +Yeah. So yeah, we, everything we do is static, is done with static analysis. + +00:30:27.120 --> 00:30:39.720 +this there was another project called ipyflow which was like a reactive kernel for jupiter it's still around um they took like the runtime tracing approach of like checking on every assignment like okay + +00:30:39.720 --> 00:30:40.520 +okay + +00:30:40.520 --> 00:30:58.320 +there's a read and now where is that where is that object in memory and then running cells that depend on it um i think in practice you know i was talking to steven mackie the author of that article of that extension and then also like based on some work that i saw at Google TensorFlow, that's really hard to get 100% right. + +00:30:58.350 --> 00:31:00.460 +It's basically impossible to get 100% right. + +00:31:00.880 --> 00:31:05.260 +So you will miss some low references and definitions. + +00:31:06.100 --> 00:31:11.840 +And so there's this weird usability cliff as a user where when you run a cell, you're not sure what else will run. + +00:31:12.210 --> 00:31:17.760 +Whereas if you do it based on static analysis, you can give guarantees on what will run and what won't. + +00:31:18.210 --> 00:31:23.580 +Yeah, I guess the static analysis is a good choice because there's only so many cells. + +00:31:23.700 --> 00:31:31.840 +You don't have to track it down to like, well, within the DOM and all the JavaScript objects, we're linking all these together and they can do whatever they want from all these different angles. + +00:31:32.040 --> 00:31:35.620 +It's just like, when this cell runs, what other cells do we need to think about? + +00:31:35.800 --> 00:31:37.460 +So it's kind of like, what does this create? + +00:31:38.060 --> 00:31:38.860 +What does it change? + +00:31:39.240 --> 00:31:40.760 +And then push that along, right? + +00:31:40.860 --> 00:31:42.040 +So it's a little more constrained. + +00:31:42.500 --> 00:31:43.080 +Exactly, yeah. + +00:31:43.240 --> 00:31:45.380 +So basically, yeah, that's exactly right. + +00:31:45.480 --> 00:31:49.740 +And so it's a data flow graph where the data flowing on the edges is the variables across cells. + +00:31:51.420 --> 00:31:52.320 +Yeah, I don't know. + +00:31:52.360 --> 00:31:58.060 +I think I've been thinking about Dataflow Grass for a long time, ever since TensorFlow, doing my PhD too when I worked on CVXPY. + +00:31:58.240 --> 00:32:02.460 +So it's just a thing that I enjoy thinking about and working on. + +00:32:02.620 --> 00:32:11.440 +Yeah, well, what about your experience working on TensorFlow and at Google and at Stanford and stuff that sort of influenced this project? + +00:32:11.940 --> 00:32:13.960 +Yeah, a couple of things. + +00:32:14.780 --> 00:32:27.840 +So definitely a big thing that influenced this project was working as a scientist at Stanford where like I and my colleagues use Jupyter Notebooks like on almost a daily basis because they want to see our data while we worked on it. + +00:32:29.020 --> 00:32:37.300 +And so really valued the iterative programming environment, really didn't like all the bugs that we kept on running into, which are + +00:32:37.300 --> 00:32:37.800 +like kind of our + +00:32:37.800 --> 00:32:38.320 +fault, right? + +00:32:38.460 --> 00:32:40.800 +Because of like, oh, we forget to run a cell, et cetera. + +00:32:42.280 --> 00:32:48.400 +Didn't like that it was not easy to reuse the code in a notebook in just other Python modules. + +00:32:49.140 --> 00:32:55.120 +Didn't like that I couldn't share my artifacts in a meaningful way with my PhD advisor who can't run Python notebooks. + +00:32:55.480 --> 00:32:58.220 +I couldn't make a little app to showcase my + +00:32:58.220 --> 00:32:58.920 +research project. + +00:32:59.860 --> 00:33:15.640 +It's so interesting how PhD advisors and just professors in general, how they're either embracing or their inability to embrace or their prejudices for or against the technology so influence science, the people in the research teams, everything. + +00:33:15.860 --> 00:33:21.840 +It's like, well, we'd like to use this, but the principal author really doesn't like it, doesn't want to run it, so we're not using that. + +00:33:21.910 --> 00:33:22.240 +Or, you + +00:33:22.240 --> 00:33:22.700 +know, just + +00:33:22.700 --> 00:33:24.240 +these little edge cases. + +00:33:24.290 --> 00:33:26.480 +Like, I was made to learn Fortran in college. + +00:33:26.660 --> 00:33:27.720 +I'm still sour about it. + +00:33:29.800 --> 00:33:30.220 +That's funny. + +00:33:30.420 --> 00:33:31.080 +Yeah, no, it's true. + +00:33:31.280 --> 00:33:32.860 +Yeah, it does have a big effect. + +00:33:33.380 --> 00:33:37.880 +And then at Google, I guess the things that influenced – so we used Notebooks 2 there. + +00:33:39.340 --> 00:33:49.820 +Since I was doing the systems engineering work at Google, I personally didn't use them too much, although I did use like Google Colab for like training, like creating training courses and stuff for, for like other engineers and for demos. + +00:33:50.440 --> 00:33:55.520 +But I guess the part of Google that influenced it was just like thinking about data flow graphs for a + +00:33:55.520 --> 00:33:56.780 +year. And you + +00:33:56.780 --> 00:34:13.220 +know, I, we were, there was like a couple of parable projects, like one that was using runtime tracing for like figuring out like how to make a DAG. And it was just, I don't know, kind of traumatic just because you like you miss things and the user experience is just like it kind of falls over. + +00:34:13.340 --> 00:34:18.460 +And so I guess the part from there just made me not want to use runtime + +00:34:18.460 --> 00:34:18.899 +tracing, + +00:34:19.080 --> 00:34:20.960 +made me embrace static analysis for this. + +00:34:22.340 --> 00:34:22.520 +Yeah. + +00:34:22.899 --> 00:34:23.740 +Yeah, it makes a lot of sense. + +00:34:24.159 --> 00:34:24.520 +All right. + +00:34:25.340 --> 00:34:27.419 +Well, let's talk some features here. + +00:34:27.840 --> 00:34:37.240 +What are the, I guess probably the premier feature is the reactivity, in which case it doesn't matter which cell you run, they're never going to get out of date, right? + +00:34:37.560 --> 00:34:38.679 +Yeah, that's correct. + +00:34:38.980 --> 00:34:44.379 +So the reactivity, in other words, the data flow graph behind it is the premier feature. + +00:34:45.159 --> 00:34:47.820 +And it lets you reason about your code locally, right, which is really nice. + +00:34:48.060 --> 00:34:49.639 +Like you can just look at your one cell. + +00:34:49.970 --> 00:34:54.179 +You don't really have to worry about, you know, you know what variables it reads and what it defines. + +00:34:54.669 --> 00:34:57.800 +And you don't have to worry about, oh, but what cells do I need to run before this? + +00:34:58.780 --> 00:35:06.960 +So not only does it like make sure your code and outputs are in sync, it also like makes it really like fun and useful. + +00:35:08.420 --> 00:35:11.320 +to really rapidly experiment with ideas. + +00:35:11.960 --> 00:35:25.880 +As long as your cells aren't too expensive, if you enable automatic execution, you can change a parameter, run a particular cell, and it'll run only what's required to update subsequent outputs. + +00:35:27.820 --> 00:35:39.600 +And then another consequence of reactivity that I think is really neat and our users really like is it makes it far easier to use interactive UI widgets than it is in like a Jupyter notebook. + +00:35:40.540 --> 00:35:46.820 +And the way that this works is that, so Marimo is like both a notebook and also a library. + +00:35:47.140 --> 00:35:49.740 +So you can import Marimo as Mo into your notebook. + +00:35:50.220 --> 00:36:11.220 +And when you do this, you get access to a bunch of things, including a bunch of different UI widgets, ranging from the simple ones like sliders, drop-down menus, to like cooler ones like interactive selectable charts and data frame transformers that automatically generate the Python code needed for your transformation. + +00:36:12.180 --> 00:36:30.920 +And the way that reactivity comes into here is that because we know when you create a UI element, as long as you bind it to a global variable, just say my slider equals mo.ui.slider, then when you interact with the slider anywhere on the screen, well, we can just say, okay, that slider is bound to X. + +00:36:31.130 --> 00:36:33.260 +We just need to run all other cells that depend on X. + +00:36:33.840 --> 00:36:41.140 +all of a sudden you have really nice interactivity, interactive elements controlling your code execution without ever having to write a callback. + +00:36:42.380 --> 00:36:44.460 +And so I think people find that really liberating too. + +00:36:44.900 --> 00:36:47.840 +So you don't really have to hit backspace, change + +00:36:47.840 --> 00:36:48.740 +a + +00:36:48.740 --> 00:36:51.720 +character value for your variable, shift, enter, shift, enter, shift, enter. + +00:36:52.980 --> 00:36:55.860 +Just change a number and everything automatically recalculates. + +00:36:56.200 --> 00:36:56.460 +Yeah. + +00:36:56.820 --> 00:37:02.060 +In this dependency flow, is it possible to have a cyclical graph? + +00:37:02.460 --> 00:37:09.860 +Like if I did this in Jupyter, I could go down to the third cell and define X and then go back up and then use X. + +00:37:09.950 --> 00:37:12.660 +If I run it in the right order, it won't know that there's a problem. + +00:37:13.720 --> 00:37:16.060 +Imagine it's only possible to go one way, right? + +00:37:16.060 --> 00:37:17.560 +You can't get into a weird cycle. + +00:37:19.040 --> 00:37:19.100 +Yeah. + +00:37:19.370 --> 00:37:21.740 +So we do give people escape patches. + +00:37:22.060 --> 00:37:26.860 +We ask them, please don't use it unless you really know what you're doing and you really want to do this. + +00:37:27.410 --> 00:37:39.980 +Because in most cases we find, especially for like if your goal is to just like work with data data scientist, data engineer scientist. We feel you don't need that cycles. You don't need cycles. + +00:37:40.480 --> 00:38:07.080 +But if you're making an app, and I guess I should mention Marimo lets you run any notebook as well a notebook, but also from the command line, you can serve your notebook as a web app, like similar to Streamlit if you've seen that. In those cases, sometimes you do want state basically, right? Like cyclic references. So by default, Marimo actually checks for cycles. If you have a cycle across cells, politely tells you, hey, that's not really cool in Marimo. + +00:38:07.780 --> 00:38:08.560 +Please break your cycle. + +00:38:08.920 --> 00:38:10.380 +Here's some suggestions how to do that. + +00:38:11.180 --> 00:38:20.280 +But if you're really insistent, then you can go to our docs and learn about the state object that we have that does let you get into some runtime-based cell execution. + +00:38:20.620 --> 00:38:26.240 +You can update a state object and kind of in a React from 10D way, it'll run dependence of that state. + +00:38:26.480 --> 00:38:27.420 +Okay, interesting. + +00:38:27.710 --> 00:38:28.880 +Yeah, I guess that makes sense for apps. + +00:38:28.880 --> 00:38:31.660 +You definitely need some sort of global state there. + +00:38:32.460 --> 00:38:34.680 +Yeah, I + +00:38:34.680 --> 00:38:41.280 +think we've seen that sometimes our users will reach for state in call VIX just because that's what they're used to. + +00:38:41.400 --> 00:38:44.780 +And that's what they maybe were required to use in Jupyter, et cetera. + +00:38:45.260 --> 00:38:48.800 +So we're trying to tell them, hey, in most cases, you actually don't need this. + +00:38:49.160 --> 00:38:49.780 +Yeah, very cool. + +00:38:50.620 --> 00:38:55.600 +So when you create a Jupyter notebook, you don't really create Python files. + +00:38:56.100 --> 00:39:05.460 +you create the notebook files, and those notebook files are JSON with both embedded cell execution bits, the code, but also the answers that go below. + +00:39:05.680 --> 00:39:16.080 +That's why if you go to GitHub and you look at a notebook, you can instantly see what could be super expensive computation, but pictures and stuff there because it's residual in the artifact, right? + +00:39:16.380 --> 00:39:16.500 +Yeah. + +00:39:17.760 --> 00:39:18.780 +If you guys do Python + +00:39:18.780 --> 00:39:25.960 +files, do you have a way to save that state in a sort of presentation style or something like that? + +00:39:26.170 --> 00:39:26.460 +You know what I mean? + +00:39:26.540 --> 00:39:28.880 +Is there like an associated state file? + +00:39:29.840 --> 00:39:30.020 +Yeah, + +00:39:30.130 --> 00:39:31.240 +so that's a really good question. + +00:39:31.540 --> 00:39:35.200 +And the answer is yes, but that is the biggest trade-off, I think, of our file format. + +00:39:35.430 --> 00:39:47.780 +So like a surprising number of people actually have come to us and told us, I didn't expect this, but they tell us the reason we decided to try Marima Notebooks is because you say it is versionable with Git. + +00:39:47.930 --> 00:39:52.240 +Like that's like, they're like, that's the one reason I, especially like people in software and industry. + +00:39:52.800 --> 00:39:54.120 +And then they stay for all the other things. + +00:39:55.180 --> 00:40:00.440 +So that's what the pure Python file format allows, in addition to modularity, running it as a script. + +00:40:02.480 --> 00:40:05.840 +For seeing outputs, yeah, they're not saved in the Python file. + +00:40:05.970 --> 00:40:09.380 +So we have a couple of ways to get around that. + +00:40:09.650 --> 00:40:14.280 +So one, you can export any Marimo notebook to an IPython notebook file, actually. + +00:40:14.550 --> 00:40:14.980 +So you + +00:40:14.980 --> 00:40:17.860 +can set up this, we have a feature, automatic snapshotting. + +00:40:18.360 --> 00:40:26.820 +You can automatically snapshot your notebook to like a parallel IPIMB file that's stored in a subdirectory of your notebook folder. + +00:40:28.300 --> 00:40:30.920 +And so you can push that up or share that if you like. + +00:40:32.580 --> 00:40:45.280 +When working locally, though, we actually have this cool feature that my co-finder Miles recently built is that we actually – so when you're working, you're working with your notebook, the outputs are actually saved. + +00:40:45.520 --> 00:40:50.560 +The representation of the outputs are saved in this subdirectory, underscore underscore marimo. + +00:40:51.200 --> 00:40:59.900 +So that the next time you open the notebook, it just picks up those outputs and then like loads them into the browser so that you can see where you left off. + +00:41:01.160 --> 00:41:01.480 +So + +00:41:01.720 --> 00:41:03.900 +I see. So kind of like PyCache. + +00:41:04.360 --> 00:41:04.600 +Yeah. + +00:41:04.900 --> 00:41:09.080 +I would actually assume given the name, but I haven't actually looked into PyCache too much. + +00:41:09.320 --> 00:41:14.400 +Well, I mean, just in the sense that it's saved like right there in your project using that directory. + +00:41:14.620 --> 00:41:16.520 +Oh, yes, yes. You have a PyCache folder. Exactly. + +00:41:16.860 --> 00:41:16.980 +Yeah, + +00:41:17.020 --> 00:41:17.560 +yeah, yeah. Exactly. + +00:41:18.280 --> 00:41:18.980 +And we have one + +00:41:18.980 --> 00:41:32.300 +other feature that another contributor, his name is Dylan, has been developing, which I think is really cool, which is along those same lines, but based on Nick's style caching. + +00:41:32.800 --> 00:41:39.360 +So it'll actually save the Python objects themselves using a variety of different protocols. + +00:41:40.240 --> 00:41:47.280 +And because we have the DAG, he can actually guarantee consistency of the cache. + +00:41:47.720 --> 00:41:47.860 +But that + +00:41:47.860 --> 00:41:49.160 +means not only are your outputs + +00:41:49.160 --> 00:41:54.200 +automatically loaded, also the variables are loaded, so you can literally pick up where you left off. + +00:41:54.960 --> 00:42:01.100 +Yeah, it sounds like some sweet SQLite could be in action there instead of just flat files or whatever. + +00:42:01.500 --> 00:42:01.720 +Yeah. + +00:42:02.060 --> 00:42:02.600 +Yeah, very cool. + +00:42:02.970 --> 00:42:10.020 +So you talked about when the reactivity fires, it can sometimes be expensive to recompute the cells. + +00:42:10.420 --> 00:42:11.580 +Sometimes they're super simple. + +00:42:12.060 --> 00:42:15.160 +Sometimes they're trained in the machine learning model for two days. + +00:42:16.500 --> 00:42:16.740 +Yeah. + +00:42:17.300 --> 00:42:19.440 +Are there caching mechanisms? + +00:42:19.620 --> 00:42:33.140 +You know, in a Python script, we've got functools.lrucache, and then there's other things that are kind of cool, in-process caches, like PyMocha is kind of like the functool stuff, but way more flexible and so on. + +00:42:33.440 --> 00:42:43.420 +And you could put those onto functions that then would have really interesting caching characteristics, like, hey, if you're going to rerun it, but I've run it before with this other value, here's the same answer back, right? + +00:42:43.540 --> 00:42:49.680 +Is there a way to set up that kind of caching or performance memoization type stuff? + +00:42:50.100 --> 00:42:50.700 +Yeah, definitely. + +00:42:51.220 --> 00:42:52.980 +And that same contributor, Dylan. + +00:42:53.840 --> 00:42:57.140 +So he's implemented these and is continuing to build it out. + +00:42:57.820 --> 00:43:01.420 +But we have, I guess, two options. + +00:43:02.180 --> 00:43:10.080 +One is in-memory cache, which is basically the API is the same as functools.cache, but is designed to work in an iterative programming environment. + +00:43:10.140 --> 00:43:17.080 +If you use functools.cache naively, if a cell defining a function reruns, your cache gets busted. + +00:43:18.320 --> 00:43:24.720 +So modoc.cache is a little smarter, and using the DAG can know whether or not it needs to bust the cache. + +00:43:25.370 --> 00:43:27.240 +But to the user, it feels the same. + +00:43:28.340 --> 00:43:37.520 +And then we also have a persistent cache, which is the one that I was alluding to when I mentioned you can automatically pick up where you left off by loading objects from disk. + +00:43:37.940 --> 00:43:43.260 +And so you can put an entire cell in a context manager with MoDub persistent cache. + +00:43:44.160 --> 00:43:45.440 +It'll sort of do the right thing. + +00:43:46.420 --> 00:43:55.940 +We've been talking about experimenting with letting users opt into global cell-wide caching, just automatically memorize cells or just have a UI element + +00:43:55.940 --> 00:43:57.440 +to do something like that. + +00:43:57.580 --> 00:43:59.100 +But we haven't quite done that yet. + +00:43:59.380 --> 00:44:01.240 +Yeah, I mean, it sounds real tricky for Jupyter. + +00:44:01.660 --> 00:44:11.020 +But because you all know the inputs and the outputs, It's almost as if you could sort of drive a hidden LRU cache equivalent at the cell level. + +00:44:11.200 --> 00:44:11.660 +You know what I mean? + +00:44:12.079 --> 00:44:13.120 +That's exactly right. + +00:44:13.360 --> 00:44:13.460 +Yeah. + +00:44:14.580 --> 00:44:19.520 +I think, yeah, because we know the inputs and outputs, that's kind of how everything flows from a project. + +00:44:19.960 --> 00:44:23.920 +You can think about projects like if you turn a notebook into a data flow graph, what can you do? + +00:44:24.320 --> 00:44:27.260 +And yeah, caching is, smart caching is definitely one of them. + +00:44:27.480 --> 00:44:27.700 +Yeah. + +00:44:27.830 --> 00:44:31.760 +I mean, it's not a, I make a sense like, well, you know, the inputs, outputs, so it's fine. + +00:44:32.080 --> 00:44:39.420 +But, you know, you could have the same data frame, but you could have added a column to the data frame, and the LRU cache goes, is it the same pointer? + +00:44:39.640 --> 00:44:40.100 +It is. + +00:44:40.210 --> 00:44:40.740 +We're good to go. + +00:44:40.820 --> 00:44:41.760 +It's like, well, yes. + +00:44:41.960 --> 00:44:42.140 +However, + +00:44:42.460 --> 00:44:43.860 +it's + +00:44:43.860 --> 00:44:44.400 +not the same. + +00:44:44.570 --> 00:44:47.040 +You know, you almost got to, like, hash it or do something funky. + +00:44:47.740 --> 00:44:48.940 +Yeah, it's not perfect. + +00:44:48.940 --> 00:44:49.580 +Automatic sounds hard. + +00:44:49.910 --> 00:44:50.000 +No. + +00:44:50.280 --> 00:44:51.160 +Yeah, yeah, exactly. + +00:44:51.440 --> 00:44:53.380 +That's why we haven't quite enabled it. + +00:44:54.660 --> 00:44:55.580 +Yeah, it is hard. + +00:44:56.000 --> 00:44:56.180 +Yeah. + +00:44:56.800 --> 00:44:58.800 +Side effects, too, network requests, all these things. + +00:44:59.600 --> 00:44:59.900 +All right. + +00:45:00.240 --> 00:45:04.460 +I have three more topics I want to cover before we run out of time here. + +00:45:04.540 --> 00:45:07.240 +We may cover more, but three are required. + +00:45:07.600 --> 00:45:09.800 +First one, because this is the one I'm going to forget most likely. + +00:45:10.720 --> 00:45:20.180 +You were at Google, and then by way of going through a PhD, you now are not at Google creating this project, which is open source on GitHub. + +00:45:20.740 --> 00:45:24.300 +And I don't see a pricing page at the top, but I do see it for enterprises. + +00:45:24.720 --> 00:45:27.660 +Like, how is this sustaining itself? + +00:45:27.940 --> 00:45:28.940 +Like, what's the business model? + +00:45:29.780 --> 00:45:30.040 +So + +00:45:30.040 --> 00:45:41.860 +when we first started right after my PhD, so that was early 2022, we were actually lucky enough to get funding from a national lab associated with Stanford. + +00:45:42.040 --> 00:45:44.080 +So the Stanford Linear Accelerator. + +00:45:44.660 --> 00:45:45.500 +So I was talking with + +00:45:45.500 --> 00:45:46.960 +some scientists there + +00:45:46.960 --> 00:45:50.360 +and we were talking about, I knew I wanted to make this thing. + +00:45:50.660 --> 00:45:51.560 +And they were like, what are you up to? + +00:45:51.620 --> 00:45:57.680 +And I was like, oh, well, I want to make this like notebook thing that like, you know, fixes all notebooks inspired by Pluto JL. + +00:45:57.820 --> 00:45:58.920 +And they were like, that's awesome. + +00:45:59.080 --> 00:45:59.520 +We're scientists. + +00:45:59.640 --> 00:46:03.240 +We use notebooks every day and we're getting a little tired of reproducibility issues. + +00:46:03.460 --> 00:46:04.280 +We want to make apps. + +00:46:04.720 --> 00:46:06.120 +So they're like, we'll fund you to do this. + +00:46:06.440 --> 00:46:07.240 +And so we... + +00:46:07.240 --> 00:46:07.600 +Awesome. + +00:46:07.780 --> 00:46:10.800 +So they kind of wrote you into one of their larger grants, something like that. + +00:46:11.300 --> 00:46:14.500 +We got like a subcontract that like was covered by one of their grants. + +00:46:14.760 --> 00:46:14.880 +Yeah. + +00:46:15.960 --> 00:46:21.480 +So that wasn't enough for me and my co-founder, Miles, to work on it for two years full time alone. + +00:46:21.780 --> 00:46:26.160 +And like, I think it really let us polish the product and develop + +00:46:26.160 --> 00:46:26.460 +it. + +00:46:27.020 --> 00:46:28.500 +And then mid + +00:46:28.500 --> 00:46:40.420 +last year, in July. One of our users is like one of our probably biggest power users actually is Anthony Goldblum, who is the founder and former CEO of Kaggle. So + +00:46:40.420 --> 00:46:41.960 +really + +00:46:41.960 --> 00:46:57.360 +into data science. So he's been one of our biggest power users. And at a certain point he last year became, I guess, passionate enough about our project that he reached out to us and asked us, hey, like, can this venture fund I'm part of, can we invest? + +00:46:58.320 --> 00:47:00.180 +And so that was July of last year. + +00:47:00.690 --> 00:47:09.680 +And so we raised money from them as well as from a bunch of prominent angel investors like Jeff Dean from Google, Clem from Hugging Face, Lucas B. + +00:47:09.730 --> 00:47:10.460 +Walt from Weights & + +00:47:10.460 --> 00:47:11.020 +Bias and others. + +00:47:12.000 --> 00:47:17.040 +So now our team is funded primarily through the venture funding. + +00:47:18.400 --> 00:47:22.900 +Right now our business model is build open source software based on our venture + +00:47:22.900 --> 00:47:23.220 +funding. + +00:47:23.280 --> 00:47:28.100 +We do have plans for commercialization, but we're just not at a point where it makes sense to work on that right now. + +00:47:28.760 --> 00:47:33.000 +But what I can promise is that Marima will always be open source Apache 2.0 license. + +00:47:33.240 --> 00:47:38.660 +We never plan to sell the notebook itself, but instead plan to work on complementary infrastructure. + +00:47:39.360 --> 00:47:42.180 +Yeah, I can already think of two good ones. + +00:47:43.680 --> 00:47:43.860 +Cool. + +00:47:44.360 --> 00:47:44.480 +Yeah. + +00:47:45.280 --> 00:47:45.400 +Awesome. + +00:47:45.800 --> 00:47:49.640 +Okay. The reason I ask is, you know, people always just want to know either + +00:47:49.640 --> 00:47:50.460 +how, + +00:47:51.140 --> 00:48:05.660 +what is kind of the success story of open source, but sustainable, you know, as a full-time thing, or if they're buying into it, how likely is that going to stay open source or like what's the catch, you know, that kind of stuff? + +00:48:06.440 --> 00:48:07.960 +Yeah, no, it's a good question. + +00:48:08.180 --> 00:48:18.340 +I mean, honestly, I think when we raised money, it actually assuaged some of our users because they were like, oh, Miles and Akshay are really cool. + +00:48:18.480 --> 00:48:21.820 +They're building this really cool stuff, but how are they paying rent? + +00:48:22.200 --> 00:48:28.240 +And so when we told them, yeah, we raised a few million dollars from X Ventures, they were like, okay, that's great. + +00:48:28.400 --> 00:48:29.000 +You deserve it. + +00:48:29.240 --> 00:48:30.040 +Please continue building. + +00:48:31.560 --> 00:48:31.800 +Nice. + +00:48:32.620 --> 00:48:32.820 +Yeah. + +00:48:33.440 --> 00:48:34.000 +Yeah, that's awesome. + +00:48:34.540 --> 00:48:34.840 +All right. + +00:48:35.300 --> 00:48:37.520 +Let's see, which one are we going to talk about next? + +00:48:37.950 --> 00:48:38.060 +Not + +00:48:38.060 --> 00:48:38.900 +this one yet. + +00:48:39.340 --> 00:48:42.000 +We'll talk about this because there's a nice comment from Amir out in the audience. + +00:48:42.220 --> 00:48:44.100 +I love the new AI feature in Mario. + +00:48:44.640 --> 00:48:47.040 +So you have an AI capability. + +00:48:47.290 --> 00:48:55.120 +And then the other thing I want to talk about is, so we save some time for it, is publishing or running your Reactive Notebook as an app. + +00:48:55.480 --> 00:48:58.660 +But let's talk AI first because we started the show that way. + +00:48:59.020 --> 00:48:59.720 +I don't want to start to + +00:48:59.720 --> 00:49:00.800 +round it + +00:49:00.800 --> 00:49:01.560 +out that way, you know? + +00:49:02.460 --> 00:49:06.740 +So we have like a few different ways that AI is, I guess, integrated into the project. + +00:49:08.260 --> 00:49:14.560 +So one thing that we're trying really hard to do is like build like a modern editor, like designed specifically for working with data. + +00:49:15.170 --> 00:49:19.260 +And I think these days modern means one of the requirements is you have AI stuff built in. + +00:49:20.360 --> 00:49:22.320 +So you can like generate code with AI. + +00:49:22.510 --> 00:49:31.960 +You can like even tell the, you can like tag like data frames and like tables that you have in memory and like give them as context, give their schemas as context to your assistant. + +00:49:32.340 --> 00:49:32.760 +in + +00:49:32.760 --> 00:49:33.760 +like a cursor-like way. + +00:49:35.700 --> 00:49:50.920 +We're also like experimenting with a new service right now like marimo.amp.ai where you can go type a little prompt like generate me a notebook that plots a 3D quadratic surface with Matplotlib because I always forget how to do that. + +00:49:51.740 --> 00:50:02.520 +And then it'll do its best to in one shot create that notebook for you and then you can play around with it and then you can either download it locally or share it out. + +00:50:04.060 --> 00:50:04.900 +Yeah, very cool. + +00:50:05.200 --> 00:50:09.340 +I'll put a link to the online AI feature that people can play with. + +00:50:09.340 --> 00:50:12.580 +It looks really, really nice and it worked super quick when I asked it to go. + +00:50:12.680 --> 00:50:21.840 +Now, one thing I want to talk about really quick now that I'm looking at this is you've got the code, but it's below the presentation of the result of the code. + +00:50:22.240 --> 00:50:22.540 +Yeah. + +00:50:23.860 --> 00:50:30.040 +So that is a stylistic choice, which is configurable because people told us, Many of them told us we don't like this. + +00:50:30.250 --> 00:50:31.420 +Please just put the output below. + +00:50:31.990 --> 00:50:32.080 +But + +00:50:32.080 --> 00:50:33.000 +other people like it. + +00:50:33.330 --> 00:50:33.880 +Okay, you love + +00:50:33.880 --> 00:50:34.020 +it. + +00:50:34.320 --> 00:50:34.880 +Okay, yeah. + +00:50:34.990 --> 00:50:35.460 +And so this + +00:50:35.460 --> 00:50:35.720 +is... + +00:50:35.720 --> 00:50:36.600 +Okay, well, what is the point? + +00:50:36.660 --> 00:50:38.740 +Is the point for me to see the code or the answer? + +00:50:39.760 --> 00:50:42.680 +The point is to see the graph and the tables and + +00:50:42.680 --> 00:50:43.740 +if I care, + +00:50:43.880 --> 00:50:44.560 +I'll look at the code. + +00:50:44.940 --> 00:50:46.740 +So the way that I... + +00:50:46.740 --> 00:50:49.620 +So I think I mentioned Pluto JL really briefly. + +00:50:49.710 --> 00:50:51.720 +So Pluto JL is one of our biggest inspirations. + +00:50:51.810 --> 00:50:53.560 +It's a Julia project, a reactive notebook. + +00:50:54.160 --> 00:51:23.900 +Now, a lot of the great ideas that I think are great about Marimo came, honestly, very straight from pluto and one thing that fawns like the creator of pluto's you know he likes to say is he said code is the code of a cell is a caption for its output and that's the that's the way you think about it and when you think about it that way it makes a lot of sense um but i think like pluto and then observable also both have outputs on top i don't know it's like i feel like if you it's like one of those heuristics you look at a notebook output on top it's a reactive notebook otherwise it's imperative + +00:51:23.900 --> 00:51:30.460 +i gotcha gotcha yeah i can see why you might not like it but i like it so i think + +00:51:30.460 --> 00:51:31.360 +it's pretty + +00:51:31.360 --> 00:51:40.320 +cool yeah and then this um marmo.app slash ai once you create one of these you have a shareable link that you can hand off to other people right yeah + +00:51:40.320 --> 00:51:48.400 +so you can open a new tab which i think will so actually this all of this actually interestingly enough is running in our WebAssembly playground um + +00:51:48.400 --> 00:51:49.140 +yeah so there's + +00:51:49.140 --> 00:51:56.720 +that url right there which you can just share You can also do like the little, there's a hamburger menu, which you can click on to get a permalink that's shorter. + +00:51:58.620 --> 00:51:59.020 +But + +00:51:59.020 --> 00:51:59.420 +yeah, + +00:52:01.200 --> 00:52:05.280 +honestly, we just built this generate with AI feature like a little over a week ago. + +00:52:05.980 --> 00:52:08.300 +And just curious to see what people use it for. + +00:52:08.580 --> 00:52:08.720 +There's a + +00:52:08.720 --> 00:52:09.660 +lot more we could invest + +00:52:09.660 --> 00:52:09.980 +here. + +00:52:10.020 --> 00:52:11.960 +I mean, you know, everyone talks about agents. + +00:52:12.020 --> 00:52:14.140 +You can think about, oh, data science agent. + +00:52:14.580 --> 00:52:18.800 +But for now, we're just trying to one shot thing and see how it lands with folks. + +00:52:19.220 --> 00:52:19.580 +Yeah, sure. + +00:52:20.060 --> 00:52:20.600 +Let's start with that. + +00:52:20.680 --> 00:52:21.720 +Well, it looks really great. + +00:52:21.810 --> 00:52:23.740 +And the UI is super nice. + +00:52:23.960 --> 00:52:24.760 +So well done. + +00:52:25.420 --> 00:52:25.740 +Appreciate it. + +00:52:25.800 --> 00:52:26.700 +I'll pass that on to Miles. + +00:52:26.880 --> 00:52:29.700 +He built this as a weeknight project. + +00:52:30.859 --> 00:52:32.300 +Sometimes you just get inspired. + +00:52:32.390 --> 00:52:33.220 +Just like, you know what? + +00:52:33.800 --> 00:52:34.320 +I'm doing it. + +00:52:34.780 --> 00:52:35.680 +I'm just taking + +00:52:35.680 --> 00:52:38.340 +two days off the regular work and I'm just doing this. + +00:52:38.350 --> 00:52:38.680 +You know what I + +00:52:38.680 --> 00:52:38.760 +mean? + +00:52:38.980 --> 00:52:39.100 +Yeah. + +00:52:39.480 --> 00:52:39.920 +Yeah, definitely. + +00:52:40.620 --> 00:52:40.720 +Yeah. + +00:52:41.280 --> 00:52:41.540 +All right. + +00:52:41.740 --> 00:52:50.180 +So the final thing I want to talk about, which you kind of hinted at a little bit there with the WebAssembly stuff, is I want to run my app. + +00:52:50.460 --> 00:52:54.920 +So tell us about, as I'm fumbling around to find a place to show you. + +00:52:55.000 --> 00:52:57.920 +Anyway, tell us about running the app. + +00:52:58.660 --> 00:52:59.100 +Yeah, so there's + +00:52:59.100 --> 00:52:59.860 +a few different ways. + +00:53:02.060 --> 00:53:08.160 +The sort of traditional way, if you have a client-server architecture, + +00:53:08.760 --> 00:53:09.400 +so you have your notebook + +00:53:09.400 --> 00:53:12.160 +file, notebook.py. + +00:53:12.580 --> 00:53:14.260 +It has some UI elements, et cetera. + +00:53:14.920 --> 00:53:15.560 +It has some code. + +00:53:15.840 --> 00:53:18.140 +You get an app, but just by default, you hide all the code. + +00:53:18.240 --> 00:53:19.820 +Now you have text, outputs, UI elements. + +00:53:20.220 --> 00:53:21.240 +You can think of it as an app. + +00:53:21.880 --> 00:53:36.040 +So if you type marimorun, notebook.py, at the command line, it'll start a web server that's serving your notebook in a read-only session that you can connect to, et cetera. + +00:53:36.600 --> 00:53:37.340 +Can you do the widgets? + +00:53:40.500 --> 00:53:44.020 +Like if it's read-only, can I slide the widgets to see it do stuff? + +00:53:44.560 --> 00:53:44.900 +You know what I mean? + +00:53:44.960 --> 00:53:45.100 +Yeah, + +00:53:45.160 --> 00:53:45.620 +yeah, yeah. + +00:53:45.760 --> 00:53:47.160 +So you can slide the widgets, see it do stuff. + +00:53:47.240 --> 00:53:49.140 +You just can't change the underlying Python code. + +00:53:49.700 --> 00:53:50.500 +Like you can't like, + +00:53:50.860 --> 00:53:51.140 +got it. + +00:53:51.740 --> 00:53:52.980 +But yes, that's true. + +00:53:53.080 --> 00:53:53.280 +That is + +00:53:53.280 --> 00:53:53.560 +real. + +00:53:53.960 --> 00:53:54.960 +Exact RM dash RF. + +00:53:55.160 --> 00:53:55.240 +Okay. + +00:53:55.460 --> 00:53:55.820 +Let's go. + +00:53:56.440 --> 00:53:56.600 +Yeah. + +00:53:56.780 --> 00:53:57.120 +Yeah. + +00:53:57.600 --> 00:53:58.420 +So that is not allowed. + +00:53:58.730 --> 00:54:00.860 +And so that's the traditional way client server. + +00:54:01.720 --> 00:54:04.960 +Um, but, last year we actually, I think it was last year. + +00:54:05.210 --> 00:54:13.480 +Uh, we, we added support for, WebAssembly through the PyDype project, which is a port of CPython to, to Wasm slash Unscriptum. + +00:54:14.140 --> 00:54:17.020 +And so now you can actually take any Marima notebook. + +00:54:17.240 --> 00:54:27.300 +And as long as you satisfy some constraints, which are documented on our website, you can export it as like static HTML and some assets and just throw it up on GitHub pages or server it wherever you like. + +00:54:28.380 --> 00:54:30.480 +And that's also what our online playground is powered by. + +00:54:31.040 --> 00:54:39.120 +And we found this to be just like a really easy way to share notebooks, easy for folks in industry, easy for educators, and also just incredibly satisfying. + +00:54:39.240 --> 00:54:52.140 +In our docs, we have tons of little Marima notebooks iframed into various API pages, etc., that just let you actually interact with the code as opposed to just reading it statically. + +00:54:52.500 --> 00:54:53.040 +Yeah, that's nice. + +00:54:53.460 --> 00:54:53.900 +That's super cool. + +00:54:54.320 --> 00:54:56.700 +So it's pretty low effort, it sounds like. + +00:54:57.000 --> 00:55:04.100 +And it's a lot of work to run a server and maintain it, especially if you're going to put it up there so other people can interact with it. + +00:55:04.260 --> 00:55:07.140 +And you've got to worry about abuse and all that kind of stuff. + +00:55:07.220 --> 00:55:11.920 +If you can ship it as a static website, people are just abusing themselves if they mess with it anyway. + +00:55:12.520 --> 00:55:13.040 +Exactly. + +00:55:13.880 --> 00:55:17.920 +I had to give huge props to the PyDive maintainers. + +00:55:19.260 --> 00:55:23.580 +They are amazing, really responsive, and just building this labor of love. + +00:55:24.760 --> 00:55:31.800 +They are really pushing the needle on the accessibility of Python. + +00:55:32.260 --> 00:55:33.000 +Yeah, that's awesome. + +00:55:33.190 --> 00:55:34.460 +Have you done anything with PyScript? + +00:55:35.280 --> 00:55:37.100 +I haven't done anything with it. + +00:55:37.120 --> 00:55:37.860 +I know of them. + +00:55:37.900 --> 00:55:39.920 +So they also use Pyodide underneath the hood. + +00:55:40.380 --> 00:55:41.000 +Yeah, they let you + +00:55:41.000 --> 00:55:43.560 +pick between Pyodide for more data science-y stuff and + +00:55:43.560 --> 00:55:45.620 +MicroPython for more faster + +00:55:45.620 --> 00:55:46.320 +stuff. + +00:55:47.760 --> 00:55:51.280 +I am pretty not educated about MicroPython. + +00:55:51.340 --> 00:55:56.680 +I read about it, but I'd be curious on your take also on choosing one versus the end. + +00:55:56.960 --> 00:56:00.920 +Yeah, well, I think it's brilliant if you want a front-end framework equivalent. + +00:56:01.000 --> 00:56:05.260 +If you want PyView or PyReact or whatever, + +00:56:05.680 --> 00:56:06.060 +You don't + +00:56:06.060 --> 00:56:07.500 +need all the extras. + +00:56:07.580 --> 00:56:12.160 +You kind of need the Python language and a little bit of extras and the ability to call APIs, right? + +00:56:12.440 --> 00:56:21.780 +And so MicroPython is a super cut-down version that's built to run on like little tiny chips and self-contained systems on a chip type things. + +00:56:22.240 --> 00:56:23.520 +So it's just way lighter. + +00:56:23.600 --> 00:56:26.640 +It's 100K versus 10 megs or something like that. + +00:56:26.980 --> 00:56:27.780 +And if + +00:56:27.780 --> 00:56:35.240 +it solves the problem, right, if it gives you enough Python, I guess is the way to think of it, then it's a really cool option for front-end things. + +00:56:35.500 --> 00:56:36.980 +I don't know that it works for you guys, right? + +00:56:37.180 --> 00:56:41.360 +Because you want as much compatibility with like Matplotlib + +00:56:41.360 --> 00:56:42.620 +and Seaborn + +00:56:42.620 --> 00:56:43.320 +and all these things. + +00:56:43.520 --> 00:56:52.200 +But if your goal is like, I want to replace the JavaScript language with the Python language, and I know I'm in a browser, so I'm willing to make concessions, I think it's a pretty good option. + +00:56:52.580 --> 00:56:53.340 +Yeah, that makes sense. + +00:56:53.480 --> 00:56:53.860 +Yeah, you're right. + +00:56:53.910 --> 00:56:55.020 +We want max compatibility. + +00:56:55.420 --> 00:57:05.100 +Like we have one demo where like you can load sklearn, take PCA of these like images of numerical digits, put it in all Terraplot, select them, get it back as a data frame. + +00:57:05.360 --> 00:57:06.420 +That's all running in the browser. + +00:57:06.640 --> 00:57:06.940 +And, like, + +00:57:07.280 --> 00:57:08.060 +it's magical. + +00:57:08.480 --> 00:57:09.560 +It's on your home screen. + +00:57:09.720 --> 00:57:10.860 +People can watch the little + +00:57:10.860 --> 00:57:12.280 +embeddings + +00:57:12.280 --> 00:57:12.680 +explore. + +00:57:12.820 --> 00:57:13.300 +It's super cool. + +00:57:13.640 --> 00:57:14.180 +It's very cool. + +00:57:14.480 --> 00:57:18.100 +It's honestly magical what PyDat has been able to enable. + +00:57:18.920 --> 00:57:19.040 +Yeah. + +00:57:19.480 --> 00:57:20.660 +I imagine it's just getting started. + +00:57:21.160 --> 00:57:21.900 +I think so. + +00:57:22.380 --> 00:57:22.540 +Yeah. + +00:57:22.840 --> 00:57:23.040 +Yeah. + +00:57:23.420 --> 00:57:24.380 +They're constantly shipping. + +00:57:24.440 --> 00:57:26.240 +They just added support for Wasm GC. + +00:57:27.760 --> 00:57:29.240 +So performance is increasing. + +00:57:29.680 --> 00:57:30.080 +Expose + +00:57:30.080 --> 00:57:30.380 +the bug + +00:57:30.380 --> 00:57:31.500 +in WebKit. + +00:57:31.920 --> 00:57:32.720 +along the way. + +00:57:33.200 --> 00:57:33.900 +Wow, okay. + +00:57:34.420 --> 00:57:34.660 +That's good. + +00:57:35.360 --> 00:57:35.540 +Yeah. + +00:57:36.700 --> 00:57:36.860 +Cool. + +00:57:37.300 --> 00:57:40.420 +All right, well, we're getting to the end of our time here. + +00:57:40.640 --> 00:57:42.540 +So, you know, people are interested in this. + +00:57:42.920 --> 00:57:44.020 +They might want to try it for themselves. + +00:57:44.400 --> 00:57:49.000 +Maybe they're part of a science team or a data science team at a company. + +00:57:49.320 --> 00:57:52.720 +What do you tell them if they're interested in Marmo and they want to check it out? + +00:57:53.700 --> 00:57:57.520 +The easiest way, I think, is to just start running it locally. + +00:57:57.960 --> 00:58:00.960 +And so our sort of source of truth is our GitHub repo. + +00:58:01.500 --> 00:58:08.620 +So github.com, marimoteam.com, or if it's easy to remember, our homepage is marimo.io, which links to our GitHub. + +00:58:09.410 --> 00:58:14.860 +But then you can install from pip or UV or whatever your favorite package manager is and just sort of go to town. + +00:58:15.500 --> 00:58:24.680 +If you really just don't have the ability to do that for whatever reason, you can go to marimo.new, which will create a blank Marimo notebook powered by WebAssembly in your browser. + +00:58:25.740 --> 00:58:29.700 +It has links to various Marimo tutorials, as an example, notebooks. + +00:58:31.180 --> 00:58:38.740 +We didn't even get to talk about the UV integration, which may be another time, but that's another way that Marimo makes notebooks reproducible down to the packages. + +00:58:39.580 --> 00:58:40.320 +Oh, yeah, okay. + +00:58:41.160 --> 00:58:41.680 +UV is fantastic. + +00:58:42.000 --> 00:58:44.680 +I'm a huge fan of UV and Charlie Marsh and team. + +00:58:45.000 --> 00:58:50.300 +Yeah, but GitHub, Marimo.io, and docs.marimo.io is what I would recommend. + +00:58:51.040 --> 00:58:52.280 +Thank you so much for being on the show. + +00:58:52.980 --> 00:58:54.060 +Congratulations on the project. + +00:58:55.000 --> 00:58:58.020 +It's really come a long way, it sounds like, so it looks great. + +00:58:58.520 --> 00:58:59.700 +Thanks, Michael. I appreciate it. + +00:58:59.740 --> 00:59:00.620 +It was a lot of fun to chat. + +00:59:00.960 --> 00:59:01.700 +Yeah, you bet. Bye. + +00:59:02.120 --> 00:59:02.220 +Bye. + +00:59:03.420 --> 00:59:06.000 +This has been another episode of Talk Python to Me. + +00:59:06.920 --> 00:59:07.760 +Thank you to our sponsors. + +00:59:08.230 --> 00:59:09.460 +Be sure to check out what they're offering. + +00:59:09.680 --> 00:59:10.900 +It really helps support the show. + +00:59:11.640 --> 00:59:14.140 +This episode is sponsored by Worth Recruiting. + +00:59:14.720 --> 00:59:19.120 +Worth Recruiting specializes in placing senior-level Python developers and data scientists. + +00:59:19.760 --> 00:59:24.500 +Let Worth help you find your next Python opportunity at talkpython.fm/worth. + +00:59:25.200 --> 00:59:26.080 +Want to level up your Python? + +00:59:26.540 --> 00:59:30.140 +We have one of the largest catalogs of Python video courses over at Talk Python. + +00:59:30.660 --> 00:59:35.340 +Our content ranges from true beginners to deeply advanced topics like memory and async. + +00:59:35.670 --> 00:59:37.880 +And best of all, there's not a subscription in sight. + +00:59:38.350 --> 00:59:40.880 +Check it out for yourself at training.talkpython.fm. + +00:59:41.560 --> 00:59:45.760 +Be sure to subscribe to the show, open your favorite podcast app, and search for Python. + +00:59:46.170 --> 00:59:47.080 +We should be right at the top. + +00:59:47.600 --> 00:59:56.460 +You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. + +00:59:57.100 --> 00:59:59.360 +We're live streaming most of our recordings these days. + +00:59:59.800 --> 01:00:07.200 +If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. + +01:00:08.200 --> 01:00:09.340 +This is your host, Michael Kennedy. + +01:00:09.760 --> 01:00:10.600 +Thanks so much for listening. + +01:00:10.780 --> 01:00:11.740 +I really appreciate it. + +01:00:12.060 --> 01:00:13.700 +Now get out there and write some Python code. + diff --git a/youtube_transcripts/378-flet-flutter-apps-in-python.vtt b/youtube_transcripts/378-flet-flutter-apps-in-python.vtt index b663771..2da013b 100644 --- a/youtube_transcripts/378-flet-flutter-apps-in-python.vtt +++ b/youtube_transcripts/378-flet-flutter-apps-in-python.vtt @@ -1305,7 +1305,7 @@ we you can build Flutter app in Python and it's good you know like flat it's 00:19:20.700 --> 00:19:29.200 -not just about like reflecting all the widgets all the the power available +not just about like reflecting all the widgets all the power available 00:19:29.200 --> 00:19:34.920 diff --git a/youtube_transcripts/390-mastodon.vtt b/youtube_transcripts/390-mastodon.vtt index c031dcf..220e420 100644 --- a/youtube_transcripts/390-mastodon.vtt +++ b/youtube_transcripts/390-mastodon.vtt @@ -949,7 +949,7 @@ there are capybaras and squirrels in my timeline way more than there were before 00:13:58.480 --> 00:14:05.680 -enjoy that stuff and there is still the bad things and still the the serious stuff and +enjoy that stuff and there is still the bad things and still the serious stuff and 00:14:05.680 --> 00:14:11.680 diff --git a/youtube_transcripts/391-pyscript-powered-by-micropython.vtt b/youtube_transcripts/391-pyscript-powered-by-micropython.vtt index b68a950..fcb6f62 100644 --- a/youtube_transcripts/391-pyscript-powered-by-micropython.vtt +++ b/youtube_transcripts/391-pyscript-powered-by-micropython.vtt @@ -2657,7 +2657,7 @@ Australia team that entered the Robo World Cup for Americans, that's soccer as i 00:32:01.280 --> 00:32:09.280 -And which is rather appropriate given the the fact that the world cup in for human beings is happening right now +And which is rather appropriate given the fact that the world cup in for human beings is happening right now 00:32:09.280 --> 00:32:16.000 diff --git a/youtube_transcripts/462-pandas-and-beyond-with-wes.vtt b/youtube_transcripts/462-pandas-and-beyond-with-wes.vtt index bbb0b3c..a9bb4a9 100644 --- a/youtube_transcripts/462-pandas-and-beyond-with-wes.vtt +++ b/youtube_transcripts/462-pandas-and-beyond-with-wes.vtt @@ -727,7 +727,7 @@ uh cycle. There's like intrinsic operations. Instead of multiply this number by multiply that number to these eight things all at once something like that. That's right yeah or you 00:53:41.120 --> 00:53:48.800 -might say like oh I have a bit mask and I want to select I want to gather like the the one bits +might say like oh I have a bit mask and I want to select I want to gather like the one bits 00:53:48.800 --> 00:53:54.640 that are set in this you know this bit mask from this array of of integers. And so there's like a