André Prado and James Spadafora are engineers at Industrial Light and Magic, a visual effects studio whose work includes Star Wars and Marvel movies. We discuss the immense technical demands of modern visual effects and how Python and Django are integral parts of the overall pipeline.
This podcast does not have any ads or sponsors. To support the show, please consider purchasing a book, signing up for Button, or reading the Django News newsletter.
Carlton Gibson 0:06
Hi, welcome to another episode of Django Chat a fortnightly...I don't know how often we do a podcast on the Django web framework. I'm Carlton Gibson joined by Will Vincent. Hello Will.
Will Vincent 0:15
Hi, Carlton.
Carlton Gibson 0:15
Hello Will. Today we've got with us, James Spadafora and André Prado from Industrial Light and Magic, we're going to talk to us about all things of how Django is used in wonderful world of audio visual and such, I think thanks for coming on. It's really exciting to have you.
André Prado 0:32
Yeah, thank you.
James Spadafora 0:32
Thank you for having us fun.
Carlton Gibson 0:33
So let's kick off like, hang on Industrial Light and Magic. You do really cool films. So how on earth is Django involved in that? How, how'd you get into programming? What's your backstory? How, you know? How does Django play a role in what you do? You want to go first?
André Prado 0:50
Sure, I can go first. So I actually graduated in computer engineering, I started working the hardware industry, and then I've changed industries a couple of times and end up in visual effects, almost by accident. And then it's been like seven years now, since I've been to visual facts. And, you know, a lot of people don't know anything about Visio effects, like I didn't, but do we actually have this massive infrastructure, to render frames, right to have like petabytes of storage 1000s of CPUs in Farm to do batch processing, we have a bunch of services, we have many data centers, and they are all there to produce these great pictures that we're seeing the screen and that we don't even know what they are, how they are made. Yeah, can I off when you say you've got these, these massive data these days? They like in the basement in the office? Or are they in the clouds?
Carlton Gibson 1:47
Because there is a lot of a lot of compute
André Prado 1:50
A lot of compute. Yeah, no, we mostly have our own data centers. And we have our own connections between the different data centers. So we have data centers in Sydney, San Francisco, Vancouver, Singapore, London, basically, everywhere we have an office, we've also been using the cloud a little bit recently. But Cloud ends up being very expensive for us, like, our workstation has, like, you know, 200 gigs of RAM and 64 cores to be able to open those heavy scenes and those, you know, do some rendering. So you can imagine how the cloud, you know, can be very expensive when you're paying for those sort of machines and very high end GPUs. Sure.
Carlton Gibson 2:28
You don't get that on the AWS free plan. We know
Will Vincent 2:34
Well Carlton and I, as a side note, both got the new MacBooks with the M one, a one Max chip, and yeah, but now I feel inadequate hearing about 200.
Carlton Gibson 2:45
Well I've downloaded Blender. So I'm almost there. Right?
André Prado 2:52
You're one step ahead. Yeah, I basically come from, you know, more recently, more DevOps backend, systems engineering, and I actually don't have a lot of knowledge in the proper, you know, video facts. And you know, I've been in the industry for a while I help people with workflows and whatnot. But I think James has been much closer to the product than I have. So maybe that's a good segue for you is bad.
James Spadafora 3:19
Yeah, sure. Yeah. So I graduated from Savannah College of Art and Design with a Master's of visual effects. And I started at ILM, right out of the school, working as what we call a technical assistant, which is a way of saying I worked from midnight to 10am. Watching these renders on these massive machines that Andre was talking about. And then, you know, followed the support path role helping artists troubleshoot issues, with their scenes, until eventually I moved into the production tools team, where we develop tools for interacting with data, such as like, what shots need to be done, what tasks need to be done, generating media, those sorts of things.
Will Vincent 4:07
Can I ask what the? What is that? What do you study in that degree? Because I know, Andre, you gave a fantastic Python talk in the show notes on sort of the pipeline. And you know, James, I can see fluent Python on your bookshelf behind you. So I know we're gonna talk about Python, but what is what does that curriculum look like? So I have no idea what that what that would entail.
James Spadafora 4:28
Sure, yeah. So it was a mix between art and programming. So we learned you know how to use visual effects software, create images and render them. Naturally, I kind of gravitated more towards the technical concepts, which is how I ended up in the position I'm in now. We did things such as learning to write shaders, which are programs that determine how light interacts with surfaces as well as just you know, compositing putting images together all kinds of things.
Will Vincent 5:03
Now, would that be like C++ for vs. Python? Because I know, Andre, just from your talk, you mentioned that a lot of the engines themselves are written in C++, but the glue that puts them together is typically in Python. Is that correct, James?
James Spadafora 5:17
That's true. We did. We did do a little bit of C++ in that program, if I remember correctly, but we never got into like writing full fledged renderers. Sure. It's more like yeah, Peter science thing. You could think of it as like the program was learning to use the end result of those tools, if that makes sense?
André Prado 5:36
To give an overview. Like you can imagine, I think the majority of coding visual factors spied on by a large degree. And all those renders and C++, you know, simulations, and whatever needs to be super fast rendered in C++ offers a Python API. So basically, every tool that we use, we use you you have like a Python binding a Python API that you can import into your environment, and you can interact with the scene in real time via Python. Let's say you know, you're making a I don't know a space battle and you have a spaceship, then through Python, you can you know, create 10 spaceships instead, using an API and rotate them and change. You know, the texture or what have you. Are via Python API's very similar to how you do import requests, or what have you. But you actually import the DCC, we call this a digital content creator API's. Yeah, we're just importing the Millennium Falcon. It's yeah, that's
Will Vincent 6:37
In your talk Andrew you had, you know, import my and you had some examples of that. So, is that so what is it? Well, I have to jump in my, I told my wife, I was interviewing you all, and she's a huge Mandalorian fan. And she was like, what about that huge screen they used to film it would write the volume, I think, just to like, skip ahead, which I believe uses Unreal Engine, which is like a video game for the background. And then would that still use Python to move things around? Because I've seen videos where they can, you know, right there, change things?
André Prado 7:10
Yeah, I think interesting. You mentioned that because we were just starting a new what we call stage craft studio in Vancouver and Spad has actually been part of setting that up. But yes, definitely, it's not. As far as I'm aware, we haven't been using Unreal as much we have our own rendering engine that we wrote in house, for reasons and I don't think I'm the most qualified person to talk about that. But for sure, there is Python there. We run our own software that we wrote in house and there is Python bindings for all the different things. I don't think a lot of the really heavy lifting is done in C++, though. We have some genius engineers working on that side of things. Now, if you have anything to add Spad
James Spadafora 7:53
No, I think you summed it up pretty well. We have a lot of engineers writing some great tools for interacting with the screens. We also have a inter we've created interfaces like on I don't know how much detail can go but
Will Vincent 8:08
This is a technical podcast so go deep.
James Spadafora 8:10
Yeah. I met like legally.
Will Vincent 8:14
We can edit it out. If no one listens. It's fine.
James Spadafora 8:18
Well, we can we can cut this if it doesn't work, right. So we've developed like interfaces on the iPad. That let's
Will Vincent 8:28
Yeah, that's on the video I saw. I saw that on the the volume is that there's, it's from 2020, it was showing someone with the iPad and saying they were, you know, moving it around and showing the screen moving. So that's out there.
James Spadafora 8:40
Correct? Yeah. All right. Well, that's good, that I can go into detail on that. Yeah, so typically, we'll have someone on set with the iPad. And what they'll do is they'll manipulate the light in the scene. So in typical film production, if you want to change the lighting, you would need to like move your lights around. But with the volume, what we're able to do is with the iPad, create these squares of light up on the walls, to illuminate the scene. And that allows the director to get much faster feedback. And all the tooling around that I think is I believe is written in Python to kind of stream that information over to our proprietary software. Because it's all LEDs, right? It's the 21 foot screen and then also the ceiling right so there's no actual lights right? It's just LEDs everywhere. Pretty much. I'm actually not sure Okay, well, how much practical lighting versus the LED wall but I know that we do use the LED wall a lot to manipulate the light in real time. Right? That's what working
Will Vincent 9:38
And a suit of armor is pretty bad for that. Sorry, Carlton, I'm just so geeking out. Right? Isn't that awesome? Last point is not also true that like you know, lighting is super hard, but then even reflective surface. Like that really magnifies more than skin. Right. The issues.
James Spadafora 9:53
Absolutely. I mean, I don't think we could get as good looking results without the volume because the reflections in real time on the Mandalorian suit of armor like that, that's all from the wall. Other in the old days, we probably would have used a green screen, it would have been kind of tough to do that. So yeah. And I think it's also worth noting that we've been using the wall for a lot of projects, not just Mandalorian. But for example, the last Batman movie used some of our technologies and yeah, we have a lot of things in production using that. Super cool. It's like reinventing filmmaking to some degree. Absolutely.
Will Vincent 10:27
Carlton, you you say something?
Carlton Gibson 10:29
Well, no, it's, it's quite funny. Because James and Andrea, from listeners at home, James and Andre are in Vancouver, and Will's in Boston, I'm in Spain. So like, they've got a nice little chat, and I keep trying to but in but I can't ever get it, it's fine. You know, will you just carry on? What I wanted to ask about was the the change over the last, say, decade, or, you know, or longer in terms of because you've used lots of phrases like real time, and you're talking about having an iPad in hand and being able to drag components around. And that's it. That's just raw processing power. Right? That's, that's that stuff that wouldn't have been possible, you know, 510 years ago, or even, you know, can you talk about the change in the technology, and then, again, give you one example was that I saw a line from the Pixar people about rendering Toy Story for, and they were able to render it basically, in real time as they were, as they were producing it versus when they did Toy Story originally, 20 years previously, you know, it would take several hours to render a single frame. Yeah, I think it's, yeah, for sure. Technology has evolved a lot in the last decade. But I do think movies keep getting more complex, true, and keep having higher resolutions. So it's almost like, doesn't matter how much it evolves, artists on computer graphics are always using everything we've got. So
Will Vincent 11:51
yeah, it's like, we definitely do a lot more Microsoft Word always takes seven seconds to open, right? No matter what decade, you're in.
Carlton Gibson 11:59
getting an extra bin in the kitchen, you just end up in what rubbish.
André Prado 12:04
And just the amount of visual effects show that it's used nowadays, right? Basically, every single show you watch on Netflix, or Amazon Prime, or what have you, has an immense amount of effects shots, probably more than movies than I had, like five or 10 years ago, even though you know, don't even notice it. It's all they're making for a very realistic experience.
Will Vincent 12:25
So I want to talk about the pipeline, but maybe we could just jump to where does Django fit in all this? Right? Like, what? Let's start. Let's start with that. Right. So you have this whole C++ things glued together with Python, you've got massive rendering and databases. Where's that? Where's the Django? Small bit in all that?
André Prado 12:44
I think that's a great question. I think, you know, I'll give a little bit of a historic perspective plus, you know, and, and spad, can get into the details that works for you as bad cook, but I think, yeah, historically, you know, visual factors are just a bunch of Python scripts on big shared folder, and then you add that folder to your Python path. Any version of those Python scripts, I think that defines, you know, visual effects pipeline, something. And I'm talking about, you know, millions of lines of code. So it's not like some Python script is actually like terabytes of Python scripts. But I would say that, with things getting more complex, we have in multiple data center doing more shots, we needed to scale that to some degree. And I think a good way of scaling it is, you know, decoupling a little bit of the client side dependencies from the business logic behind the service. And that's where servers start making an appearance in visual facts, I believe, and also, you know, some people want some fancy web UIs done on React instead of like Qt or, you know, some more desktop based apps, which also a server helps with. So I think that transition has been happening the VFX industry, a little bit slowly toward services. And I think, you know, historically, at ILM, we are using each group has a lot of independency to the they are independent, to choose their own tools and frameworks. And we end up with a lot of different teams speaking different tools, for example, team would use tornado or Tim, you use flask, and it made it super hard to be cohesive, I feel how we deal with authentication, how we do databases, so we sort of standardize on using Django to building some of those services that abstract the business logic. So way, we set a common some common libraries that we use, like Django rest, for example, and other things like how we connect with LDAP and etc. We just made the process of developing a service more streamline that allowed us you know, to have Ansible plays or Docker files that are standard across the teams. Some specific examples on how Django is used and maybe Spad you can go ahead.
James Spadafora 15:05
Yeah, sure. Um, actually, if you don't mind, I just want to expand a little bit on what you were saying about the the scripts in the pipeline, and whatnot. So, you know, before I was on this team, I was on another team that worked as kind of the go between between the projects we're working on and the engineering teams. And we have some of the most the smartest artists in the world working at ILM. And as a result, they like to, you know, write Python code. So managing all the scripts written by all these different people across all these different teams, as always, been quite a challenge for us. You know, and also that that move from Qt, because all of our third party software, my nuke was all written in Qt. So that's what we were using for a long time before we made the shift to using web UIs, and whatnot. To get back to the specifics of the Django stuff, one area we're using Django is to deal with subscription management. So for an artist, the typical workflow is they'll do some piece of work and write out some media, that then they will submit that media for review for either the director or their supervisors. And we have a lot of artists submitting a lot of media, and trying to route that media to the right people, is a bit of a challenge, because you have, we have like five different sights all working on different projects with different people. So we developed a service that lets users subscribe to different tasks based on you know, what, what the artist is doing. So for example, if I'm a director, I could say, I only want to see lighting media from San Francisco. So they can go into this web UI and edit these filters. And every time the artist submits media, it goes to the correct people based on the filters that they've created. Yeah, filtering and organizing stuff is really hard. Even there's a site Django packages.org, the Jeff tripla has taken over that.
Will Vincent 17:11
You're just going through grids and opting into things. And I can't even imagine how complex it is to actually do that. At the scale you're, you're talking about? I mean, can you? You know, because with code, you can use Version Control and get like, how do you track changes on a image or audio file? Is there a way or what was the process for that? Well, we typically don't change track, changes on like the specific files, what we do is we track submissions.
James Spadafora 17:38
So if the artists, they'll submit something, and they'll get some notes that are attached, attached to that version, the artists will then act on those notes in submit a new version. And we keep track of all those versions associated with like a task that needs to be done.
André Prado 17:56
Yeah, I think in general terms, VFX companies should have this thing called an acid system, which is very much in the core of the company. And basically, whenever someone published a new version of a model was simulation rigging, that's a new asset. And then you basically track some metadata with the asset a bit like s3, and you have like versions for NASA to have dependencies between assets. So let's say compositing some scene and model change it or whatever, then I get a notification, like, hey, there's an more up to date version of the Ironman model. You know, do you want to update it? or what have you? And, yeah, there's a bunch of things like that. And that to have to tracking data across the pipeline is super important. As you know, as I've mentioned, we have a lot of storage and those assets are massive, like a texture can have like terabytes, you know, and it's, it can be very cumbersome to keep the disk space. Tidy, as we have many shows and artists and assets being produced
Will Vincent 18:59
Well Carlton we always talk about you know, something internet scale. And like, I think we need a new vocabulary for VFX scale, because it's just.
Carlton Gibson 19:07
It's so good. Because you've watched these over the last year or so watch these, you know, new generation of Mac's come out, and they're kind of massively overpowered for web development. You know, like, you know, if there's just more more Meg's of RAM and more GPUs, and you really need but then when you get into doing, you know, actual visual effects? Well, yeah, that's where these sort of superpower computer systems really do.
André Prado 19:32
Yeah, we definitely have had scenes that you know, just to open it, you need like, you know, 200 gigs of RAM just to open the scene. Yeah.
Will Vincent 19:41
Several questions. One, I mean, why, again, I'm just cribbing. So much from your talk, Andre, because I watched it a couple of times. It's such a good introductory talk. Thank you. I'd Well, the things you mentioned, because I think you gave the talk in 2019. And you mentioned that python two was dominant in 2018. And there's gonna be a switch to python three. We certainly have lived through that switch in the Django world. Is that, is that switch complete? Or like what version of Python? Are you using? Or is it all over the place? Because it's, it's kind of all over the place? Good question. That's pretty much all Django python three.
André Prado 20:14
That's a great, it's a great topic. I think, you know, this is like my new and other software division effects artist use the shape with a Python interpreter. And that Python interpreter has still being the standard being true seven, there's some of them that are in python three, nowadays, or, you know, I feel the industry is slowly moving towards python three, we've certainly spent huge effort on migrating to python three recently, a lot of those scripts. And as I've mentioned, we're talking about millions and millions of lines of code from Python to migrate. So it's very hard to justify the business need for it, I feel because in Ireland has been around for 40 years, and we have a lot of old code lying around, and you know, but they are working in the business function, right? So and it doesn't matter, you know,
Will Vincent 21:10
As Carlton I joke, every code is like, just good enough to not break.
André Prado 21:15
Yeah, I will say, though, that we've done new versions of those tools being python three, only that puts pressure on companies to move to python three, all of our service side code, or most of it is written in python three, I would say, when you don't have a dependency in the runtime of the artists software, which is python two, we can do python three. So really, we're trying to make all of our code compatible for two and three, our new services start on three. But I think we are winning that battle. It's slowly expanded my if there's anything you want to add,
James Spadafora 21:47
No, I think we're definitely winning that battle. We just had a big push across our teams to convert all that artist code over from Python two to python three compatible, and it's, it's given us an opportunity to do a lot of housecleaning, which has been great, like archiving stuff that isn't used anymore. And it's been an interesting exercise to figure out what's used and what's not. I think in hindsight, it would be great to have more analytics on a lot of stuff. So I think I think that's a great thing to build in, if you have a lot of software to maintain.
Will Vincent 22:21
On this topic, Carlton, you know, as a fellow, you know, maybe just for listeners, what was the two or three transition like for Django, because smaller scale, but still pretty massive amount of work?
Carlton Gibson 22:30
It lasted forever. I mean, like, you know, I was maintaining packages like dang Django filter, Django rest framework for the whole, you know, over that period. And you were just supporting 2.7, you know, and then 3.2 3.3 3.4 as it started coming out, and you're running the test suite against each of them. And it was a real, you know, at the end, it was a got to be a real pain. It was like, Oh, we do we really still supporting 2.7 here. And then I remember when I was like, no, actually, it's end of life. Now we're starting to drop it. And it was like, Ah, what a wait, it's so liberating. Yeah, yeah, it really was. And they got it where you had this nice world where you could write library code, which was both two point set python two compatible and python three point compatible, and it sort of went on, it was like, Yeah, this is doable. This is no problem. But when you could delete all those objects, support from objects and when I can't remember the things and to get rid of six, this, you know, you weren't able to call the string functions, you had to use the six compatibility shim, and it was when all of that jumped to eight, which is like, Oh, wait, like a breath of fresh air. It really was. But you still, you know, you break something, which is python two compatible, and you still get issue reports opened, as I Oh, you've dropped item two. And so python two has been in their life for a couple of years. So I don't feel too bad about dropping it. But clearly, there's still lots of people using it. And there's still it's still, it's still really out there.
Will Vincent 24:06
And then what are you and Mariusz? What what versions of Python? Are you testing for Django now? Like 3.5 likes I know, you have to know three.
Carlton Gibson 24:15
So but Jamie 3.2 is three six through till three nine or something. And then python three, or Django three is 3.8. Up. But I think much smaller space to cover then, as you say, the visual effects area and yeah, shipping with tear wood. Yeah. We've got like 30,000 tests and you know, so many lines of code. We're not talking terabytes of code. And yeah, I mean, so Okay, here's a good question tests, right? Because so I'm, I'm a digital artist, and I write up a little Python script to manipulate my my 3d scene that got invited in front of me, it works and it you know, I can do exactly what I want. I've patched it together, but I haven't written any tests for that ever. Okay, is that yeah,
James Spadafora 25:02
Yeah, that's pretty common. So so how do you how do you address that? So usually we'll use artists do and I do it live? Yeah. Do it live? Yeah. So what we found with a lot of the artists corpses, you kind of have to have a understanding of the software to test it. So this was a good exercise for our team, because a lot of them hadn't touched artists tools. So they got to learn a little bit in that regard. Ideally, I mean, all of our software that we write on our team has tests for the most part, and that that's super important to us. And something we want to continue to doing, continue doing going forward, because it just makes things so much more maintainable, and upgradable. Yeah, for artists, scripts, it's a little hard to enforce, because we have a bit of a wild west of scripts built into our pipeline where anyone can commit anything, and use it. And if you want to expand on that, Andre.
André Prado 26:05
Yeah, it feels a little bit chaotic. And it is, but it kind of works. We have a software management system that we wrote in house a little bit like, you know, peep in virtual land, but it's our own implementation. And that basically tracks artists software, so that you can version up version down, we can easily roll back, like rolling back software for us like a one second operation, you basically change a symlink to point to the previous version. So at the end of the day, it we do release, you know, hundreds of times a day, those scripts and software, so we really suffer quite often. And we make it easy to rollback. And for the artists software, in particular, in the python three migration, we in our software management system, we added a check that it basically tries to compile the software to python three. And if it fails to compile that means, you know, you cannot release it anymore, because not python three compatible. So we enforced at that level, we didn't need the you know, the unit test or whatever. Because it's just not doable, I think, for our problem right now or problem space. But we did enforce that at least compiles with both python three and two, so that you can make sure that it is compatible. And if not, you have to do a dash dash force, or some people still do it. But you know, at least to track everything. We have like a web UI sudo dash dash force, you know, make it well, that's something like, you know, testing in our RAM Carlton, right. I mean, I've often asked you for advice, because there's, at one extreme is test driven development, do the test first 100% test coverage. And then there's also, you know, happy path programming, test it and then when something breaks, write another test. That's more of the iterative approach, I think, Carlton, that you've advised to me, at least, as well on this, it depends as well, like, you know, to test, even in the quite, let's say,
Carlton Gibson 27:56
mundane realm of web development, let's call it where you're testing a web UI that's hard to test it really is. You spend a lot of time writing, you know, UI tests, and they're really flaky and they fall apart. integration tests are, yeah, to try and do that with a sort of, you know, a graphical editor like blender or whatever. I don't know what the artists use, but that kind of, you know, 3d environment taught that. I can't imagine that's even feasible to get off the ground testing that. I just can't imagine there are artists that seem impossible. Yeah. I think one thing should that's bad mentioned is that we have a lot of people writing code that are not trained software developers are terrible software developers, software, I hope they care about software developers, but not about software.
André Prado 28:44
You know, at the end of the day, our business is to ship pictures for the image for the movie, it is not to ship software to write. So we think that there is a trade off there. And yeah, usually, we just need to know how to manage it. You know, if it's some very critical service like that system, you better expect it has a great test coverage, because you can bring down the whole studio globally, if you you know, release a bug and you know, Marvel is not going to change the release date of a movie because we can't render it or whatever. So, you know, some very core bits of the software, I think, to have the love and care they deserve. But this more scripted side of things, which is a lot of the software we have to maintain. Yeah, it doesn't need or follow the same scrutiny, maybe.
Carlton Gibson 29:33
I think I think the phrase you've hit there is it doesn't need it. It like this is a this is a script I'm using to me, like a 3d scene that I've got in front of me and I'm in a GUI program. I don't need it to be, you know, unit tested in the same way. It's, it's a tool to help me what with what I'm doing not, you know, COBIT my business logic that was compiled the Django app you were describing where you're doing the subscription tracking to asset changes that sounds sounds like The sort of thing that would be susceptible to, you know, yeah, much more recourse.
André Prado 30:05
Of course those scrape tends to evolve. And then people start using each other scraped, and then they build a library around the fast forward three years that random scraped is now a core part of the business. Yeah.
Carlton Gibson 30:19
It's like the mythical Excel macro that, you know, the whole the whole multi billion dollar business runs on this one Excel sheet.
Will Vincent 30:32
I have a question about infrastructure, which this is just like the layman's version, but I think it's somewhat famous that Pixar with Toy Story two, right, they lost it, except some art, like producer was on maternity leave and had a copy on her computer. I'm just curious for you as professionals, what, what does that story say to you? Because I hopefully that's still not possible. I mean, I guess you couldn't store a movie on a computer. But like, how do you? How do you test or not cover someone like, you know, nuking everything, right, because that's a pretty crazy story. I think they actually did thought they didn't have it for a number of days. And there's so
André Prado 31:08
we had a bug a couple of years ago, when we started using with Docker and someone ran sudo, docker rm, you know, remove the container after it executes. And then it mounted an NFS Share that had show and then it deleted parts of the show. But we could catch up it caught up in on time, that must be like five years ago, I think it was a bug in Docker itself. But, uh, yeah, we do have, you know, a big, big NFS storage and we have policies for back. So I do think we are covered there. But you know, some of our data centers use tapes for backups, it is a lot of storage. But I don't know we have software to the operations I can artists to them, you know, as artists don't have root and they won't be able to arrive a show production for example, they have to use some tools to do the Disk Cleanup and etc, that check their permissions and whatnot. Everything is tracked, or almost everything is tracked. Sped I don't know if you wanna
James Spadafora 32:12
I think I think you covered it pretty well. I mean, we the tape backups are definitely an interesting area. Fun fact about the tape machine. That's actually the inspiration from I don't know if you guys ever seen broke one. The movie? Do you remember the giant machine that took the tape out? And she loves that? In the movie? Do you know I'm talking about and that's actually inspired by our tape machine? Fantastic basement. broke one behind you. All right, maybe that's not Star Wars. Oh, no. That's on episode nine. Rise of Skywalker. Yes. This guy was my first credit on the Star Wars movie. I think James, you signed here somewhere. Yeah, you got you got everybody up there to sign it. Yeah. So that was my first Star Wars at ILM. And yeah, we did some pretty amazing work there. I individual fact side. So that was a nice inside joke, though. James on the getting that in there on the process. For the visual effects folks.
Carlton Gibson 33:10
Just little nerd out moment. It must be quite awesome to be credited on the Star Wars film.
André Prado 33:15
Yeah, totally. there for eternity.
James Spadafora 33:18
Definitely.
Will Vincent 33:21
How does someone get into this? Unless if they don't have a master's degree? I mean, Andre, you said you found your way in? I mean, so let's say someone knows Python? What? Like, what advice would you give them? Or what? You know, how do you go the technical route into why Yeah, well, that what do you how do you screen like, what, what, what kind of so
André Prado 33:42
I think visual effects as a lot of different departments. For example, spad mentioned he started as a technical assistant, which is a TA, you don't really need a lot of advanced Python or visual facts to get there, you mostly need to script stuff, that's usually the entry level of the industry, where you're, you know, you're gonna be tasked with ingesting some metal, some tapes of some data from the client into our pipeline, watching some render jobs, seeing if they're having the issues and bugs taking too long. And then you start scraping some stuff to make your job easier. And eventually you make your way through the pipeline as a pipeline entity with the pipeline technical director, or in core pipeline, our group or in r&d. There's there's a little bit of mentorship there and depends where you want to go some technical assistance assistance end up being going to a tutorial or being an artist, but I think that's the entry level. However, if you do have some experience with software VFX is always hiring people. And if you don't have visual experience, that's totally fine. I'd say just go to like ILM effects careers, you can just put on Google we are hiring right now across all levels to join our team. If you know if you have the traditional backhand full stack experience and Sure you're gonna be a huge asset to our team, and we can teach you the specifics of visual facts. We don't expect you to know that. Yeah, hopefully that answers the question Spadina at something.
James Spadafora 35:12
Yeah, yeah, I could expand on that. ILM has a wonderful culture of fostering talent and empowering people. You know, as Andre mentioned, you know, I started out working midnight to 10. And I, the most Python experience I had was writing scripts, you know, the monitor to farm and Fast forward seven years, and you know, I'm writing Jenko, services, deploying with Ansible, and Docker, all this was learned through Ilm. So we are always looking for people at all levels. We love growing with from within. So apply.
Will Vincent 35:49
Yeah, that's great to have training. I mean, even at Mega corpse for traditional software, it's pretty rare to actually have it. From what I've heard, sorry, Carlton, go ahead.
Carlton Gibson 36:01
Well, I was just gonna say there's nothing there's nothing visual effects specific about, you know, deploying with Ansible set, you know, doing DevOps, you know, spinning up a new server provisioning it, these kinds of things. There's nothing, you know,effects about that. So you don't know, you don't have to be an effects artist to do that stuff.
André Prado 36:23
Totally. I do think it helps. Like, if you had, you know, you know, how a VFX pipeline actually operates, or you can talk with artists understand their pain points. Right. But it's something you can learn in the job. And, you know, we have product owners for some of our products, internal products that will be that interface with artists and technology. Yeah, you got to talk with a lot of people. We're very collaborative. So yeah, if you are interested in this domain, I can say it's pretty fun. You know, I wasn't even aware it existed. And I absolutely had a blast working on this domain, or have it's not your boring crud app that a lot of tabs complain about?
Will Vincent 37:05
Oh, yeah. All the web devs. After a couple of years, it's kind of like, the same thing. What was so the career thing? So? Because James, so you've been at ILM, your whole career? But Andre, you've you started off in hardware, you've been at a couple other visual effects? Places. I guess I'm just curious. So in traditional, you know, in web software, people kind of bounced around a lot. It sounds like maybe at ILM, it it's not so much the case. But you, Andre because you came in a different route, you've made that transition. So I guess I'm curious what, what causes someone to move around, right? Is it new challenges? Is it new learning environments, or what's I assume salary as well? Like, what what's give us a sense of that world and the career tracks? If you're not from the inside? I don't know if you if you're not inside Ilm. So you've like, how did you first get into VFX? And then you've you have been at like, when you gave your talk, you were at a different effects house?
André Prado 37:57
At method studios? Yeah, I was there for a year and a half the time. Yeah, I don't know, I think I was just looking for a job. I was working on a startup at the time in London called Open scan, we were having some troubles financially. And I started looking for for a job and, you know, Synchros, posted on StackOverflow careers. And I was like, Well, I can use my Django and Docker and Python to make Marvel movies that seems sane, like for me was like an instant apply. And I just wanted to apply the knowledge I had as a software engineer into the visual effects industry and for me, was just mind blowing, that I was able to do it. And you know, when I joined method Studios is a little bit of a shock, because I think visual facts doesn't have a lot of people from outside the VFX world building software. And like, it just felt like the you know, they're not following the best practices, etc. But they were very open for it. And I think, you know, did a very, it was really awesome and rewarding to push the company forward, the software engineering round. And I think the same at ILM, too, you know, when I joined if you there, there was a lot of great stuff in place. But there was a lot of great opportunities for improvement too. And I think that was the challenge of how do we bring this, you know, iconic 40 years old company to the future with Kubernetes cloud and all the buzzwords and we we did it right? We are running workloads and Kubernetes in the cloud. And it's just for me has been amazingly rewarding. Being able to contribute to the companies while making visual facts right? Like our product is actually something that I watch at night when I go to cinema, and I enjoy it and I can say hey, I saw this before I know who worked on this in the credits right I think it's.
Carlton Gibson 39:44
So you're actually reading. The credit Scopus is always always staggers me the number of people that are involved in making, you know, even a half hour program on you know, whatever, but a full movie. It's just like, hundreds and it's just super mouse subs.
André Prado 40:00
Of the people like usually not everybody gets credit, it's a little bit of a, you know, the client offers certain space. And then you put the people who contribute the most and whatnot, there's a lot of uncredited people, too, that are also a crucial part of those movies. It's an army, it takes an army.
Will Vincent 40:16
So what would be so, you know, people who use Django, but are somewhat outside? what would, what would improve things for you, if Django like a sink, like if you could wave a wand and add something to Django or to a web thing, what what specific to your workflows would would be helpful, I feel like, we're still trying to figure out how to handle the UI side of things. You know, we build a lot of our front ends and react, and then use, you know, Django rest framework for interacting with that. We don't use Django templates too much.
James Spadafora 40:52
You know, I feel like there's some areas that we want to explore on that front, like I heard you guys talk about HTMX, a couple episodes ago, htm, or HTML. So I think that's, so I think that's just one area we need to explore a bit more.
André Prado 41:11
I think from my side, when I see visual effects people trying to get into services is can be a little bit intimidating. You think, you know, the deployment, for example? Or do I need unicorn, I need to know engine X, I need to like, I feel like flask, for example, makes it very easy to have like 10 lines script that does the one function that people needed to do. And I really love how Django is batteries included. And people don't need to think about that. But almost like a want. I would love if Django could also have like a simple mode or something like that. That makes it you know, it you know, I think maybe in the Quickstart slash tutorial of the Django page, you could have another one that is like, you know, single page application with Django or single python script application with Django, because you do have that in the dock somewhere? I don't know I think for now, yeah.
Will Vincent 42:01
Well, two things. One is that we, we, we Django couldn't, should improve the official thing. And the second is, Carlton gave a keynote, demonstrating this, and I have a package showing, actually, you know, instead of six lines, that's what 15 lines to do with Django. So it's possible, it's just not typically used in that way. So it's more of a marketing problem, I think, than a Yeah, anything else, which
Carlton Gibson 42:29
there's two, there's two templates I'd like to create so that when you do start project, you can do these, and they'd have to be just put on GitHub, I just haven't got around to actually putting them on GitHub. And then you could point the start project command to them. But one is single file. And the other is single folder. Because you can get that sort of sync when you start app or start projects. Or you can just turn it into an app by adding a models file and adding the name of it to installed apps, and it turns into a fully featured app. But you haven't got then the project folder, and then the app folder next, which is it seems like an awful lot of files for just one model that I want to create, like, why is there so why does files here like, you know, obviously, is the project scales that starts to you know, that starts to pay dividends. But I think there's a lot of room for these kinds of too, because the other the other example that I that makes me think about this is people doing data science with pandas and NumPy, and all the rest, and they've just they got a dataset, they got nothing, they just want to quickly BAM it online, and they end up using fast, which is great. But from a Django perspective, so why aren't you just using Django? Well, it's because I have too many files. Like, ah, we should have a better story there. I think. Yeah, there's a bit of analysis paralysis, even. How do I name stuff every time I started, Django is like, should I name the project? The same was, like I always got into that rabbit hole was like, Yeah, one thing we started doing in our team was setting up a cookie cutter template. And I'm hoping that crosses to all the other teams so that way, our projects are standardized, and everything looks the same. That does help us as well. Because you get to you get paid people able to move across teams and work on a different project. And because it looks the same, they don't spend that half a day going. Where is everything?
Will Vincent 44:15
100% like, yeah, we we've recently had a discussion about starter projects in Django, because there's a couple couple that are paid now that usually had billing and stuff, but there's cookie cutter, and then I have one, Django x and I think Django itself if there was a starter project that could be deployed, because part of the problem is, you know, it's local development in Django, and it's quite the, since I teach this, it's quite a leap to get to deploying something and that loses a lot of people. So even if it was something that was just a switch, you could toggle that had some basic stuff that you could deploy the what is it the polls app, Carlton, you know, like, there is a yeah, there is probably a way to To do that, or certainly to have something where it just environment variables is just switches debug secret key and allowed hosts. Which gets you most of the way there.
André Prado 45:10
Yeah, and it's, it's interesting to err on the low in the barrier of entry. Right? I think Django does a great job at saying like, you know, hey, in production, change these, hey, this is not safe for production. But then when you have people just starting off, they're a bit scared and intimidated, like, Oh, my God, I'm doing my local dive, but none of this is safe for production. So like, what am I doing? Where do I go from here? And I think, maybe just saying the consequences to like it is okay, if your use case, you know, it's simple, you're disconnected from the internet, you're doing some local dive in a data center, as, you know, data scientists or visual effects artists, like, you know, we're not touching the internet from our data center. So a lot of, you know, matters, but doesn't really matter. As much as if we were open to the whole world with our app, we're really trying just to, you know, develop a microservice to help with this workflow thing here. That could just be a Python script in a server that I SCP and, you know, write a system did it start? I don't know, I feel there's a lot of potential to simplifying. And
Will Vincent 46:07
yeah, yeah, there's a, you know, there's the docs versus tutorials, dilemma, which is that Django has docks, tutorials are separate what, you know, people want tutorials. It'd be nice to have Django tutorials, but they're a little bit more opinionated than the docs. So
Carlton Gibson 46:26
also, there's that problem with open source projects in the people people think of as project like Django and they think of it like an organization like Industrial Light Magic, and is anything but it's like half a dozen people giving their spare time. And you know, it's it's not it's not a corporation, it's not a thing that has capacity to do everything. So it's a question of what's done all the time. And it's interesting, but yeah, all right, is worth we don't house us. Is that Is that what's the business case? Thing is? Right, right.
André Prado 47:02
And the thing for which affects on machine learning and data science as well. And, you know, Django has started recently, or maybe haven't followed much, sorry, but I'll be a sync stuff and doing long running processes. For example, we usually use Django of salary and setting that up, and the infrastructure can be a little bit too much. If there was like a simple thing to that, you know, maybe baked all of that in and you know, not performance, a salary or scalable, but at least it you could run some a sync stuff and some sort of cues. I think, you know, fast API has class that does that. I think tornado has something that does that, too, because it's just a sink, io and flask. Granted, I haven't been following that too much in the Django side, it might already be there. But I think that we have
Carlton Gibson 47:47
No I mean, Starlet, the framework, which underpins underlies fast API allows these background thread background tasks, we can just sort of spin off and go and what it doesn't have is like the retries, and the, you know, task acknowledgments and all those kinds of picture things, which is when you need a proper cue. And yes, Celery is the big beast in the room. And it's hard to it's old and skate, gnarly and whatnot, but it's super powerful. If you can get it. It's a difficult it's a difficult one. There's been many attempts at a simple Q's. But yes, that's another one that's always sort of on the list as a as a goodie.
James Spadafora 48:25
James can I ask with the the permissions on the web GUI is do you? How complex do you all get with your permissions? Are you using like groups in Django? Are you doing it the API level? Like, how are you setting that up? If you know?
Yeah, so typically, like our Django apps, they, they'll interact with some of our internal API's. And they usually handle a lot of the authentication. It doesn't, as Andre mentioned, like a lot of our tools don't touch the internet. And we're very open studio. So I think it depends on the application itself. Like, for example, that subscription management tool I was talking about our permissions on that are open so users can log into other users and mess with their settings. But, you know, no one does that. Because it's a internal company. And you know, we don't hire a trolls that go in to mess with those. So. And having that open atmosphere in our tools, makes it easier for us to debug things. So we can have somebody on our production team help a supervisor, just log into their account and see what's going on. If that makes sense. Carla, I'm trying remember there. What's it there's a really nice third party package in Django, where you can log in as any user
Carlton Gibson 49:42
kit. Yep. Django hijack. Suddenly thinking it sounds bad. But yeah, that they put you at an admin staff user to log in as another account via a button in the admin and then they can just sort of log out of that. And it, that's, that's can be really important because you've got a customer who's struggling, and they're like, this is what I'm seeing. And you're like, but it works for me. And to be able to step, you know, with their permission, obviously step into their account and step through the things and see what's going on, that can just I can solve it, because you're like, no, this doesn't happen. This is fine. This works. So the customer, and they tell you actually do it. And then you're like, oh, there is a bug, you know, they've got an accent over the one of the letters in their name, because that crashes the app, because, you know, we were Anglo Saxon programs, we never considered that. You know, any number of problems can come up. But yeah, that kind of person. I like what you're talking about there change about how it's an internal tool and the way you can balance what your threat model threat model is not, you know, hackers on the internet, the threat model is, I don't know, we spill coffee on the server. That's silly example. But you know, you, you can grade, you know, what level of protection you put on by what's realistically, something that your application needs to handle?
James Spadafora 51:10
Yeah, absolutely, we do, we do have some internal checks to keep people from seeing things they're not supposed to. Because we do compartmentalize, you know, some projects we're working on, but for the most part, it's pretty open. And that makes debugging very easy. The software management system we have that allows symlinking also makes it very easy to see what the user is seeing. So if, for example, an artist is working on their own script, and they're like, it's not working, can you help me, it's very easy just to symlink it from their home there to my home, and pick it up.
Carlton Gibson 51:47
And then that's like live coding, right. So you make some edits, it immediately appears back on there. That's super, I mean, you know, the IDE developers are only just getting to that now.
James Spadafora 51:57
Yeah. Yeah, we were looking at um, VS code, was it like, share? A little bit, but since we're air gapped, it doesn't quite work. The JetBrains folks have very similar code with me. I think this is cool. We're close on time. Are there any bits we didn't get in Fink? Just to stress that we are hiring, we have a lot of openings. And if this interests you, please apply in findout. Apply anyways. Yeah. Recruiters will get back to you and figure something out. Well, surely this audience is yours. And you're like, you can use your Django skills to build like, you know, Marvel films. No more. No more boring products work on Star Wars. Yeah. Yeah.
Carlton Gibson 52:43
Everybody apply the link for that in the show notes. Yeah. Now we will, we will for sure.
Will Vincent 52:48
Well, thank you both so much for taking the time. And thank you. I think Andre for I think we connected on Twitter. Someone had put out I forget to tweet about Django in the real world. And you said, Hey, we're using it over here. And that's how this started. So yeah, of course. Thank you for connecting. Sure. Yeah. No, it's been awesome. Yeah, it was, uh, what's the name of the book span?
James Spadafora 53:07
Which one the Django for? Will wills? Yeah. Oh, yeah. Django, Django for beginners, professionals and API's? Yeah, those books were a huge help. So thank you for that. Very happy to hear that.
Carlton Gibson 53:22
We should get you should offer an ILM discount? Yeah. Cool. Now they want she want bulk? I know. I need.
Will Vincent 53:31
Now, this is actually there are I know, a number of companies that for new recruits, you know, my books, and some others, you know, just to onboard people who know Python, but no idea how to do Django there, there's still not that many resources actually. on it. So yeah, many companies want to do that hit me up.
Carlton Gibson 53:49
I think if you want to, if you want to get someone up to speed in, like, you know, sit down, read these. That's the way to go. You know, your books are fantastic when you get to the next show, but then next that follows up the two dx dx VDP Django test. And then there's other there's other materials, but generally people write books kind of more at the level they're at. Whereas to write beginner stuff is less interesting as you progress in your own journey. So yeah, just finished his book. Yeah. Yeah. Anything he writes, recommend.
Will Vincent 54:23
Again, thank you both so much for coming on. We are at Jango chat.com, chatting on Twitter. We'll have notes to everything in the show notes. So see you next time. Bye. Bye. Thank you. Bye bye. Thank you. Bye bye. Thank you guys.