devMode.fm // Standup - "Craft CMS 4: Rector? Damn Near Killed 'Er!"

Episode 125 / 2022.01.31

Standup — “Craft CMS 4: Rector? Damn Near Killed ‘Er!”

Andrew Welch, Ryan Irelan

#craftcms #craft-4 #php

Ryan Irelan & Andrew record another devMode standup where we discuss the impending release of Craft CMS 4.

We delve into why this update won’t be as much work as the Craft 2 to Craft 3 update, but we talk about the work plugin & module developers will need to do, and what you can do to prepare for Craft 4 now.

We also talk about Pixel & Tonic’s likely release dates for the beta and general release, and how a tool called Rector may ease the process for everyone.

We also then talk about how to do long reports or exports well using batches without PHP falling over, a memoization memory leak in Commerce, and Ryan tells us about stakeholder-appropriate workflows.

0 comments

Andrew: Welcome to devmo fm a podcast dedicated to the tools techniques and technologies used in modern web development I'm Andrew welts from and ny studio one zero seven

Ryan: I and I'm Ryan Ireland from http://ryanireland.com

Andrew: And we're here doing a little podcast stand up how you doing Ryan how has the new year been treating you so far.

Ryan: It's it's been good. It's so when we're recording this. It's the almost the end of January and it's a little surprising seems because it's gone kind of quickly.

Andrew: Surprise is that ah in a good way that you've been busy or in a bad way that you have been busy.

Ryan: Have have been busy. We're in so my daughter is a girl scoutswer in like girl scout cookie season right now and my wife and I are the cookie coordinators for the troup which means we're responsible for all cookie things. So it's just kind of a busy thing.

Andrew: God.

Ryan: Busy January for us.

Andrew: Well I'm impressed that you have not hit me up to buy cookies from your your daughter.

Ryan: Oh well I will send you the link after this and.

Andrew: I will probably buy some anyway Ryan let's let's get into it so we've got something interesting that's supposed to be going on in quarter one of 2022 which is that there's supposed to be a beta of this sink called craft four that is coming out.

Ryan: She.

Andrew: And then according to pixel and tonic the public release will be in quarter 2 of 2022 is when they're going to be releasing this so I just want to frame it real quick and get people's expectations to where they should be. They've said publicly that the craft 4

Ryan: And.

Andrew: Upgrade is going to be more like going from craft three seven maybe to craft three nine. You know it's not. It's not going to be like it was going from craft 2 to craft 3 right.

Ryan: Yeah, for the most part. It's not going to be tons of breaking changes. But I guess maybe for the nor the the standard like site upgrade. We're not talking about plugin developers or anything like that.

Andrew: Right? right? So I just wanted to get that off the table for anyone who's looking at it and they're like oh my god this is gonna be like from craft 2 to k craft three it's really not It's gonna be a much more minor upgrade for the majority of people. But there are going to be some pretty interesting changes that are going on but you and I have entered into a kind of betting pool on when this thing is going to be released right? We we have no inside information on it. My guess is kind of crazy. So my guess is that the. Actual public release will be on April Fourth and I'm basing that purely I'm betting on Ocd I'm betting that Brandon or whoever at pixel and tonic they want to release craft four on the same day that they released craft three a number of years ago.

Ryan: Oh so Brandon did say two months and three days is how long people will have to fix their plugins and everything in between someone had asked.

Andrew: That is what I'm betting on.

Ryan: How long will plugin developers have to get their stuff in order to support k craft 4 and he said if everything goes as planned. You'll have two months and three days

Andrew: Right? So that means if we if we if everything goes to plan and they stick to that that means that the beta if my date is correct which I have no yeah I have no faith that is going to be correct. But if it is correct that means February first

Ryan: That'd be Tuesday. Yeah, yeah.

Andrew: Is when the Beta would need to come out.

Ryan: So we're recording this on Friday the Twenty eighth is that right? So that would be that would be in in basically five days so yes to get it. Yeah.

Andrew: Yeah, yeah. And I feel like even though we have some episodes queued up I feel like I should release this episode on Monday just in case just in case. So it doesn't sound like super old but what is your guess on when that they're going to be releasing this.

Ryan: A good idea. Ah.

Ryan: So oh I have to go get up now. Um I think I said June for the final release.

Andrew: Yeah, the final release and then the Beta release because we had to list both dates right.

Ryan: I think I said June sixth and

Andrew: Look it up I'll edit this out.

Ryan: Let's see I got to go into Discord hang on well I restarted my machine so it was nice and see.

Andrew: Oh now I'm sorry I asked.

Andrew: Um, yeah for whatever reason your video is still shit. The audio is fine. So that's good.

Ryan: That's so weird.

Ryan: Yeah June Seventh is mine.

Andrew: So you guessed June Seventh let me queue it up for you again. Let me quee it up for you again. So Ryan what is your guess for the the beta release and then also the public release of craft 4

Ryan: That so that's quarter 2 Yeah.

Ryan: So the beta release is March Twenty Ninth basically the the last possible you know point before ah and then the the craft four o final release Gm release would be the seventh of.

Andrew: I see so you make it the last day of q one? Okay, right.

Andrew: Right.

Ryan: June which is um in quarter two now the first month of quarter two is going to be April so I'm kind of betting on them releasing late in quarter one for the beta and.

Andrew: Okay.

Ryan: Not really pushing it two 8 into quarter two. You know once you get into summer here in the northern hemisphere. You know people are just kind of off doing other things so that I mean that could be an argument for not pushing it too late. Um, and I know they're probably.

Andrew: Well I don't know not if we're in lockdown Still I mean you know what are we gonna do you know.

Ryan: We'd never been in lockdown. Um, but there's also the sense that the there's that like post release support wave that comes in where you know the team works really hard at pixel tonic to get it out but they also have this like wave of support that comes after the release. So they may want to pull the release closer to the spring in order to to not have to spend their entire summer doing you know support on like all hands on deck you think I'm overthinking it.

Andrew: I don't know I don't know if that's really a concern of theirs and I also think that this maybe um I think that this release probably will go off better than the craft 3 release from a support point of view because again. Changes are not as drastic as it was from craft 2 to craft 3 but then also they have additional staffing to help with support and they also have Matt on board who's been doing an amazing job with documentation I have the feeling that this release I don't think it's going to be a super stressful release for them I don't think so. Or at least less stressful than the craft 2 or sorry craft three release.

Ryan: I mean I guess but so here's the thing I think we wanted to talk about like steps to prepare as well. But there's going to be a not ah likely a not insignificant number of of instances or cases. Where people do upgrade to to 4 without um, fixing their shit before they get there Well that I mean but bad time means increased support for sure.

Andrew: Well, they're going to have a bad time if they do that. So right? Okay, so let's talk about that Actually um, the reason why I'm saying that I don't think this update is going to be as drastic as from craft 2 to craft 3 Is just looking at what they've done so far they they haven't rewritten the whole thing which is what they did from craft 2 to craft three What they've done is they've made some some really nice changes some content authoring changes and we'll we'll cover some of those and they've made some quality of life changes. But we're not talking about the same kind of drastic upgrade now.

Ryan: It's true.

Andrew: You might say well then why are they calling it craft four. Well it's because they're moving to an annual release Cycle. So every year they're going to release a new major version of craft and the idea is that these new major versions. They'll allow them to introduce breaking changes but they're not going to be the kind of massive. Rewrite that it was from craft 2 to craft three so you're going to have major releases that happen more often and they will have maybe some breaking changes in them. But they're going to be relatively minor and this gets them in step with what laravell is doing in terms of an annual release cycle in terms of what Ph P is doing. An annual release Cycle. So. It's really just a shifting of expectations. So Don't if you're Listening. Don't get this caught up in your mind that oh my God Craft four is going to be this major major hassle as it was going from craft 2 to to craft 3.

Ryan: Unless it it wasn't that bad.

Andrew: And even then honestly the the craft 2 to craft three wasn't that bad for well hold on it wasn't that bad for people who were just working on frontend templating sites. It was a major major change for plugin developers.

Ryan: And.

Andrew: And module developers because they they went from e one to e 2 and they redid all of the guts and that's not happening this time.

Ryan: Right? So so if just to rewind let's kind of do the wavy thing on the screen and the the harp sound. Okay, so rewind back back to craft 2 to 3 upgrade.

Andrew: Nobody can see you right.

Ryan: Ah, that was the time when you actually the the recommended route was to have a new project alongside your craft 2 project get craft 3 running fresh and then migrate your stuff over and then point it at your existing database and run the migrations so that was pretty significant in terms of the. You know the work that it needs to be done. This is nothing like that. This is more like a you know craft you know, basically like a composer update and then running the migrations and of course there's some other things that you need to keep in mind in terms of plugin support and your own readiness with the.

Andrew: Okay, so that's where I want to get into where I think the bottleneck is going to be 1 of the things that they're doing with craft four is they're making the minimum version of ph p to be ph p eight which is great. You know they're they're modernizing the.

Ryan: Project.

Andrew: Code that people can write to use it and it's nice to have more modern standards that it is built on one of the other things that they have done in conjunction with that is they've added what's called typing to all of the classes that are in Kraftcms and let me explain real quick for anyone who's not familiar with what that. Is. Okay, so let's say you've got a function called foo and it takes the parameters bar and baz and it returns woof right? So you would just write. You know it could be in Javascript. It could be in Php. Whatever you would write function foo. And then parentheses bar comma baz parentheses and then that's it right? that that would be an untyped function where we're listing the parameters but we're not giving any types of of what these things are adding types to this what it it would change it where it would be function foo. And then it would be string bar comma array baz and then parentheses and then Colon string or whatever the return type of this function is going to be and the real rub or implication here.

Ryan: The output right? The return type. So.

Andrew: Is that if you have any classes that inherit from pixel and tonics classes which is very common. So if you write a plugin you subclass their base plugin class. For instance, you have to go into all of your code and you have to refactor it. Add in the types so that they match. Otherwise it's just not going to work so this is where I think the bottleneck is going to be because every single plugin for craft three will not work on craft four until it has been updated to add this typing now it is not a difficult thing to do but it is a very laborious. Thing to do and that also applies to any other kind of custom ph p that might be in your project. So if you have a custom module that also is going to be inheriting very likely from at least some of pixel and tonics classes and they they're not going to be fully typed so you can have to fix those too. I think this is where the backlog is going to be when people are updating sites. It's not the the front end twig that they've written or their javascript or their css or or even their their database and all that kind of stuff like that should probably mostly just work and there are a couple of minor edge things. But that that all should just work. Where they're going to run into issues are if they have a custom module. They're going to need to refactor it to add types and they're going to have to wait for plugin developers to update all of their plugins to have types and then to publish their craft four compatible plugins to the plugin store. And I think that is where that is going to be the thing that I think slows down people migrating to this What do you think.

Ryan: So yeah I agree and then people like me with craftquest and I have I have some well the training part aside I have some custom plugins that run critical parts of K craftquest the play Tracker The team's support.

Andrew: Ah, ah.

Ryan: Team accounts that I worked with Ben K Croker on there's there's a there's a few things in there that I would have to go update update those before I can move to craft four and that's fine because I wrote the code I maintained the code I run the whole project. But for for companies that that. Don't have either maintenance contracts with a ah client or or so forth like there might be these one off plugins out there too that need to be updated. Um, and it's not that you're gonna have to like completely like rebuild things because the Apis have all changed but but still.

Andrew: Right.

Ryan: Like someone has to go in and touch that code for it to work. There is also the piece of where you hosted and does your host support Ph P Eight and.

Andrew: You know I I think that's less of an issue these days or it should be less of an issue these days if you're using any kind of a modern hosting environment. You got your own vps and you can pick what version of Php if you're using forge it's trivial.

Ryan: Right.

Andrew: Absolutely trivial to just pick a different version of Ph P In a way you go and served is the same way. Ah Arcis tech all of these places. It's relatively easy to do that I think if we're talking.

Ryan: but but I feel like we have a little bit of bias though because we like to use good tools and platforms. And yeah, what about that vps that was spun up six years ago that still runs a bunch of sites and.

Andrew: Yeah, no, no I prefaced it I said if you are using Well this is a good excuse to get on board and start using something that will actually make your life A lot easier. As we've discussed before I transitioned to client from using their own very complicated custom Aws setup to use served and man their life is so much better So Much better, but it.

Ryan: Yeah, so yep I actually have that that is a bullet point. Um, for today's episode but ah where I did the same thing they were editing on ah on a staging server like directly on the server and then copying files.

Andrew: Yep.

Andrew: Yeah, that.

Ryan: Over to the production server project config was was disabled um and they didn't really have there was no like way for someone to to properly work on the project. So I took the whole thing. What's that.

Andrew: All right focus Ryan focus. We're not talking about your pain right now we're talking about craft four I know you're in in pain on it. But let's give people Adam you you look pained Ryan you look pained are you okay Ryan you doing good. Okay.

Ryan: Okay.

Ryan: I'm not in pain. Ah, um, good. Yeah.

Andrew: Is your your ring hanging on your finger good. Everything's all right? all right? fantastic. But so yeah, yes, it is true that you have to make sure that you have some kind of hosting that you can run PHPEight maybe I am biased. Maybe my head's in the clouds. But I think that's going to be less of a holdup just because the state of things today.

Ryan: Everything's good.

Ryan: Right.

Andrew: Just so much better than it was a long time ago in terms of being able to just spin up something with whatever you want is relatively easy and if you are stuck in a situation where you're on a shared hosting environment and you're unable to switch php versions or your. Locked in at some crappy old version of Php. It's a great time to look for better options because there are many better options out there. We listed some of them you you can do that too. But you know it's specifically talking about what it's going to look like upgrading to craft 4.

Ryan: So you can also just stay on craft 3 until you have the opportunity to move as well. Yeah.

Andrew: I do think the biggest bottleneck again is going to be any custom code that you own a plugin or a custom plugin a custom module that will need to add typing as you mentioned it doesn't require new Apis or there aren't breaking changes in the apis it's more like going in and and correcting the the grammar.

Ryan: And.

Andrew: In a way like you just have to add what you used to put in a dock block. You will now put in actual code in terms of typing and dock blocks are going to be a little less used for that I would imagine going forward and then you're going to have to wait for all the plugins that your site depends on to then be updated to go along with it.

Ryan: Right.

Andrew: Pursuant to that we did an episode on devmo fm a while ago about what even is ph p and one of the things that we talked about was a tool called Rector and I've got to refrain from making the rector damn near killder joke. But anyway that always reminds me that I think it's a terrible name. Um, it sounds too much like rectum Ryan it just does yeah in in any.

Ryan: So it sounds like a lot of things. Um, so so rec rector is instantly upgrades and automatic or automated refactoring of any Ph P five point three plus code and okay.

Andrew: That's right, Essentially, it's sort of like babel for php in that you can take your code tell it what your target version of Ph P is and it will modernize it and I believe it also will go the other way. It can take modern code and downgrade it. But I please don't do that like please just keep moving forward.

Ryan: So.

Andrew: You know and Brandon has mentioned that they are looking at rector and they've been in touch with the the author of rector. Worst case though, they'll work on their own tool but their goal is with the beta they're going to ship a tool that you run it on your plugin or module. And it will add all the typing for you just like magic. It's not going to rearrange your code. It's not going to be dealing with you know, rewriting Apis or anything like that. But it will at least add all of the typings into it which is a really laborious and error prone thing to do by hand assuming they come out with that. Think that should make plugin developers lives a lot easier now I have not seen nor use this tool and I'm very skeptical when it comes to magical tools that I've never used but it sounds good. You know and that.

Ryan: I Yeah I'm just looking through the through through the read me I guess their holdup is that it doesn't support ye out of the box yet. Maybe.

Andrew: I don't know I don't know what the the hold up is I just know that it's something that Brandon has mentioned that there's something that they plan to solve in some shape or form and it prop.

Ryan: It right now I I miss I miss red it does 2 different things it ah it helps upgrade from page p five point 3 to 8 and it helps you upgrade these major frameworks like symphony wirevel cake doctrine so forth. Okay, so those are 2 different things. So.

Andrew: Yeah, yeah, yeah, right? because the typing is only being added to crafts classes. So it's only when you inherit from crafts classes that you have to make sure that this typing is there because I have to match the method signatures.

Ryan: We don't even need ye support in this case.

Ryan: Right.

Andrew: Need to match. Otherwise it's it's not ph p is gonna throw an error so assuming they come out with this then it becomes a matter of the work that's involved in doing this now me as ah, a plugin developer I'm the exception in terms of people who who use k craft. Usually people are writing in twig they're using css maybe kind an occasional site. They have a custom module. Maybe if they've got a commerce site. They've had to write their own adjusters or you know a little bit of code here or there but I'm kind of the exception in that I got a lot of work in front of me right? and I got a whole bunch of stuff. Needs to be upgraded I don't know what that work is going to look like I don't know how much this tool is going to help you know transition it and the other thing that I have to keep in mind is I will need to fork the code base now all right? So I'm going to need to have a craft 3 fork of the code. Actually I'm going to have it as a branch and then a craft four branch of the code because I'm following pixel and tonics methodology which is that I'm going to support ah the old plugin that runs on craft three for a year of you know regular updates. And then I'm going to do security for 1 year after that. But I'm going to have to have a relatively I guess it's a short term plan for maintaining both right? and.

Ryan: I would craft 3 version and craft four version be in lockstep in terms of features or would you or not necessarily.

Andrew: Ah, not necessarily I mean it depends it really depends on the work that's involved because if they are lockstep in features what that means is adding the code in 1 place and then running Mr. magic upgrade tool or it means adding it in both places and.

Ryan: Right.

Andrew: Doing all of the changes and commits and stuff that's involved in there now. Why am I going into this Ryan if I'm the minority if I'm the minor I don't know if I'm the minority and most people aren't gonna have to do this? Well I'm going into this because even if i'm.

Ryan: I Don't know I'm just messing with you? yeah.

Andrew: The minority of people that need to do this. The majority is going to be waiting on people like me to get all their plugins updated So I'm just kind of letting people know what is involved in doing that. So in addition to.

Ryan: So you're like you're you're like Alexander Hamilton History has its eyes on you because um, you know you know the musical anyway. Um i.

Andrew: Brian it's ironic you should bring that up because my direct blood relative shot and killed Alexander Hamilton

Ryan: You Aaron Burr your related Aaron Burr really

Andrew: Yeah, absolutely yep, Yep Yeah, my great grandmother's last name was burr.

Ryan: And he's like brought the podcast to like a screeching halt I have to think about this for a second.

Andrew: So don't worry and by I think by um I think it's very likely that I'm also related to Comedian Bi Bur in that sense. He still has the last name but I'm pretty sure I've heard him mention that he's related to Aaron Burr as well. Yeah.

Ryan: So you guys have the same haircut you and Bill Bur

Andrew: Anyway, Yes, see it's genetics. You can tell we're both bald I'm not I'm not ginger but you know whatever.

Ryan: Ah, so so ah before you just kind of rocked my world with that. Um I know the reason you bring it up is and the reason I joke that all like history has its eyes on you is because seomatic one of your plugins is popular. A lot of sites run it and. You know, Obviously that's you know that's a fair amount of pressure right? to make sure that you get it updated so people can update So You're kind of a blocker in this sense or plugins are blockers. But yeah.

Andrew: Right? right? But it's not just me. Yeah I mean yes, it's me because some of the plugins are popular and that's fantastic I'm very happy that those are popular plugins.

Ryan: Right? But it comes with a certain amount of responsibility then in terms of updating. Yeah.

Andrew: Right? And it's not just the paid plugins right? So this is where the the interesting wrinkles I think start to come in is that there are a number of unpaid plugins some of which are not as maintained and if your site is using some of these plugins. It might be the time to start, you know.

Ryan: And.

Andrew: Ping the people that maintain them to ask them what their plans are for craft 4 when you're looking at potentially migrating these because again every single craft three plugin will no longer work with craft four and it's not because there's some major drastic Api Change it's because of these typings that they have added. Try and modernize the codebase and it's also probably just a good time to you know, kind of ditch anything that is no longer maintained. Anyway, you know if you got a plugin.

Ryan: So yeah, there, there'll be a fair amount of pruning that happens I think for sure. Um, there will also be you know things that are abandoned plugins that are abandoned new people will come in and either replace that functionality with their own plugin or um. Ideally ownership of ah of ah, an abandoned plugin is transferred if possible. So There's like a kind some continuity there. Yeah.

Andrew: Right? So it's just something that you need to consider when you're looking at upgrading your site for craft four and I'm letting everyone know the work that's involved in doing it because it's not just changing a version number and then the craft 3 plugin will work on craft 4

Ryan: Right.

Andrew: It's also not as bad as all of the Apis have changed but there is this kind of drudgery work of adding all the typing to all of these functions in order to even get it to run on craft four and then if there's any breaking changes that you have to deal with and there there shouldn't be many then you have to then address those. So I think it's important for developers who use Kraft to understand that this is kind of what needs to be going on behind the scenes and understand that plugin developers are going to be having to do this work and it's it's not tons of work. But it's not necessarily trivial work either that needs to be done to get these things updated. And I think that's going to be what holds up people upgrading to craft 4.

Ryan: I yeah I can definitely see the the the uptake on upgrades to be if they released in the spring to be slow over the course of the summer as people kind of wait out that initial period and then probably pick up pretty rapidly over. During the fall and then weeding into 2023 which is kind of funny because on a year or two release cycle and I know the phhpeight thing is is a bit bigger than probably what they'll do with craft five in terms of what plugin developers have to do but with the year release cycle like it could be that everybody is delayed three or four months like you know, craft 5 comes out maybe next spring 2023 but people don't really upgrade until 3 four six months after depending on the project. Um, and then the next version is out so but that's just you know what was out in John say man circle of life just how it goes.

Andrew: I thought that was from the Lion King oh yeah of course of course in in any event I think this is in any event people may look at this and they may say oh.

Ryan: Yeah, Elton John wrote the songs for the lion king on man.

Andrew: Yeah, you know that that sucks that you know I'm gonna have to I've got this old plugin that hasn't been ah updated in 3 years like what? what am I gonna do I do think that there is an upside to this I have long said that as a plugin developer I feel like I'm in the support business this is going to definitely bring to the fore. People that are supporting and updating their plugins and people that are not and that probably will influence what plugins you decide to end up using going forward now. Situations change people change jobs they move on. They use different systems. So I'm not. Pointing out that anyone who's not maintaining their plugin is is doing a bad job or anything like that. It's just you know the nature of things but this can at least in the the near term inform your decisions in terms of what plugins you're going to use or not use right? I mean maybe you'll you'll say well maybe we can figure out a way to do this particular thing without a plugin.

Ryan: Right? or or use no plugins at all.

Andrew: You know.

Andrew: And yes, you you could do that with no plugins at all I think you might have a little bit of a problem doing that and I also think that there's nothing wrong with relying on the functionality of a plugin if it saves you time you know and.

Ryan: Right.

Andrew: If It allows you to bring your value where if it allows you to spend your time where you're bringing your value I think it absolutely makes sense to use it I think some people are a little overly paranoid about you know code debt and using someone else's stuff and all that again I think it boils down to. Work with people that are actively updating their plugins and I you know I may be hanging myself by saying this because I got a whole bunch of plugins that I'm gonna have to update I'm going to do my best to get them all done with as much alacrity as I possibly can I'm going to be honest I'm going to start with the paid plugins I'm going to do those and I'm going to go through them.

Ryan: Yeah.

Andrew: Probably based on their popularity because I'm going to be hitting the the broadest crosss section of people who have been paying me money to make this thing and maintain it for Them. So I feel that's where my obligation is first but my plan is I'm very likely going to upgrade every plugin that I have. It shouldn't be too terrible to do but we'll see you know if it ends up being awful then I don't know maybe maybe some of them will go by the wayside but I I kind of doubt it.

Ryan: So yep, yeah I was starting to think yesterday about the the plugins I have out there that I don't realize especially ones that I use myself or that are part of a like ah some course material or something like that just you know if you start to realize like wow I I have a lot of.

Andrew: Yep.

Andrew: Right? And I think this is okay again, it gets people to rethink. Okay, you know what does our site really need and as you mentioned before you also don't necessarily have to update your site from craft 3 to craft four. It may be that.

Ryan: Surface area out there that I need to think about? yeah.

Andrew: Some of your long-term clients you'll you'll keep along and you'll upgrade for all the security updates and all that kind of good stuff and some of the others you'll just kind of let them stay where they are until they do a major rewrite at which point you'll use. Whatever new version of Kraft that there is. In general I encourage people to update to the latest I think it's the best way to avoid security issues. Not just your cms not just your plugins but everything you know the infra that you run on which is again a good reason to be using a more modern hosting environment that you can pick modern versions of.

Ryan: I.

Andrew: Php node whatever because they also sunset those in terms of security updates. So you you do want to keep those updated to some extent. You know we're on an endless treadmill when it comes to updating these things.

Ryan: So and that's that's my kind of my like old guy pushback against the idea of not taking on any technical debt or trying to like make the absolute best decision for for when you write your own code versus use other is that. We are in the business of creating things that the moment we create them. They start to disintegrate right? that that is you think about? Andrew you've been working in software for decades and not always on the web but still I mean even the client stuff that you did like that could have it could be even maybe the web has. Ah, much longer window of support than a lot of the client stuff that you did um but you know like projects that I worked on beginning of my career twenty years ago it don't run on anything that you know right now like it as soon as you build it. It starts to disintegrate and that's just you just have to kind of.

Andrew: Right? right.

Ryan: A little bit mindful of that and not take things too seriously make the best decision you can in the moment for the person that you're doing the work for even if it's yourself and then and then move on.

Andrew: And yeah I'm going to veer off of this detour pretty soon. But I do want to add that there is somewhat of a fallacy I think that you're eliminating tech debt by not using a plugin or you know a third party npm package or whatever because.

Ryan: Right.

Andrew: What that means is you have to replicate that functionality yourself. You have to write that all yourself you have an eliminated tech debt. You moved who the person that maintains that tech debtt is and I used to do tons and tons of sites.

Ryan: Yeah, yeah.

Andrew: Where I would do the seo just using k craft fields and inheritance and but it was a major pain in the ass to try and keep those all up to date when I found changes in the seo landscape and I updated how everything worked on 1 site backporting that to every other site was just a. Major pain in the ass. That's one of the reasons I made a plugin because ah there is a well-defined system in which this thing can be updated and then everything that uses it will just magically get all of those features so that's another thing to keep in mind when you're updating to craft 4 you could go off on your own and and rewrite some of this stuff yourself, but just understand you're you're not eliminating the tech debt you're moving where it is sometimes that will make sense sometimes that won't but I think that stepping back to the overall idea of updating to craft 4 again I think plugins are probably going to be the the biggest thing that you're going to be waiting on and doing that we are going to have a about two month window to be working on stuff I would not expect that every plugin is going to be done in that timeframe.

Ryan: Right? right.

Andrew: I Can tell you for myself. You know, part of my day job is doing support for people that are using my plugins now fixing bugs now working on stuff now and then if you add on top of that the work of updating these things to work on the new system. It's going to take a little bit of work to to put that all in I'm going to be putting a. Ah, Kabash on the client work as soon as the Beta comes out you know so that I have the the time to do this stuff but it will take a little bit of time to do that and then beyond that as you mentioned the hosting got to find somewhere that runs PhHPEight or lets you switch to Ph P Eight and then other than that.

Ryan: Yeah, right.

Andrew: You know it looks like it's going to be in theory relatively minor. You mentioned something about deprecation errors and.

Ryan: For yeah, um, so if if people that were around when during the craft 2 to craft 3 trendop sorry for people that were around during the craft 2 to craft 3 transition. One of the big things was that a bunch of functions and twig. Tags and um, other things were were deprecated in craft three but then they are they weren't going to be removed until craft four. Well craft four is here and if you look at the commits you can actually see those like commits where they're actually removing. Support instead of just you know, leaving the method in there logging a deprecation notice and then just calling the new method which is how they you know how they did it I have an article where I showed like an example of that. It's just gone. So if you aren't on top of your deprecation errors in your craft 3 project. Go to the utilityities section to deprecation and then ah look at them then you're not gonna build upgraded craft 4 or you you can. It's going to break all your templates are going to break. You'll get hard errors. So Andrew there is a way to um work. To get hard errors now on those deprecations instead of just logging a notice What do they have to do to do that.

Andrew: I yeah and I'm kind of happy about this because this is actually I did a pull request to enable this because I'm the kind of person I think Brandon ended up rewriting it in some way so I don't think it's actually in there anyway. But the functionality is there.

Ryan: Nice, nice.

Andrew: That in your um app ph p your config app dot ph p you can set the deprecator to throw exceptions. So it's not going to log them It's not going to add a database query and entry for every single one that you've got there. Instead you're just going to see an exception thrown on the front end template and that's something that I always have turned on for local dev I don't allow myself to just let these deprecations build up because it it always gets to the point where the project has to ship. And you've got 38 deprecation errors and you're like well you know we're just going to ship them with that in there I always leave it in local dev I always leave it on hard mode. We'll link to the article in and I think ah in Kraft's official little guide on preparing for k craft four. They also mention it there.

Ryan: Yeah.

Andrew: But it's really just a simple config change and what's going to happen is it's going to throw deprecation errors in your template So You basically it's kind of like gandalf thou shall not pass right? like it's just gonna error if you hit any of these deprecations I think this is the best way to make sure that you are forced. Fix these things and I always leave it on So. The nice thing is my sites that I work on like I'm gonna have almost no work because all of those deprecation errors are already fixed.

Ryan: Yep I know that your craft starter project that I used when I migrated craft quest over to the vet and Docker setup has that enabled by default. So it'll it'll through all your error So fix your deprecation errors.

Andrew: And everyone and everyone who mocked me Ryan everyone who said oh I'm not going to turn that hard mode on I don't want to throw deprecation errors your day of reckoning has come.

Ryan: Um, start now with.

Ryan: Ah, ah yep, So so fix those we'll put some links in the show notes to the the craft cms their official getting ready or preparing for craft for and then there's a what? what.

Andrew: Oh wait a minute wait a minute Ryan wait a minute you have an article called getting ready for k craft four which I think it came out after pixel and tonic released a preparing for kra for article. Ah.

Ryan: I Yes, that's correct. Well I technically I mean correct it came out after I had been sitting on it I had written it um like a few weeks ago and I was like well I'm just gonna wait because it might be confusing.

Andrew: Ah.

Ryan: People might get waried but and then I was like I'm gonna wait till February to publish it and then they publish was Thursday yesterday. So um, but it's fine because we all have different takes and more information the better but I will put a link to all of the articles including mine.

Andrew: Oh man, this is gonna kill you so I just googled kraf c m s 4 upgrade guide right? Well I did anyway, that's what I upgraded. That's what I googled for and.

Ryan: Who's who who 2 googles for that.

Andrew: The first 3 The first hits are for kraf cms which which have and then my article on up updating k craftfs without comes up none of those hits by the way have anything to do with craft four which is interesting.

Ryan: So of course oh and then yours yeah of course. It K Crafts cms right? Yeah, it's still early right? We're only 24 hours into this it all. We'll get there.

Andrew: Yeah, but in any event I think that's gonna be it I think that the upgrade to K Craft four is not going to be as big of a deal as 2 to 3 I think that there are going to be some plugins that are going to go by the wayside people that don't maintain them anymore. They work somewhere else. they' yak herders somewhere out in Wyoming you know they have no interest in this stuff anymore. You're either going to have to find another another plugin. Someone's going to have to write another plugin or someone can take ownership of it and upgrade it if it's you know, popular or whatever enough or you find some way to do it without a plugin.

Ryan: That right.

Andrew: And then you're just going to have to give the plugin developers that are working on upgrading their plugins to craft four a little bit of time to get around to doing it but 1 of the things you can start doing now is take an inventory of the plugins that you use look at any custom modules or adjusters or. Anything else that you use and and start getting some idea of what it's going to take to be able to move this thing to graph 4 so.

Andrew: So you have anything to add Ryan right? all right? so.

Ryan: Yeah I don't I'm just updating some Winks I like your doesn't edit this out. That's a good natural ending. So.

Andrew: All right? So that's craft four and now I want to talk about a couple of things Ryan one of the things is that I have been doing some client work and 1 of the things that they needed to do.

Ryan: Okay. Me too. Yeah.

Andrew: Is they have a whole bunch of commerce orders and they needed to export them on a regular basis and what it does is it their exporter. They've got a little cli function that loops through and it goes through the orders and it goes through the line items because anyone has never done anything in commerce. Line items are the the various things that are on an order right? and they can be variance and ah any number of other things and it loops through them and it exports them so that their accounting department can for instance, reconcile things or analyze trends or do whatever. And the problem is their export was falling over and dying and what I the first thing I did when I went in and looked at this was I put in some memory checks. So every time through the loop I would put in how much memory is free. What is our maximum memory limit and I ran it. And I would just watch it go up and up and up and up and then it would die right? So I tracked this thing down and so the first thing I did whenever you're doing an export in a large data set what you don't want to do is query for everything right? You don't want to query all 10000 orders because that's going to result in a massive number of order objects or entry objects or whatever you're querying. That's going to take up a ton of memory and you don't really need all of them at once right, you need them just one at a time. So as you iterate through them. You can export them. So one of the first things that I did was I added a there's a method I believe was added in kraf cms 3 point 7 called db Colon Colon batch and what that does is it does a real batch query of these things and when you iterate on that you do. For each db batch and then in the parentheses your your query. It will do a real query on that that will only grab the each record as you're iterating through them so it'll grab a batch of 100 but then it will at it will get just one of them at a time. As you're coming over so the obvious benefit here is you're not using up all the memory for every order because the other way it's eventually going to fall over right? once you reach a critical mass of things you're querying too much memory is used. Php is going to fall over. It's going to die then the next thing that I looked into was. So okay, we're we're querying these things in ah in a nice way. So we're not using up much memory but the other thing it was doing was it was accumulating everything to export in an array. So even though we are dealing with each of order item. Even though we're dealing with each order element one at a time.

Andrew: Based on our batch query. It would then take the exported result of that and add it to this array now again, that's going to get bigger and bigger and bigger as you get more and more orders all in memory which you you don't need right? You don't need that So one of the things that I added is a php generator and it's something that a lot of people.

Ryan: Right? All in memory. Yep.

Andrew: Don't really know about nphp. They also exist in Javascript. The basic idea is that a generator function imagine a function with a 4 each loop in it and instead of doing a return at the end when you've got all the results in the middle of the for each loop. You do a yield right. When you call yield it returns that one result back to whoever called it. But in this case, 1 line of stuff to be exported at which point we can write it out then when you call that function again. It picks up where it left off in the for each loop right. So it's a way that you can have a loop that will write to disk or do whatever with the data so you don't have to store it in memory. So I did all this stuff I was real proud of myself and it still fell over right? still is going to write and the stupid thing still fell over.

Ryan: Oh.

Andrew: And I did a whole lot of spelunking a whole lot of looking around and I found it. It was in craft commerce. Do you know what? Memoization is Ryan.

Ryan: So ah, no tell me.

Andrew: So Memoization is essentially a short term cache and the idea is like let's say I need this thing and it's expensive to get the thing. Maybe it's a query to get all of the line items right? What it will do is in in a in an array property. It will store. Each of those results so that when you ask for it Again. It can just return it rather than doing it Again. Memoization caches are wonderful. They they go away after a request is done so they speed things up for the duration of it. The problem is if you're iterating through all these orders. You've got a memoization cache. That's just going to get build up and up and up and it's just gonna eventually blow up same yeah well,, that's this was actually the actual source of the Problem. So I found it in line items service in Commerce that there what they were doing is they were storing. They.

Ryan: So same problem as before. Okay, the actual problem.

Andrew: In a local sorry in a class property. They were storing each result indexed by the order id so they were storing all of the line items by order id so you ask for give me all the line items if it's in this array. It just returns it. And if it's not it does a query and then it stuffs it into this local memoization cache. It's called well you can see the problem if we're iterating through thousands of orders. This array is going to get huge because it's stuffing more and more of that ah into the cache and that's what was causing it to blow up. So I filed an issue and I think ah they they actually fixed it very quickly. Unfortunately, we couldn't wait for them to release this fix and we also and there also is a little bit of work in in. Upgrading to new versions of commerce they didn't want to disturb their store in the interim so we didn't know exactly when this thing would be released that had the fix in it and we also didn't know when we would actually be able to upgrade to this thing right? So I ended up having to go in and what we kind of in the business call monkey patching.

Ryan: So you had did you have to go into the vendor directory. And oh oh your appointment right.

Andrew: Which is kind of like no no no I can't do that right? because we have a modern deployment system that uses composer so we can't go in and modify their code which you know in the bad old days. Maybe you would do that or temporarily as a hot fix or something you know a lot of people have done this type of thing.

Ryan: That's you yeah.

Andrew: But I actually had to do was add a whole series of behaviors that mimicked what commerce was doing but in a way that avoided the caching behavior that it had so it's something that it was a really long and elaborate process. It took a while to to get it all working now it works great. No memory leaks. Everything works wonderful.

Ryan: Oh wow.

Andrew: Just think this might be interesting to anyone who has to do anything that generates a report on a large data set or exports on a large data set. These are things that you need to keep in mind is the php generally doesn't care too much about memory leaks because as soon as it's done handling the request. Everything is wiped out and it doesn't matter anymore. Well if you have a long running process. Maybe a Cli command you actually do have to care about this so these are things that you need to keep in mind and the basic idea is only have in memory. The amount of stuff that you need to do one step. Whatever your process is and then write it out to disk or send it somewhere or do whatever you need to do with it and then get rid of it and in that way, you're never going to use more. You could do millions of orders and it would still work. You know, have you run into any kind of stuff like this before.

Ryan: I have not um, but that's the ah I actually hadn't used the the Dv batch yet that came out three seven but I did see that there's a knowledge-based article about it from pixel and tonic but I mean you got to do. 3 cool things. Although I guess fixing the memoization probably was less fun than the other two. But.

Andrew: Ryan you and I have very different definitions of cool. Ah, you know if I was up on stage playing a guitar or something I'd be like yeah I got to do something cool I don't know I don't know I don't know well.

Ryan: That's cool. Ah yeah, we don't live rock Star life Me yeah.

Andrew: You know it obviously is satisfying to be able to come up with something that solves a problem and to fix the problem. It was really really tedious actually monkey patching this in a way that it would work because commerce ended up calling this line item service from all over the place. So I ended up with like. 5 additional behaviors and a bunch of other stuff but it works everything works great and it's going to be painless when they actually do release the update and they actually we actually are able to upgrade to it. It's not going to be a big deal. Yeah, it's not going to be a big deal to remove it. But that's it in the indeterminate future and they needed it now.

Ryan: Yeah.

Ryan: You You just pull your stuff out right? yeah.

Ryan: Right? You do.

Andrew: So You gotta do what you gotta do sometimes you know. All right I rudely interrupted you before trying to stay on track about craft 4 but you were talking about some laborious process that you had with workflows and copying stuff and what was that all about.

Ryan: So oh I didn't have a laborious process I was asked to to to help fix ah or a lack of process and with the idea being that it it like creating a an appropriate. Development workflow for the particular person that needed that was going to use this and this person is more of a designer frontend developer in the frontend developer in the traditional sense like pre everybody writes Javascript. So this is to manage.

Andrew: Okay, help me understand to do what.

Ryan: Um, maybe 3 times 4 times a year updates to a craft site where the the templates are kind of like rebranded for a new version of the event and there might be some other like additional fields added and. And then to make it so it was easy for those changes to be done locally and version controlled and then deploy it to a staging server and then deploy to a production server everything before this was manual where they just edited I believe on the staging server directly and then tested their changes and then.

Andrew: Oh boy, oh.

Ryan: Moved everything the file changes over to the production server and recreated like any field changes and stuff like that because project config was turned off and so essentially I was asked to like please bring us into the modern day but it has to be appropriate for the people they're going to use it. It can't be you know. It can't be the whatever something that's super elaborate like Gitflow right? like that wasn't going to work because it they didn't need Gitflow. So um, it's kind of inspired a a course that I put up recently on creating a development workflow but the idea is that it was what I call like stakeholder appropriate.

Andrew: Right.

Ryan: Not what I think is what I would use but it's what would be helpful and appropriate for the stakeholders to get for them to get their work done and it it. We talked about served which is why I was talking about it. It's served for the hosting and that's the reason I chose served is because of. The deployment like the the the deployment ah build and deployment tool and also because I used moved all of their assets which were being directly uploaded to the server disk move those all into the served assets host and then. And then ddev for the local development. So what I created was a github project that I wrote up a read me this person Reddit they got d dev installed and they just cloned the repository and ran ddev start.

Andrew: Nice. Thanks.

Ryan: In the directory and and then then they had and then they had to run a one import command to get the database and then that was it and then they had using the same local hosting urls that I use and that anyone else that colons the project would use. Everybody uses the same one because it's stored in The. Dev Config file they had a fully working local hosting environment that rendered everything properly There wasn't like odds and ends broken right? And then I had it pull from the local volume and by local I mean local remote volume that serves sets up so they well.

Andrew: Local remote volume. Okay, ah.

Ryan: So they have ah they have these different environments on served assets production staging local and you can tell served via the plugin that you install and craft for all assets either pull them from local disk or pull them from the remote environment volume.

Andrew: Right.

Ryan: Which in that this case is called local and that way you are actually getting like assets that you need to render the site rather than you know like oh I don't have like this image or this thing or that thing they're all there and then you can run command line commands to migrate assets between the environments. A really nice setup but the. What kind of the payoff was is that this person was able to set it up and get all these changes done within a week and then we were able to turn around and get that up on staging like via auto deployment and then get it up on production as well actually set up auto deployment on production as well because if. If this person is doing all the testing then it should be there shouldn't need to be like this manual stage to get thing on production just merge into the main or master branch and then push to remote to the origin origin and then and then it deploys out it builds un served and deploys. So.

Andrew: Yep, yeah.

Ryan: Been really nice and for them. It was a huge ah shift in a positive way because they basically saved a bunch of error prone work to update the site with something now that they can use into the future including updates and and and all of that. So. It's. And Roy nice.

Andrew: It's funny Ryan because we work in tech and we think our jobs are so important, but the reality is the vast majority of what we do is just to allow someone else to do something. You know what? I mean like you know they've got.

Ryan: Right.

Ryan: Right? Yeah, exactly? Yeah, so.

Andrew: Ah, important things that they're doing and and the tech for them is just a platform for doing it. So we're just kind of ah a speed bump in the way of them being able to do their thing. It's just like ah a person who thinks building a house is very important. It's nothing compared to the people that want to live in the house. You know what? I mean you're just enabling.

Ryan: For 30 years yeah

Andrew: Yeah, you're just enabling someone else to to do their thing. So anytime you get too caught up in you know I'm in tech so I do important work understand you're you're usually just enabling someone else to do something else. It's usually what you're doing.

Ryan: Well right? And that's what gets me so frustrated about the idea of um, we're talking earlier about like avoiding technical debt where like oh like you know I shouldn't use any plugins because plugins are technical debt where like. That's just a selfish approach and I feel like that we're as developers people that enable other people to do things we if we get too selfish or self-centered about the work or too self-important about the work we do then we're actually making bad decisions for the people that are going to use the things that we create.

Andrew: Yep.

Andrew: I Agree we're bringing our opinions into what we're building for them. Yeah, it be cut if I was building a house I'd be pretty pissed off if the contractor was like yeah we're not going to order your bathtub from Bane Ultra were we're gonna build it ourselves because you know.

Ryan: Yeah.

Ryan: Yep, So that's that's.

Andrew: I Don't know about their supply chain and if something goes wrong with it. You know? Ah, we're just gonna I would be mad. You know I would be mad.

Ryan: Right? Yeah, or we're not gonna build. We're not, We're gonna um, you know these whatever something like you said something prefad that comes for like the roof trusses right? most of those roof trusses come prefab now and they just crane them in. Oh we're gonna build them ourselves like now how about we just crane in the Re after ones because that factory that made those.

Andrew: Yep yep.

Ryan: That's all they do is prefab roof trusses. So maybe we'll maybe we'll do that. But yeah, don't be you know use the tools that that get the job done for the people that you're building it for that's why I didn't like I made this super simple. My my first reaction was gonna be oh well this has to have like a.

Andrew: Right? yeah.

Ryan: Proper like get you know, workflow and all this and I was like you know what this person can just commit directly into the develop branch is not. You know we're not I'm not like you know going to burn down a virus because they're committing into the develop branch so you have to think about those people and what they need.

Andrew: Of course, not so.

Andrew: Yeah, yeah, no for sure for sure. Oh that's really cool and both of us you know in both of the client work that we did. It's just work to enable other people to do their thing in in my case, it's for them to be able to run their business and in your case, it's for them to.

Ryan: Get out of the project.

Andrew: I Don't know I mean run their business in some form right.

Ryan: Yeah, and I was actually enabling a ah ah someone that that does work for the end client even though I was also employed by like the actual client but I was working with another agency and that person just needed to be able to get these changes up in a very timely manner so that the client could. You know launch their event and their registration and all of this So. It's cool I'm doing some cool stuff now on the project with um with ah sorry hang on.

Ryan: So Let's have to respond to something We're good going. Um, the project's going. Well I'm doing some some fun stuff with service workers right now because it is an event website and I wanted to give them better support for low Bandwidth or no bandwith situations like. In an event building. So So it's It's been Fun. So I get to do some cool stuff too. But the focus is always on who are we building these things for.

Andrew: Nice.

Andrew: Don't get too cocky. We're all just cogs in the big machine right? and because I've been doing a bunch of this client work. A lot of this was kind of like oh my God we have to get this done now kind of stuff I still have not come out with my matrix.

Ryan: It's right man.

Ryan: So well I thought that was gonna be the end of 2021 you're gonna have that ready.

Andrew: Facades article Ryan I still think it was now one of the things that I have done is I made a github repo and we'll link to it in the show notes. But it's ny studio one of 7 sorry. It's http://github.com/andny studio 1 of seven slash matrix facades and it's something you can just clone down and with one command you can just fire it up and go into the cp and you can play around with the matrix facades. So this is actually all of the code is done. All of the examples are done I actually have examples of 2 different facades and you can fire it up locally and you can play with it. The article is something that I'm planning to make time for hopefully this weekend or next week because some of the client work should be easing up now that we got this stuff done and I really want to write the article because. Think it's kind of cool but I wanted to mention it real quick that that repository is out there. So for anyone who wants to fire it up. All you need to have is docker desktop installed and then there are literally it looks like 2 commands you have to type you have to clone the Repo and then you type make dev and then you can just log in and you can play around with the matrix facades and. I actually harassed you and made you fry it out so we can guarantee that it works right.

Ryan: So yep, you did and we'll have a link to the video that you and I did about the initial version but I know you've updated now with different different example.

Andrew: I right? I Just added an additional example and I did also have people test this on windows and linux. So It all just works through the magic of Docker you can just fire it up and and play around with it and I. And think it's kind of a cool way to show off an example because you're not just showing people. You know a blog with a bunch of code that they then have to paste in and screw around with you can actually show them the thing working and then they can peek in the code and tweak the code and tweak a working example. And think it's kind of a neat way to to package up and and example, what do you think.

Ryan: I agree and we talked about this that's going to be a focus of mine. This actually came up in another chat I was having but a focus of mine This year is going to be making the example projects faster and easier to install and access.

Andrew: This nice. Yeah and this is a case where Docker actually makes a whole lot of sense because the people consuming it have to know, almost nothing all you have to do is have Docker desktop installed.

Ryan: And think it's important to.

Andrew: And you literally then clone the Repo down and type one command and the local environment is going to be up and running. There's no setup. There's no if you're running Map. Do this if you're running nitro do that if you're running ddev do that. It's literally the same no matter what platform and I think it's a really nice way to distribute. Example stuff and.

Ryan: So Andrew you have a note here that you have a book you might read. It seems like a good way to wrap up the show is to talk about what's next on your reading list.

Andrew: And that about wraps it up for another episode of the demo dot fm podcast if you enjoyed the show make sure to subscribe tell a friend drop press a review. We've really appreciated it for the demo dot fm podcast I'm Andrew welds.

Ryan: Ah, no Nope. Nope. Nope.

Ryan: Nope. Ah, and I'm waiting for Angie to tell me about his book all right.

Andrew: It's you're going to hate it. It's a letdown. It's a book on Webass assembly and I think I'm actually going to read this one. It's a different. It's a different book. Yeah I decided I would really read this webassembly book.

Ryan: And this you brought this on vacation right? The wisem. Okay, all right I So it's not like ah it's It's not a nontechnical book then.

Andrew: That about wraps it up for another episode of the demo fm podcast if you enjoy the show make sure to subscribe tell a friend drop us a review. We really appreciate it for the demo fm podcast I'm Andrew Welch

Ryan: And I'm Ryan Ireland and

Andrew: All right I think that was a good way to end the show that was pretty funny and for for anyone out there if you want to know the difference between silicon and silicone silicon goes in chips and silicone goes in breast implants and I think also in like cawing materials I'm not gonna. Um, not going to get into why this conversation came up but Ryan did know difference between silicon and silicone.

Ryan: Is this is this show all about just outing my my ignorance of all sorts of topics.

Andrew: Now it's about adding my ignorance to we can be 2 ignorant fools floundering around together. How about that all right stopping the backup recording.

Ryan: Or learning it live. Yeah.