Building a Project With Quality in Mind
and not hire half a dozen QA testers 3 days before launch
If I had anything to recommend you when you launch a project, it would be to actually think with a quality mindset. I would it would be best if you got a QA on board on day one, but we all know that it’s often not the case, so here’s a little heads-up of things you can put in place, depending on the project state, to limit those big “crunch and monkey test hours” 3 days before launching in production.
Prototyping
A funny as it sounds, this is probably the phase where we care as much as possible about quality. We get architects on board, we get senior developers, we often get a Lead Designer for all the UI/UX. We need to court the investors, get that funding round that will get us going. In video games, we need a good vertical slice, that means it needs to be carefully thought out. No place for errors, worst case we’ll change something after. We build carefully but since we have an experienced team, things build fast. You can go from zero to hero in a matter of weeks, or a few months.
Often we don’t care about a few things breaking here, as long as the general experience is good. But we can start putting a few things in place for later. The longer we wait for adding quality features (here, mostly automated tests) the harder they are to implement in en efficient way. Because the quickest and most efficient tests are those closer to your code and structure.
Things you can start setting up early:
- Unit testing everywhere! Pad your core features with unit tests. They’re quick to setup, quick to run, and when things will start changing a lot, they’ll be the guardians of your code.
- Contract-testing. This one might be a bit more complex to setup, and you might not see the return immediately but believe me when I say you’ll be grateful to have those in place, especially if you’re working on multiple front-end (web, desktop, mobile…) for you application. A solution like PACT can serve both the need for unit and contract testing and works with many programming langages so you stay close to your code.
- Run those tests in the pipeline. Prevents everyone forgetting about running them in a rush.
- Mobile-first. Unless you are developing a desktop, or a console application, keep in mind that most of your users will explore using their phone. Even with more people doing remote work, the rate of people using their phones is high.
Active Development
Congrats, you got a bunch of money and praise for your project, either from an investor or from the big company that wanted you to try starting that project, now is not the time to let things go sideways. You’ll be hiring a bunch of developers. Thanks to your previous efforts with unit testing, the code is easier to maintain. When something doesn’t work you get an alert (because it’s all in your pipeline).
If the plan is to hire QA, it’s the time to be picky. Don’t hire a tester, hire someone with knowledge that will challenge your teams. You need to plan your feature ahead as more and more part of your project start moving together. That’s where your quality expert comes in. Train them to be your product expert. Have them use the product as much as they can. Your team needs to be able to hear them out. No place for “yeah, but that’s not in scope”. It will be eventually, worst-case, note it down somewhere for later. Having them around, earlier than later will also have your teams think twice before doing something, because there are no safe-net. The quality representative are not there to save your ass. They’re there to help you save your own ass.
On top of the quality practices you already have in place you can start adding these :
- Early integration testing. The systems are starting to talk together. Obviously, if you’re running micro-services, close isolated testing of those services should be enough. But there are a few cases you want to do a bit more. That’s where integration testing comes in place. Put those in the pipeline as well. Control the data (use mocks) and test the core code in a controlled environment.
- Performance/security testing. This is often done way too late. Don’t wait to have 10k users before your back-end can handle the load of weird requests. Keep in mind, it’s always easier to fix early on than later.
- API testing. This phase is all about edge cases. Your team, with or without QA, should be able to think ahead of those case.
- Which leads us to: Planning with examples. Sometimes it’s not a bad things to have an outside view, from another team if you don’t have a QA coaching your teams, to get a step back on how you develop a feature, or how it going to be used. Don’t wait to get a phone call to crash your app on your Android device before thinking “What if I get a phone call while I’m streaming?”
Pre-launch
It’s almost time. Most of your features are working. Don’t get over-confident, things WILL break. But most of it should be minor fixes at this point, quick minor fixes. But things can go wrong quickly and to help you react as quick as you can, there are a few things you should try to implement.
- Monitoring. At this time, you should have your production environment running, even if they’re not public yet. Time to target potential weak points and monitor them with a custom tool or a outside service. Monitor services uptime, endpoint availability, page load times, crash rates. It will help you act quickly if something unexpected, or if you get a DDOS attack for some reason.
- Set up a status page. You’re down because there’s a service outage somewhere? Even if you’re not responsible, keep your customers informed. People are less likely to complain if they see “Oh, they’re aware!”.
- End to end testing. Just to make sure everything works as you expect them. Don’t put too much efforts here, that kind of testing is often fragile and unreliable in my experience. They’re valuable for the main path your users will take, but any edge or special cases should be handled in the previous phase in the integration testing.
End of project
You’re live! You will add features over time, and maintain the rest. If you did all the previous steps you should be in a pretty good spot, even if you have none or only a few QA on board. It’s all about relying on your team’s confidence rather than expect somebody to play goalie. If you expect the latter, there will always be a puck that will get in the net.
Also keep in mind that if you applied all (or most) of the testing philosophy early on in your project, it’s easier to maintain and pass on to the next generation of devs that will touch the project. If everyone thinks it’s slower to test and should deactivate most of them, you failed.
If you want to add a few quality gateways here, you can consider adding:
- Visual tests. You don’t usually set those up early one because they’re probably the easiest to break, and will lower your confidence in all the other tests you set up. People are like that, 1 test can break the confidence for the whole test suite/types. Usually visual tests will handle the unplanned weird data that comes in. They’re really there for end-of-line, minor visual tweaks.
- Adding completely new features should follow the same path than the project as a whole. It will keep a healthy code-base, happy customers and happier teams in the long run. It’s like doing sports. It’s hard to get it on, but once the routine kicks in, you’re happier overall.
Agility changed a lot in the mindset of IT projects worldwide. We’re nowhere near where we were 30 years ago. Hopefully you’ll staff up with quality experts (doesn’t need to be a QA, you can have a dev with that knowledge) along the journey and deploy with the confidence at 4PM on a Friday before a 3–day weekend.
