At some point in your QA career you will inevitably test out a new feature, and come up empty-handed for bugs. We’ve all been there. It may even continue for a few consecutive builds! If it continues for much longer than that though, your work becomes boring, and you may even question your place on your team. You’re not finding any bugs so QA clearly isn’t needed, and you question why you’re here at all. You slowly become demotivated but keep plodding along anyway.
Suddenly though, it happens — you find a bug.
It’s small, but it’s a bug nonetheless. You’re overjoyed to have found something and write it up in immaculate detail. You go back to testing, and suddenly you’re finding things left, right, and center. You’re seeing the app in a new light, and are thinking of edge cases that you didn’t think of before. Your purpose on the team is no longer in question, you remember why you got into QA in the first place, and go home happy.
What just happened? Why are you suddenly finding all these bugs that were hiding in plain sight?
Hockey sticks
What I described above is a development methodology that includes QA late in the process, and QA is performing exploratory testing due to lack of requirements. With exploratory testing I’ve found that finding bugs often follows a “hockey stick” distribution — at the start of testing you spend a lot of time poking around and finding nothing, but finding one or two bugs blows the doors off the app and you’re finding things everywhere.
Why does this happen?
Finding a bug engages you and your executive thought processes. We’re often distracted by team chat, food, or existential dread about our place in the universe. Finding a bug captures your attention and curiosity, and makes you focus at the task at hand. You then think if there’s a bug stemming from this case, then there is probably a bug with a different related case too, and so on.
Hacking the graph with carrots
I was talking to one of my senior devs the other day, and naturally we got to talking about bugs. As the conversation continued, he made a comment that was something along the lines of “whenever I put a carrot in there you guys never fail to find it.” I didn’t quite get it at first, but I eventually realized he was saying he sometimes intentionally leaves things to be found. While I don’t love the idea of seeing if QA is doing their job by catching a bug that was intentionally introduced or left behind (even if I’m proud that my team “never fails to find it”), the idea got me thinking — can we short circuit the amount of time it takes to find that first bug during exploratory testing so we can get right to the meaty part of the curve?
The next time one of my QAs was exploratorily testing something from this dev I kept an eye on things. I saw the typical pattern starting to form, but suddenly, a happy path for a failure state was disrupted by an odd looking button that didn’t match anything else. The QA immediately jumped on it being an issue, collected the details, and was possessed by the thought that if this was an issue with failure states, then he NEEDED to induce other failure state edge cases to see what else was broken. They ended up quickly finding other legitimate bugs related to failure states that would’ve taken far longer to find if they were just “poking around”.
Did I ask if the first bug was intentional? No. I’m choosing to believe that this was a brilliant play by the senior dev.
How to realistically cut down the curve long-term
While it makes for a good story, intentionally leaving bugs behind is not something that scales long term, or is even good practice.
The good news is that there are other ways to skip the flat part of the curve through thoughtful planning with the team. Things like Design or Test Driven Development are great ways to shift your testing left and not only catch bugs sooner by planning for them, but to avoid them altogether. I’ll have a future article outlining how to work through those development methodologies, but be sure to look them up in the meantime to see if they could work for your team.
If changing up your development methodology isn’t feasible right now, then you should work with your QAs to practice mindful testing. Make sure you are also consistently cross-training within the team to level out your product knowledge and experience. These methods are more reliably repeatable in speeding up exploratory testing than getting developers to leave carrots in the app, even if they don’t sound as cool.
fin
I hope you enjoyed the article — if you did, please share it to other thoughtful people or interact with it in some way!
