OG Testers Mental Tricks: Heuristics, Biases and Autosuggestion
I never thought that psychology would be so important in my job. And yet, after several years of testing, I learned how people work, especially developers, but not only.
When you test something, you use more than just technical skills. You also rely on a kind of gut feeling, to guide your testing process. Junior testers may call it intuition, but experienced testers know better words to describe the mental process.
Heuristics, biases, autosuggestion, … All these psychological mechanisms influence the way you approach testing. To become a better tester, you need to know about them.
Heuristics
Heuristics are mental shortcuts we use to solve problems quickly.
Why is it usefull for testing? Suppose you have to test a feature made of 10000 lines of code and you have very little time to test it, the everyday life of a tester. You will probably base your tests on heuristics, to quickly find where the bugs are most likely to be found.
Here are some common heuristics which you may already be relying on to find bugs.
Change heuristic: When a part of the code has been changed, we focus on testing that part to check that no bugs have been introduced.
Past bugs heuristic: Remembering past bugs and making sure to test those scenarios first. This heuristic is particularly useful for finding recurring bugs and the most critical bugs.
Complexity heuristic: More complex parts of the code tend to produce more bugs. By focusing on the most complex areas, we maximise our chances of finding bugs.
Interoperability heuristic: This heuristic involves testing the system in conjunction with other systems to ensure that they work together correctly.
If you want to go deeper about testing heuristics, I recommend this article with some famous mnemonics on test heuristics :
Biases
I love biases. That’s what makes us human after all. If heuristics are the “how” of finding bugs, then biases are the “why” of the existence of that bug. If you know the biases you will be able to see the features to be tested from a better angle and you will know when to be vigilant to avoid this or that bias.
Confirmation bias
This is one of the most common, but also one of the most dangerous biases. You know the one about “all the tests are green, so it’s OK”. It’s overconfidence in positive signals without digging deeper to find the negative ones.
It is also the one that the developer relies on when testing his code. He will validate 1 or 2 test cases and consider that everything is good.
Congruence bias
It’s a human tendency to place too much reliance on direct testing a hypothesis and the neglect of indirect testing. In testing this translates mainly into testing positive cases and forgetting the negative ones.
I could go on and on about bias, but to summarise, most bias occurs when you focus on a single source of information and ignore the bigger picture.
How to overcome biases? Confront ideas, always involve third parties or third data in your decisions, base your tests on customers datas.
Example: to avoid biases, we never have a development tested by the person who did it.
Tester’s fatigue
It is good to know others well. But it is even better to know yourself well.
You must know that when you have tested the same thing 10 times, you are bound to become less and less vigilant. And it is often at the time when you are least aware that the bug will go into production.
How to overcome this fatigue?
For me, it’s all about staying focused and prepared. Test well, test early: always write your test cases in advance.
Then, before you start the test session, relax a bit, get some fresh air. And, when you feel ready, take on the posture of the careful tester.
You can take ont the attitude of the application breaker or the crazy user, or whatever, as long as you have a fresh perspective. And that’s what we’ll see in this final part
Autosuggestion
It is often said that a good tester is someone who can put themselves in the user’s shoes. But for me it goes much further than that. When it’s time to test something, a great tester has to put himself in the shoes of every possible user. He has to be both an expert and a novice. He has to be paranoid. He has to adapt his psychology to the point of personality disorder.
Knowing how to convince yourself is a formidable weapon in any situation. Much to my developers’ misfortune, and with all my good intentions, I always convince myself that they have left huge bugs in their development. Thanks to this technique, every time I click, I’m convinced I’m going to hit a bug, which makes me much more vigilant than ever.
Think like an expert user
Thinking like an expert user of software means being able to push the limits of that software in the most appropriate cases. It means aiming hard and true.
I once failed to think like this and it was one of my biggest mistakes in QA because I and my team missed bugs that occurred in such a common situation. Testing like an expert user means getting the use cases of your application from users and incorporating how they use your product into your tests.
Think like a fool
Test a product in many different ways. And one of those ways is to completely forget about the product specifications and let the interface guide you, while keeping the purpose of the test in mind. This may seem counterintuitive, or even unpopular, but it’s a way of finding lots of different bugs. However, be careful not to do this kind of testing too late in the development cycle, as it can compromise the UX.
Setting higher standards every year
By putting yourself in the customer’s shoes, you will have to raise your expectations over the years. If that doesn’t ring a bell, read this famous letter from Jeff Bezos about always having high standards and remember that phrase:
One thing I love about customers is that they are divinely discontent. Their expectations are never static — they go up. It’s human nature.
If you think you’ve mastered all these concepts then well done, always keep them in mind. There is a lot of talk these days about not having a “last line of defence” QA anymore, but we are grateful when they see what others do not.
Keep the bug hunter in you awake, stay on the lookout.
Never trust appearances, be critical and questioning, your colleagues will thank you for it.
