Your problem isn’t test scripts
I am worried about a trend. I am also frustrated. Every tester at some point in their career has said: “You can’t test everything.”. You only really need one testing assignment in your career to realize this, if you are paying attention. Now it seems to me that there are people in controlling positions that have not said this and/or are not listening to the experts they hired.
I can guarantee you, any tester, that consider themselves professionals do not follow a test script. Maybe the headline for a general idea but that’s it.
Now if you are certain that your team does not deviate from scripts and you have forced them to follow them to the letter, you really need someone to shake your tree.
You should go wide in your thinking (what else could affect this), go deep with your thinking (what if i do this), analyze (what did just happen, why), hypothesize (this area had this kind of problem because of x, maybe x is causing problems somewhere else), wonder (If i have these preconditions, i wonder how the product behaves), question (should it do this?).
Overblown regression suites that are full of junk. Percentage of test cases automated. Percentage of code covered. These are all signs that the fundamental limitations of resources has been forgotten by some parties. No matter how hard we try, even with the help of bots, we can’t test everything and be sure that nothing will ever go wrong.
As far as testing goes, the metrics above are more or less useless anyway.
For me it looks obvious that with even a split second of reflection and talking with your team reveals where the time is spent on testing sessions. So, either it is deliberate ignorance or lack of awareness.
Here is some pointers that i personally encounter everyday to give you some perspective:
- Common understanding. Interrupting my testing session to ask clarifying questions from the developer or business owners/customers.
- Environment is lacking. The time it takes to set up a test takes a long time. Needed data is not clear. Someone tampered my test data, i need to fix it. Someone launched a new deployment in the middle of my session. It will wipe out everything i did.
- Scope. What am i supposed to be looking at? What are others suppose to be looking at? How far in the rabbit hole should i go?
- Third party integration is unstable. I am once again blocked by the integration not working. Testing is postponed for days.
- Strategy. What is the goal of our testing overall? What is the goal of testing this functionality? How does it fit to the bigger picture?
- Testability. This thing i am supposed to test is really hard to control and observe.
So many questions to answer, so many things to evaluate, analyze, decide.
Let me say it again. The problem isn’t test scripts or the time it takes to “execute” it.
“Executing” these scripts takes a long time. Lets have bots execute them for us.
Yet, somehow the leaders have a fixation that if we automate the click of a button, we are somehow saving time.
Now it seems, that this fixation has reached a new peak and every leader is blinded by a new “gold rush” about how ChatGPT or CoPilot creates the tests for us.
“Writing” these scripts takes a long time. Lets have bots create them for us.
It will absolutely have zero influence on making the bullet points on top any clearer. It will not speed up my work, if anything it will get even more disruptive as i will be spending more time on evaluating the scripts and trying to give it better input and/or modifying the script itself or at least these are the things i should be doing, and not just blindly accept everything.
What this will surely achieve is to make problems even worse. Even bigger regression suites that nobody knows what they do, because we are even more efficient at producing useless scripts. People who don’t know how the product behaves because nobody bothers to try it out.
Your problem isn’t test scripts and having software create them for you will not fix your real problems. Nor will they enable the team to work on the problems as their time is spent fixing the code, managing the useless artifacts etc..
So please, please reconsider. I really am on my knees here typing. Don’t. Have some respect for yourself and towards the work that you do.
You know, as i am finalizing this post, it suddenly dawned on me. Your test scripts might actually be your problem. Just not the time spent. The problem might be your fixation towards them in replacing strategical decisions and planning.
If this made you feel any emotions, you are more than welcome to have a clap and leave a comment.
