I Need a Tool. Now
This is a second story in the series based on my presentation “No Team, No Tests, No Time or How To Survive and Thrive when Resources are Limited” that I gave at Test Leadership Summer 2020. The first one No Time To Test and No Time To Automate can be found here
Selecting the right test automation tools takes time, few weeks at least to do it properly. QA Engineers need to build a proof of concept, automate a sufficient amount of tests (think double digits, not single numbers), and use it for a sprint or two. Wrong test automation tool can result in a significant amount of wasted time and resources. I would definitely recommend saving time somewhere else and do all of the necessary due diligence when selecting a test automation tool.
In reality, we often need to make the most important decisions in our lives in a very short amount of time. When I am forced to decide what tool we will use for automation the first question I ask is the money question. Can we pay for the automation tool? If the answer is no it cuts down the list of tools at least in half.
The next thing to consider is a set of non-negotiable requirements for a tool. What is the must-have? Should the tool support a specific programming language? or should it be able to run many complex SQL queries? or maybe should it have screen recording? These requirements depend on the needs of a specific project and team. Ideally, the must-have requirements list shouldn't be longer than 2–5 bullet points.
When done with requirements think about who will be automating the tests. Can they write code? Can they write good code? Based on that you can decide if can you use a tool that requires a lot of programming or you need a low-code, play-record tool.
If you select a tool that requires a lot of coding pay attention to the programming language this tool supports. Never ever select the tool that only supports language nobody in the team knows. Most likely than not you will end up with a stinky code that will result in a maintenance nightmare, flaky tests, and endless hours spent on debugging and fixing test automation issues.
Consider all additional services the tool provides. Great support? Performance and load testing using the same tests that are used for functional testing? Link crawlers? Multi-browser support? Parallel execution? All these “free” additional features can help to choose between tools that made to the finals in your list.
Finally, do “2 hours” POC. This can be more or less than two hours, the idea is that QA engineer time-boxes the efforts and tries to automate the simplest use case. If after selected time there is nothing valuable — probably it’s not the right tool. I do understand that there is a learning curve but if an experienced test automation engineer cannot do the simplest things after a few hours of work — most likely it is the wrong tool.
In the end, if you still don't know what to do — use the most popular tool for a specific type of application. In this case, you at least will have a lot of information available on the Internet and a large community to answer your questions.
The suggested approach does not guarantee that you will pick up the best tool (honestly, there is no approach that guarantees it), but at least you won’t end up with the worst one. This strategy can be also used for selecting any other tool or framework, not only a testing tool.
