As you might imagine, I get a lot of test automation tool emails. A LOT. I open pretty much all of them and skim them. If something catches my interest, either positively or negatively, I read it in depth. I speculate that some, if not most, of y’all do something similar.

The messages that I find to be “negative” typically aren’t hateful or completely inappropriate (I would handle that differently), but often they espouse test automation ideas with no context as to when these ideas are appropriate and what is required to make them attainable. I usually just delete and move on. Occasionally, I’m inspired to respond to the sender, politely, of course, that I think they or their company is off target in one way or another. Sometimes, I use them for blog fodder; this is one of those cases.

I recently received an email from a test automation tool vendor that had a link to one of their blog posts; that post was about how automation makes your life easier. The remainder of this post will explain my thoughts on their thoughts. I will not name the company or the tool they are selling; that’s unfair to them since I think they are sincere in their beliefs about test automation. Their beliefs are like many providers of test automation tooling, both vendor-created and open source (As you might imagine, I have stories). Also, since I didn’t interact with them, it would be unfair to call them out.

One of the points in their blog post is that automation reduces stress and pressure; that has not always been my observation. In many cases, adding automation can increase stress and pressure, especially initially. Depending on how an automation endeavor is rolled out, one of the causes of additional stress and pressure is the unrealistic expectation that teams can do all of the “extra work” required by automation without reducing the scope or deadlines of the next release/deployment/etc. and without adding additional members to the teams. Regardless, if this additional work is “extra” or “something you should have been doing already”, a change in expectations is typically in order, at least for a while.

Another point raised in the blog post is that test automation reduces disruptions. In my experience, automation will cause more disruptions, at least initially. Yes, you have some disruptions now. “Test this new deployment”. “We have an emergency patch going out; it’s all hands on deck to test it before it gets released”. Yes, automation can help with the “do this now” nature of some disruptions, but it can also cause additional disruptions. Remember, automation is software. When first starting an automation endeavor, there will be issues to find, fix, and retest; that’s a disruption. Because it can run unattended, you will likely run the automation more frequently, but each time there’s a failure of the automation, you’ll need to investigate that failure; that’s a disruption. If you have “too many” failures, you’ll likely suffer from failure fatigue and you may neglect to take some of your testing’s potential energy into the kinetic energy of risk analysis; this can lead to additional disruptions.

The blog post mentions a point that I don’t think I’ve heard before, namely, that test automation avoids tester-developer disagreements. Consistent with my stances above, automation will increase the number of tester-developer disagreements and tester-rest-of-the-team disagreements, at least at first. Until there is trust, consistency, and availability of the automation being built and the information it delivers, you will hear things like, “The automation fails but it works manually”, “The automation passes most of the time, but it fails every once in a while”, “The automation isn’t catching bugs”, and the ever popular, “It works on my machine”. Once trust is built, however, you may have fewer disputes, but even if you don’t, the disputes and the associated resolution should be more valuable.

Their final point is that, with test automation, you don’t need to store your test cases in poor ol’, misunderstood, and misused Excel. Yes, some teams store their test cases in Excel. Yes, working this way can easily get out of hand. Yes, some teams, even those who are using automation, use Excel for test and configuration data, which can also easily get out of hand. But all of these are implementation and business decisions: the use of Excel for any of the above is a choice that may be useful to some but can quickly erode value. There are tools that testers can use for test case management and test data management; though some of these tools are integrated with test automation tools for “seamless” execution, as I previously wrote, there is a case to be made that you should not always rely on the “baked-in” test data capabilities. The use of test automation does not eliminate reliance on Excel unless you decide to not rely on it.

The blog post has a few other specific points, but I’ve condensed them into the points above because I believe my observations are consistent across those points.

And now, to answer the initial question, “Does test automation necessarily make our jobs easier?”, my answer is “No, it does not necessarily make your life easier”. It can, and should, but only if we keep an eye on our ultimate business and technology goals so that we automate appropriate activities in appropriate ways. Notice in my points above, I talk about negative impacts at the initial point in your automation journey so you must be patient; this is true for the introduction of most new technologies and processes. Remember, don’t expect that ease to come quickly.

Like this? Catch me at or book me for an upcoming event!