What causes test automation chaos in most companies?
In my opinion, test automation is an area in the IT industry that has seen limited innovation. While this statement may sound unusual, it is based on my 20 years of experience in the field. Whether it is a well-established company with a long history or a newly formed startup, both face challenges in finding a reliable test automation framework. Although the dynamics differ between established companies and startups, the need for a robust automation framework is crucial for both.
It is widely acknowledged that a good automation framework can significantly improve productivity, reduce costs, and enhance product quality. Regrettably, I have witnessed companies that have suffered due to the lack of a solid test automation framework, resulting in inadequate testing and delays in product development.
Established company
Let’s begin by examining the issue within established companies. Typically, these companies tend to have multiple frameworks, and in some cases, each team within the company may even have its own framework. The development of new frameworks often occurs as the company diversifies its product offerings or through acquisitions. Unfortunately, due to the lack of a viable framework or unifying mechanism, attempting to consolidate these frameworks becomes practically impossible. As a result, the proliferation of frameworks persists, leading to increased operating costs for the companies involved.
The idea of “consolidating” or “unifying” test automation frameworks within established companies is often discussed. It involves the effort to merge multiple test frameworks into a single framework. Typically, at the request of management, different teams gather to present their frameworks to others. However, this endeavor is likely to face resistance, as teams tend to cling to their existing frameworks and resist any attempt to impose a new framework upon them. Having ownership of a framework is seen as a way to enhance job security, making it understandable why teams are reluctant to let go. In some cases, management might enforce the adoption of a specific framework on all teams, but this can lead to challenges in the long run. The teams on the receiving end may attribute any issues in their testing process to the “new framework,” resulting in frustration and blame being placed on the framework itself.
I believe there are two primary reasons why consolidation efforts often fail and why test automation can be a messy endeavor in many companies:
- The absence of a comprehensive or near-comprehensive framework within the company: In most cases, the existing frameworks within the company have their own limitations and shortcomings. It becomes challenging to select a single framework that addresses all the requirements and resolves the existing flaws effectively.
- The lack of qualified individuals with an independent perspective: Without the involvement of knowledgeable and unbiased individuals, it becomes difficult to make informed decisions about selecting an existing framework or developing a new one. The absence of qualified experts who can provide guidance and contribute to the framework selection process hinders progress and leads to confusion.
To summarize, the absence of a suitable Uber framework and the lack of qualified individuals to guide the selection process contribute to the challenges and messiness experienced in test automation consolidation efforts within companies.
Startup
The startup environment offers a relatively simpler and more straightforward approach. As startups build everything from scratch, test automation is also developed from the ground up, free from any existing baggage. It provides the opportunity to adopt the latest technologies available.
However, challenges arise due to the complexity of the product being tested and the expertise of the hired automation engineer, which significantly impacts the outcome. Finding a qualified engineer can be difficult, leading to the development of sub-optimal frameworks that are reluctantly accepted. In some cases, an engineer may struggle to produce desired results within a specified timeframe, resulting in their departure and the need to restart the process. Outsourcing test automation to third parties can further exacerbate the problem, as lofty promises often lead to less satisfactory outcomes. When faced with difficulties, third-party engineers may simply quit, leaving the company in a precarious situation.
The issue of engineers quitting their jobs is not exclusive to startups; it can also occur in established companies, although it happens less frequently. In established companies, the situation is slightly different as they have more resources and the ability to tolerate a substandard framework due to limited visibility beyond the team manager. Consequently, the manager becomes the de facto owner of the framework and may choose to rely on a mix of automation and manual testing or hire additional resources to compensate. However, this approach comes at a significant cost to the company.
What is the next step forward?
The purpose of this article is not to cause panic among those seeking to hire test automation engineers, but rather to sincerely address the pitfalls of test automation and assist companies in making better choices. While there are many dedicated test automation engineers out there who are contributing to the success of companies, not every company is fortunate enough to have a skilled automation engineer. Furthermore, there are numerous committed engineers who are willing to work tirelessly, either through manual testing or adhoc scripts, to compensate for any shortcomings in automation.
The initial question that may arise is how an external company can assist in testing a wide range of diverse products. The second question may revolve around trust, particularly for those who have had previous experiences with exorbitant charges and subpar services. To address these concerns, I look to the successful business model of Mercury Interactive (acquired by HP), particularly their product WinRunner, a GUI automation tool. Their approach involved providing a base product, accompanied by sample scripts and documentation, which allowed users to delve into the product. Additionally, they had a team of support engineers who were readily available to answer questions and provide solutions for complex scenarios. I believe this model is key to resolving automation challenges, where a customizable base product can be tailored to meet the specific requirements of each customer, providing them with the necessary support to succeed.
As I work on my startup to offer test automation services, I am more than willing to help if you are facing any test automation challenges. Feel free to reach out to me via email, as I am just one message away.
