Ever called on a team to “fix” automation?
Legacy test automation code 😨 …. what should QAs do?
Pointers to help you with decision making!
At times, you get to work on legacy systems. Such systems may come with legacy automation frameworks too. Rarely such frameworks get talked about. So, let’s see how QAs should tackle them.
Your first instinct might be to rewrite the whole thing quickly from scratch 🙂. However, before you do, there are factors that come into play which affect this decision. Following are some of those factors :
Test content factor:
- Are the existing functionalities covered in here?
- Is the test data correct and enough?
- Do the tests have correct and sufficient asserts?
- Are the user journeys clearly described in the tests?
Dependency factor:
- Are the tests independent or inter-dependent?
- Is adding new tests, or editing existing tests going to break the framework?
Execution factor:
- Do the tests have potential to ever catch important bugs or are they just bloating the framework?
- Are the automated tests flaky enough to render them unreliable and useless?
- Are there any automated retries? If yes, what has been the approach towards them so far?
- Does it take too long to execute the tests? Can the execution time be optimised?
Test pyramid factor:
- Are the tests already covered at a lower level of the test pyramid?
- Can any one of these tests move to a lower level in the test pyramid?
Other factors:
- Does the framework have enough logging in order to track issues, within the system & the framework?
- Have principles like SOLID , DRY been implemented correctly?
- Is the framework unnecessarily complicated, like using Ruby meta-programming where not needed?
- What’s the AUT’s (Application Under Test) release cycle like?
- Is there a scope of negotiation with the stakeholders to replace the framework based on data and facts?
- Does everyone in the team have sufficient understanding of the product, domain, tech stack?
A list of such questions would help you understand the legacy system and automation framework comprehensively enough. Accordingly, the team can determine whether or not to:
a) replace the framework, or
b) refactor it to the best possible state at the time.
If you cannot replace the framework :
- Refactor the existing framework in logical pieces and test the changes thoroughly.
- Make smaller commits and describe them well.
- You can work with Git branching, which can help keep refactoring more organised.
- While picking up refactoring, it is a good opportunity to evaluate if you can use the latest dependencies.
The refactoring principle that I feel is central to this effort:
Boy scout principle: “Leave the code in a better state than you found it in”. This is a continuous process, and must be done whenever you need to add / edit tests in any framework.
It will have a positive impact as long as one takes a judgement call on when to stop refactoring for the current story.
Effectively, the health of the codebase improves iteratively. Therefore, we do not have to take special time out for refactoring (which is rarely feasible) nor impact delivery timeline / quality.
If you can replace the framework :
If your team is convinced that replacing the framework is a better bet, then comes the question of planning this really well.
Depending on where the product is in terms of its existing features and upcoming ones, it can be a challenge.
In order to assure the stakeholders, you need to give a clear plan along with timelines for each milestone. Keep them regularly updated on the progress.
When the story is played, you must write tests for new features and for other existing feature(s) that relate(s) to it. This will help you overcome the backlog without overwhelming the team / the system.
Don’t discard the old framework right away. You can have both the frameworks in the pipeline, running different scenarios. Keep adding to the new framework and removing the duplicates from the old one.
I hope this article elucidates some of the Dos and Don’ts for you!
