The evolution of Quality Engineering
The evolution of Quality Engineering
Combining Software Engineering and Quality
The common understanding is that Quality Assurance is mostly about manual testing. What if I tell you that is not where Quality starts or ends? Where Quality Assurance is mostly testing the work there is Software Engineering to implement it. Those two should not be treated like separate islands, but can perfectly go as one. Combining these will give us the Quality Engineer.
What
My name is Kylie and I’ve started my tech career as a manual tester for Just Eat Takeaway.com.
Soon I learned about test automation which immediately highlighted the benefits of using code to automate manual work. What I could test in several minutes, a UI test could do in seconds.
From there on I’ve spent a lot of time learning how to write UI tests and optimizing the Automation project within the company growing to an Automation Engineer. Today I’m working as a cross-platform Quality Engineer with a background in Android, which is still where I’m specialized in.
I quickly noticed that Manual Testing and Automation in relation to Software Engineering were still seen as two separate islands, which didn’t strengthen each other as they could do when using both in the correct way.
Additionally to that Quality is not only about manual and UI testing. There are many more facets we can cover to improve and optimize flows within a company where not only the Quality Engineers, but also the rest of the team will benefit from.
And that is where Quality Engineering comes in.
Why
I think it’s safe to say that we are all on the same page when the end goal is to deliver a high quality product.
Where Quality Assurance in many companies has the knowledge to make sure all quality gates are being achieved by testing until the release — Quality Engineers additionally to that, also have the knowledge about how those quality gates can be implemented, maintained, and improved.
This additional knowledge and oversight can make a huge difference in the way the quality of a product is being handled, because now we can extend quality to a much deeper level and offer support in a much broader way.
How
There are a lot of different ways to check the quality of a product. Think of manual and automated testing, but these are not the only things I want to talk with you about today.
A part of Quality Engineering is about implementing the right set of testing tools, providing a safety net for engineers and the rest of the team.
The implementation of these testing tools will be different for every company and that is why I want to guide you through the process of how these were implemented and handled within Just Eat Takeaway.com.
This article will not contain all the specific testing tools that are available, rather it will show you a combination of tools that is currently working for us, creating the quality architecture we are having today and that I’m still working on with great passion.
Manual and Automation testing and how to achieve efficiency
Manual and Automation testing are two examples of types of testing that can be perfectly combined to fasten up processes if this is done correctly. Manual testing should always be from additional value, instead of double testing scenarios that are already automated. Another pitfall could be spending a lot of time on manual testing for scenarios that can easily be automated. Every scenario that you test manually, will run as an automated scenario faster. With this in mind it is better to spend a bit more time at the start to set up an Automation project, but safe double the time on a later stage.
UI tests can be set up in a lot of different ways which all work and are correct, but that doesn’t mean that they are all as efficient.
You can set them up in a live environment which means that they will depend on things like; the API and the network. This can be convenient for some scenarios, but it can also slow them down and cause flakiness. UI tests should test the functionality of views and components which means that running them on a live environment in most cases isn’t of any additional value.
To achieve this efficiency you can think of mocking, which will give you a more stabilized environment and will prevent flakiness. What can help determine if a UI test should run in a live environment or a mocked one is to check if it is a business critical scenario or not. A business critical scenario that isn’t allowed to fail is a valid reason to run them in a live environment where all non critical scenarios should do fine in a mocked one. Another example is that you need to think of the purpose of the test suite. A live environment would do great when testing for a release, while UI tests that run against merge requests should not be bothered with that load and can use a test suite that has mocked UI tests.
A next step could be to make Automation a part of the Software Engineering cycle as they are quite often seen and treated as two separate islands. UI tests should run during development, to detect bugs. What can help with this is categorizing the UI tests to the modules in the repository, so that the affected folders will run automatically when a merge request is created.
If the pipeline passes the engineer will be able to merge, but if the pipeline fails the affected test(s) needs to be fixed before merging is allowed. During a release the UI tests; critical and non-critical can be run against the release pipeline for a final check.
Of course these approaches can differ for everyone, but I hope that I can give you a bit of insight about how certain processes can be handled, giving the Quality Engineer more time to focus on improving the quality architecture in your team.
Screenshot testing
Next to manual- and automation testing you can use screenshot testing to cover various areas within your project.
Screenshot tests can efficiently cover things your UI tests cannot. Screenshot tests compare the current screenshot with a stored expected screenshot, so they can quickly identify any unexpected changes in the output.
They are particularly useful to test visual components such as the UI or layout elements, but are less convenient when detailed information about individual components or views should be tested as it can be challenging to pinpoint the exact cause of failures.
They also always need to be used alongside with other forms of testing as well, such as UI tests, E2E tests and unit tests to validate the performance of the underlying code.
SonarQube
We talked about testing the end product, but what about testing the code quality itself? What if you can prevent bugs by using a tool that performs a code-style analysis to secure the code quality and code security. SonarQube is a popular tool that can be used for this and is the one we use.
It provides a platform to manage code quality across projects. Allowing you to track multiple projects at once and their quality trends over time, making it easy to enforce coding standards and achieve a consistently high level of code quality.
The tool also allows you to set up multiple sets of coding rules, so that the clean code principle of your project will be checked through all merge requests.
Sonarqube also has the big advantage that it integrates seamlessly with popular CI/CD tools like Jenkins, Azure DevOps and GitLab CI/CD.
It also can be enforced on merge requests, ensuring that issues are caught early and preventing the introduction of poor-quality code into the codebase.
I can imagine that you work in a company that supports more platforms than only Android, for example: Web and iOS. Within a company as Just Eat Takeaway.com, where we work cross-platform, a tool such as SonarQube has huge benefits as it can be implemented for all platforms where it will work the same way as well.
SonarQube also provides detailed reports and metrics, giving an overview of code quality trends, security risks and tech debt. This information is not only valuable for engineers, but for the whole team and can help them prioritize addressing critical issues, improve code quality and facilitate continuous improvement of the codebase.
CI/CD
Continuous integration and continuous delivery. A well known software development practice that emphasizes the continuous and automated integration, testing and delivery of code changes to ensure that software can be released reliably and quickly.
It also offers an environment where you are immediately notified about failing pipelines, so the team can handle these quickly. Overall we can say that handling the CI/CD approach stands for continuous improvement. By automating the build, tests and deployment process, engineers and QE’s can focus on adding new features and improving the software continuously.
TestOps
Another set of skills within quality engineering is the usage of TestOps. This involves using automated tools to scale up testing resources if needed. At Just Eat Takeaway.com we use this as a tool to increase the visibility of testing tools we are using.
It can be of additional value for team members to get updates of the outcome of certain testing tools where they don’t have an immediate insight. Writing scripts that forward reports to the instant messaging software your company is using can help improve this visibility.
At Just Eat Takeaway.com we use Slack as our main communication tool, where we have a team forwarding reports to a dedicated Slack channel about flaky UI tests. The script is running as a set of functions on AWS lambda and is set up with a CRON job, to trigger a run on a day of preference.
So how this works is that it will scan through the recent Firebase test results and scan the files for any mentions of flaky tests. We use firebase as a platform to show a detailed report of UI tests that have run in the pipeline. The script will then compile these results and print them to a slack room, where all stakeholders are able to see the report and address them to the correct teams.
We have a lot of teams working on the same things and this report is used as a centralized way of having visibility of unstable tests, which would otherwise be hidden by retries in our test tooling. This report is used to prioritize work on unstable tests to stay ahead of tech debt.
Quality architecture
Within Just Eat Takeaway.com you can say that all these tools are a part of the quality architecture. They are used to set up a testing environment where efficiency and optimizing performance are as balanced and important as the other requirements.
The quality architecture should be set up in a way that speeds up the process without losing product quality and goes through all phases of the development cycle.
In order to set up the Quality Architecture you need to have insights about the team dynamics, what management methodology your team is working with and which development process is linked to that. From there on you need to investigate where quality checks are lacking and in which form. In which phase of the development cycle most bugs are coming from and how long does it usually take before you are detecting them?
A worst case scenario can be that, because of a lack in planning a critical bug is found in the latest phase of development to deployment. Statistics show that the cost to fix a bug that late in the cycle is 6 times higher when found in an earlier stage. This is exactly why implementing the right set tools to cover all quality gates throughout the development cycle is so important.
In Just Eat Takeaway.com we are working in an Agile environment where the set of tools I’ve told you about works really well. This is because the testing tools are checking all the quality gates throughout the entire development cycle.
A good quality architecture will also result in higher confidence throughout the team which has its effect again on the quality of the product and speed of delivery. Such an environment in combination with a Quality Engineer that can work cross functional, has the knowledge and is taking the lead with maintaining this environment, can inform stakeholders at any point and also coach the team is in my opinion a killer combination and definitely the future of Quality Engineering.
Future of Quality Engineering
The digital landscape is evolving rapidly. These changing trends are something all big companies have to keep up with if they want to stay in the competition, right?
Being able to stay in the game means that the way of working in a tech company is changing as well and the quality architecture isn’t something that can stay behind.
The world of quality is so much bigger than doing manual testing only or separate manual testing from automation and engineering.
Personally I see quality as an umbrella that is covering all parts of the development cycle, making sure quality gates are checked on the right time in the right order, with the correct actions taken from that. Quality for me also is working together with all different stakeholders making sure to secure quality through all phases of development.
Combining this knowledge and these skills with the engineering side of implementing testing tools, automating and optimizing flows, striving for efficiency without compromising on quality is where the future of Quality Engineering lies. Changing the way a role is will take time as it will change the dynamic in the team as well.
Within Just Eat Takeaway.com the role of a Quality Assurance Specialist and Automation Engineer changed to the Quality Engineer I’m now talking about and this took time as well. Time that was invested in optimizing and automating a lot of processes, but also about adding new testing tools that filled the gaps that were still there. Quality gaps that were checked manually previously are now automated freeing time for me to focus on actually improving the process.
The time that we invested in this process has paid off. It didn’t only increase the quality of the product we are delivering, but also increased my quality time and where I can now spend it. Not only has this had a major positive impact on the product, it is also having a major impact on me improving as a Quality Engineer, gaining a lot of experience and technical knowledge, more than I learned before all the changes were implemented. It allows me to offer next level support throughout the whole development cycle.
The thing with technology is that the more advanced it gets, the quicker it gets advanced. This may sound a bit weird in this sentence, but let me explain it. The last couple of years some technologies were introduced that probably many of us are already familiar with. Look at the introduction of ChatGPT, or Github Copilot.
This is taking development to a more advanced level, which automatically means that quality needs to step up as well.
And this is exactly why I think combining quality with engineering will offer you a huge benefit that can be used on any platform.
‘Where the developer thinks about the architecture of the product they want to build — a Quality Engineer thinks about the architecture of the quality tools and processes this product should have in place, creating a perfectly synced development cycle’