Reliable Visual Validation using Playwright
Today, web development has become very fast. Developers are shipping complex and UI heavy features at a very fast pace. Web apps are becoming increasingly sophisticated and run across wide range of devices — phones, tablets, desktops. This increase in velocity and target puts immense pressure on the testing process to make sure no bug gets missed.
While writing automation scenarios, a tester needs to be very careful to include all the validations and assertions to ensure utmost quality of the application under test. It is essential to ensure that no UI component on the web page is missed and there is a better and effective way to ensure this — Visual Validation.
What is Visual Validation?
Visual validation/testing, in simplest terms, is verifying that everything appears correct, or as intended.
“Appear” is an important thing to note here, because visual testing is all about verifying what the user sees. If we wanted to expand the definition for more clarity, it is:
- Making sure that everything that we want on a page of a website is actually there.
- Making sure that everything we don’t want to be there is not.
- Making sure that the visible output of the website is as intended by the businesses and for the user.
In Web testing, visual testing is the process of detecting and reviewing visual UI changes of the web page. It is a quality assurance activity that specifically focuses on ensuring that web page’s UI appears as intended for the end user.
Visual Validation can be incredibly powerful: event the slightest of the UI changes can verify easily, ensuring quality — at least in theory. In practice, visual validation can be slow, flaky and hard to maintain.
Today we are talk about Visual Validation using Playwright — a powerful framework for efficient visual validation testing. Playwright is a cross-browser automation library to write tests that are fast, reliable and capable.
Getting started with Playwright
Playwright Test was created specifically to accommodate the needs of the end-to-end testing. It does everything we would expect from the regular test runner, and more. Playwright test allows to:
- Run tests across all browsers.
- Execute tests in parallel.
- Enjoy context isolation out of the box.
- Capture videos, screenshots and other artifacts on failure.
- Integrate your POMs as extensible fixtures.
Installation
* Using init command
The easiest way to get started with Playwright Test is to run the init command.
This will create a configuration file, optionally add examples, a GitHub Action workflow and a first test example.spec.ts
* Manually
Add dependency and install browsers.
We can optionally install only selected browsers, see install browsers for more details.
Visual Comparison
Playwright framework provides the ability to produce and compare UI elements visually. Playwright Test makes use of pixelmatch library for pixel-by-pixel matching.
When test is run for the first time, it will take a screenshot of the current page and save it as a baseline screenshot (or golden image). Subsequent tests will take new screenshots and compare these with the baseline.
When we run above for the first time, test will fail with following error:
Error: example.spec.ts-snapshots/example-test-1-chromium-darwin.png is missing in snapshots, writing actual.That’s because there was no golden image yet. This method will take a bunch of screenshots and save them in the project repository. It is now ready to perform the visual comparison.
When test run command is executed again, as follows:
Playwright will use the page object and call the screenshot method to save a screenshot. It will then use the pixelmatch library to compare both of these screenshots for visual differences.
Sometimes, we might need to update the golden images, for example when the page has changed. We can do this with the --update-snapshots flag.
We can pass various options to finetune the pixel matching in our screenshots. If we want to share the default value among all the tests in the project, we can specify it in the playwright config, either globally or per project:
Comparing binary or text formats
Apart from screenshots, we can use expect(value).toMatchSnapshot(snapshotName) to compare binary and text formats payloads. Playwright Test auto-detects the content type and uses the appropriate comparison algorithm. This allows us to compare DOM elements for identical text content. For example, we can write tests where we match the text contents of a DOM element with a pre-defined text which we know will not change.
An example of such a test case is located below:
Here we take the text from the .login_title DOM element and compare it to the text inside the login.txt file.
Conclusion
The web platform is more capable than ever before and is continuously evolving. Hence, Visual regression testing is becoming a popular trend among QA and developer teams. Playwright offers its own, built-in visual regression testing solution through Playwright Test.
With Playwright’s visual comparison testing, we can perform visual validations in a fast, reliable and efficient manner.
