Capturing Quality: An Introduction to Screenshot Testing and What Is Coming To Android Studio
In mobile application development, ensuring a seamless user experience is paramount. A crucial aspect of this is verifying the visual correctness of the app, ensuring it looks and behaves as expected across many devices and screen configurations. Let us explore screenshot testing, a type of UI testing that goes a mile further.
What is Screenshot Testing?
Screenshot testing, as the name suggests, involves capturing screenshots during the automated testing process to ascertain the visual correctness of an app. It provides a pixel-perfect assurance by comparing the generated screenshots with reference images to spot deviations.
🔔 Some people argue that screenshot testing cannot replace traditional UI testing. To me, the problem is with the binary thinking they might have, rather than the testing approaches. Both tests are not mutually exclusive. Nobody said we can only take either approach instead of the best from both.
The Manual vs Automated Debate in Screenshot Testing
Screenshot testing sits at an interesting crossroads between manual and automated testing paradigms, sparking discussions within the developer community. Here’s a breakdown of arguments surrounding this debate:
Arguments for Screenshot Testing being Manual
The lure of screenshot tests comes from the desire to validate the visual representation of UI components coded by developers. Some argue that screenshot tests fall into the realm of manual tests because they often require a manual review to ascertain the differences or discrepancies between the images.
💬 For instance, tools like Paparazzi can conduct UI testing directly, but the results may necessitate a manual review to ensure accuracy and relevance.
Arguments for Screenshot Testing being Automated
Conversely, robust arguments support screenshot testing as an automated process. Automated screenshot testing tools can capture screenshots in bulk facilitated by test cases. This process helps realise how seamlessly an app renders across different screen sizes, all in a single go.
Furthermore, automated screenshot testing is lauded for empowering developers and QA teams to discover visual regressions and format inconsistencies with precision. As technology evolves and the demand for seamless user experiences heightens, manual testing may fall short in efficiency and coverage.
Some screenshot test libraries can exemplify the automated nature of screenshot testing by offering an improved workflow compared to manual testing methodologies. They integrate with various tools to build and test code, skipping manual tests and screenshot recording altogether, aligning more with automated testing paradigms.
The dichotomy between manual and automated testing in the context of screenshot testing reveals a layered discussion. While the manual review underscores a manual testing characteristic, the bulk capture and comparison of screenshots, often facilitated by software tools, align more with automated testing paradigms.
💡 This debate underscores the importance of choosing a testing methodology that aligns with the project requirements while also appreciating the evolving nature of testing tools and methodologies in the software development landscape.
Digging Deeper: The Unique Benefits of Screenshot Testing
Screenshot testing offers many advantages that set it apart from other Android development testing approaches. Let’s delve into some of these unique benefits:
Ease of Validating UI Elements and Styles
In traditional UI testing, validating the position and style of UI elements can become a tedious chore. It often requires extensive code to check individual properties like dimensions, colours, and positions. On the other hand, screenshot testing simplifies this process by capturing the entire UI in a snapshot. With a pixel-perfect reference image, developers can easily spot discrepancies in UI elements and styles without having to pen down lines of code for each property.
Streamlining String Value Validation
While it may not be the primary use case, Screenshot testing can also aid in validating string values displayed in the UI. By comparing screenshots, developers can notice unexpected text content or formatting changes. This might be a side benefit, but it can help catch unintended UI string changes without the need for additional code to check text values.
Efficient Validation Across Form Factors and Screen Sizes
The swift nature of screenshot tests is a boon when validating UI layouts across a myriad of device configurations. Unlike emulator-instrumented tests, which can be time-consuming, screenshot tests provide a quick feedback loop. This speed enables developers to test their apps across various screen sizes and form factors in a fraction of the time, ensuring the UI remains consistent and visually appealing, regardless of the device it’s being viewed on.
Screenshot Testing Libraries
Several libraries facilitate screenshot testing on Android. Here are some of the most common ones:
- Facebook Screenshot Tests: This is one of the first Android screenshot-testing frameworks. It allows for capturing screenshots as part of Android’s instrumentation tests.
- Shot: Built on top of the Facebook library by Pedro, Shot simplifies the screenshot testing process and is particularly friendly for projects using Kotlin.
- Android Testify: This is a library initially from Shopify, and it provides an easy way to review changes to the UI and establish a comprehensive set of screenshots for the application.
- Dropshots: Developed by Dropbox, Dropshots is another Android screenshot testing option.
- Paparazzi: Created by Square, Paparazzi is a JVM-based framework for screenshot testing, which might be a good fit if you prefer running your tests on the JVM instead of an emulator.
- Roborazzi: Although not as common, Roborazzi is another tool mentioned in a tech talk regarding screenshot testing libraries on Android.
The Upcoming AGP Update: A Game-Changer
The Android ecosystem is about to witness a leap in screenshot testing efficiency with the upcoming AGP update, as discussed in “The Android Show” on 19th Oct 2023. The core of this advancement is leveraging Compose UI previews, which developers adore for manual validation of different UI facets across various form factors and themes sans deploying to an actual device.
With the new update, developers can effortlessly transition these Jetpack Compose UI previews into screenshot tests. By executing a gradle task, Gradle navigates through all the previews in a module, rendering and capturing screenshots. Any alteration in the UI, like a subtle colour change, can be easily identified through a comparison between the new screenshots and the previous reference screenshots.
A striking feature of this update is the rapid execution, with a demo illustrating the process completed in approximately 10 seconds. The comparison results are rendered as HTML test reports, detailing the precise pixel differences, thus providing a clear insight into any UI alterations.
These screenshot tests operate in the JVM, promising a swift performance. The AGP update is slated for release within the year as a plugin, ready for integration in continuous build processes. Moreover, full integration into Android Studio is on the horizon, facilitating the review and management of screenshot tests directly from the IDE. This integration is part of a broader initiative to accelerate and simplify Android development, particularly for Kotlin builds.
The upcoming AGP update encapsulates a significant stride towards a more streamlined and efficient screenshot testing process, making it an exhilarating time for Android developers to delve into screenshot testing.
Bottom line: To Wait or Not To Wait?
The upcoming AGP update is undoubtedly promising, but the existing libraries are mature and well-equipped to serve your needs if you're hurrying to implement screenshot testing. However, if you’re willing to wait a little, the official Android solution might offer a more integrated and smoother experience.
💡 Another obvious point is that the new feature only works with Jetpack Compose. If your project contains View-based layouts, you may have other considerations or have to work around the previews to enjoy the benefits.
Screenshot testing is a powerful tool in a developer’s arsenal to ensure mobile applications' visual accuracy and quality. With Android’s forthcoming AGP update, the process is set to become more developer-friendly, making it an exciting time to delve into screenshot testing.
