How to Make iOS UI Testing fast and reliable
We write user interface (UI) tests with the XCUITest framework for the Trendyol️ iOS app to cheer up our unit tests. Unlike unit tests that verify methods' logic, UI tests interact with an app’s UI in the same way a user does, without access to internal functions and variables. Because of that, there are two processes involved, test runner and application under test. XCUIApplication is a proxy for the app, and UI tests call XCTest APIs to tell the test runner how to interact with the app’s UI.
In fact, even after this detail, we can’t expect similar performance from UI tests. While there are a lot of other dependencies out there waiting for undermining your tests, if the code isn’t designed well to support UI tests, they can easily become slow and unstable. Let’s see some solutions we apply to create a better UI testing project for the Trendyol iOS team.
Reliability:
Like many other UI test projects, we use the POM pattern to keep our tests easily maintainable and reduce code duplication.
A page is where your app’s screens are represented as classes, and all elements on these pages are defined as variables on class. All actions that the user can enter with these elements are defined in this class as a function.
How we implemented it?
We have UIElementPage as BasePage, which provides methods to access UI elements. Each page should extend this class by specifying the UIElements enum. With this approach, we are taking advantage of using enums and autocomplete.
By using the required init method in the base page, we force all pages that extend it to define the init method. Inside the init method, each page should call the check method. Check method explicitly controls whether all elements in the page are in the correct state when this page is initialized.
- While setting the identifiers and associating them with the pages, using an enum protocol speeds up their implementation and maintenance.
- By waiting for all elements to be correct state while creating page objects with inits, flakiness incredibly decreases and your tests can find known unknown bugs which are not supposed to be found by current test cases.
Speed:
Mobile applications are structures that all work asynchronously. Reliance on external dependencies, uncertain network responses, or animations delays can create flaky tests. That's why waiting is an important part to build a fast and stable UI test project.
Using static or implicit waits may solve some problems in the short term, but will be pain points in the long run. That’s why Xcode announced the explicit wait method waitForExistence in 2018. It waits the specified amount of time for the element’s exist property to become true or returns false.
Our way of waiting:
The problem using the waitForExistence method is it only determines if the element exists within the app’s current UI hierarchy. This means elements can exist offscreen, or exist onscreen but be hidden by another element.
That's why we need to create a new method that has the same explicit checking capability as waitForExistence method but also can check other states like hittable or selected.
An XCTNSPredicateExpectation is an XCTest class that continuously evaluates an expression until NSPredicate is satisfied or the timeout is reached.
XCTWaiter class waits on a group of expectations for up to the specified timeout. So this method is basically checking the state of the given element until it reaches the desired state or timeout expires.
At last, because unfulfilled expectations don’t automatically fail we have to check the result and fail tests according to them.
How we speed it up by controlling all the elements at once?
As I mentioned earlier, we have check methods inside each page that runs with init to be sure all elements on the page at the correct state before we start to interact with them. The biggest disadvantage of this method was the time it took. Since it needs to pull the app’s hierarchy to check each expectation is fulfilled, your tests should wait for these synchronous calls to end before move on.
To avoid these huge amounts of time loss, We figure out if we map all of the expectation queries and run them at once we can check hundreds of elements in a second.
It takes less than a second to check many items after switching to the waitForElements method as seen in the debug logs below.
Conclusion
At Trendyol, we believe that such methods can expand the ability of the frameworks so we keep trying to find new ones. Thank you for reading and feel free to contact me if you have any questions.
