Mobile test automation practices — Part 1: verifications, modules and basic actions

Dmitrii Makarenko

·

Follow

Published in

Bumble Tech

·

12 min read

·

Aug 26, 2021

--

2

Listen

Share

Bumble Inc is the parent company that operates Badoo and Bumble, two of the world’s largest dating and connection apps with millions of customers worldwide. Delivering new functionality as fast as possible is a priority. However, it is important that the speed with which we work in no way compromises the quality of our apps.

Automation is a great help in that regard. The position of automated testing has changed a lot over the past two years. The number of people currently actively involved in test development has quadrupled from 10 to 40. And any new functionality in our applications now has to be covered by tests prior to release. (Find out more about how we got there in Katerina Sprinsyan’s talk.)

Operating in our particular conditions, it is highly important that we develop tests as quickly as possible. And, at the same time, they need to be stable — so we keep maintenance time to a minimum. In this article, we share the practices we use both to speed up the development of tests and increase their stability.

My name is Dmitrii Makarenko and I am a Mobile QA Engineer working on both apps, Badoo and Bumble. I am involved in testing new functionality manually and covering it with autotests. My colleague Viktar and I have spoken on this at several events but now we want to make the info more widely available.

In this article, we cover the following:

1. How we chose the test automation framework
2. Practice no. 1. Where and how to write verifications
3. Practice no. 2. Automated testing of several apps
4. Practice no. 3. Steps for basic actions
5. Summary.

The examples we look at in this article are relevant not only for those just starting to roll out automated testing in their project but also for everyone actively using it already.

Let’s begin with a brief account of the automation framework at our company so the context of the practical examples we use is clear.

How we chose the test automation framework

Our Badoo and Bumble apps are native: i.e. Android apps are developed by one team, in Kotlin and Java, while iOS apps are developed by another in Swift and Objective C.

At the same time, functionality in Android and iOS apps is to a major extent similar. It is precisely for this reason that one of our main criteria when choosing an automation framework was having the option of reusing test scenarios. Guided by this, we chose Calabash, a cross-platform automation framework. We write tests in Ruby and use the Cucumber framework for writing scenarios. Later, we also added another cross-platform framework, Appium.

It is important to say that it doesn’t really matter which framework you use. We have generalised our recommendations here to make them easy for you to apply.

Practice no. 1. Where and how to write verifications

To better understand where you can write verifications let’s look at the test structure. This structure consists of three levels.

1. Scenario. This is where the test itself is written in Gherkin, a human-readable programming language; all the necessary actions and verifications are described here.
2. Step definitions. Depending on the actions and verifications from the first level, the relevant Ruby code is put in place, which then performs them.
3. Pages. This is where we describe our app’s screens, and actions we can perform on them — for example, obtaining the text of a required element, tapping on a required button etc.

Now to the practical side of things.

Task definition

We need to create verification for a particular element. As this formulation is very general we shall consider a specific example. This will be the verification of a message about a missed video call.

Examples of solutions

Let’s begin with a scenario for verification of this message.

In the text of the scenario you may notice an abbreviation, QaApi. This is an API, which allows us to change the status of the system during testing. For example, we can send messages, or upload profile photos for test users, using ordinary client-server requests. (We talk about QaApi in more detail in this article.)

Let’s first focus on the step for verifying a required message (the final step in the code example).

Now to define what we need to verify. This is the text of the message and the “call back” button. Let’s review one possible approach to this task.

In this example, at the level of step definition, we create an object of the ChatPage class and call the “await” method to wait for the necessary screen to load. We then call the verify_missed_video_call method to verify that the necessary elements are displayed.

At the page level, this is what it looks like when we implement this method. First, we define the expected results for the lexeme, taking this from the static method, declared in the CallLexemes module, using the class << self technique:

We then take the actual result from the screen:

ui is an object which provides us with access to the framework methods.

We then compare the expected value with the actual value:

We get the following method as a result:

You may have picked up a mistake here. The person who developed the method for one of the platforms omitted something important: verification of the “call back” button. A test developer for another platform, in turn, added this verification and created the following method:

Here are the drawbacks of the first method:

1. High risk of error. Of course, it is difficult to imagine some verification being omitted in a case involving two elements, as in our example. However, when we are verifying five or even ten+ elements, there is an increased risk of the test developer omitting or incorrectly implementing a particular verification.
2. Duplication of code. As you may have noticed, the implementation of verification is duplicated on various platforms, and this slows down test development.

What can we do to overcome this? Instead of implementing verifications at a page level, we move verifications over to the step definitions level:

And we implement two new methods on the pages. One will return the message text:

The other will return the button text:

It is a lot more difficult to make a mistake in such methods, even where there are lots of elements. What is more, verifications implemented at the level of step definitions get reused by different platforms — and so we avoid duplication of test code.

Takeaways

This is what we recommend:

• Create verifications at the level of step definitions
• Keep pages simple i.e. create them so that page classes return information on the status of the app screen, and do not contain implementation of verifications.

Generic recommendations:

• Implement verifications at the top level, i.e. where you are writing the body of the test
• Implement simple test objects (do not mix the test logic with the object itself).

Practice no. 2. Automated testing of several apps

Now let’s explore how the approach from the previous example helps us not only when working with two platforms, but also when working with two apps.

The recommendations from this example are also relevant when working with a single application.

Task definition

We needed to implement a shared chat page for two applications, based on the example of Badoo and Bumble apps. (This came up when our developers created a shared chat component and used it in both apps. In the tests we needed to do the same thing.)

Solution example

We started by dividing the chat screen into logical modules, namely:

• Upper panel (toolbar)
• Dialogue area (conversation)
• Input field (input source).

We divided the conversation area into smaller modules for various message types: text, audio, GIF images and photos.

The input module also consisted of smaller-scale modules, such as a field for sending photos, text, GIF images and audio messages.

After that, we created a module structure for Android and for iOS similar to the one in the screenshot below.

After this, it became easier for us to create classes for chat pages in both applications. Essentially, the only difference is in the usage of different base pages. This is because these differ a little between applications.

Now we simply include the required modules into the page for each application. At the same time, we are only including modules that are relevant for the app in question. In Badoo, for example, this could be a message about a gift — something that doesn’t exist in Bumble:

On the other hand, Bumble has another type of message which does not exist in Badoo, namely “reactions”:

The existence of this component allows us to easily create a chat page in any other new application should our company implement the application in question — it literally takes just a couple of minutes.

We also use the same steps for both applications. This is where we return to the methods for obtaining the text of a message about a missed video call, and the “call back” button from practice no. 1. They are located in the module for video call message type.

This module is included in chat pages for both Bumble and Badoo. That is to say, the module is written once for each platform, while a message in four of our applications (Badoo for iOS and Android, and Bumble for iOS and Android) can be verified in a single step. So, writing new tests to verify the chat functionality in different applications is certainly not onerous.

Takeaways

We create similar components in those cases where we do not need to reuse them in different applications, for example for screens with a high number of UI elements. This approach in one application allows you to avoid what is called God object (more information here and here), classes with a large number of properties and methods. It is far simpler to support pages broken down into modules as well as to extend them when new functionality is introduced.

Drawing conclusions from this example, we see that using components helps us in:

• Creating tests for different applications, using existing steps and saving lots of time along the way
• Supporting and extending classes of pages for screens with a large number of UI elements.

Generic recommendation: in the case of objects with a high number of properties and methods, you need to create components comprising logically separated modules.

Practice no. 3. Steps for basic actions

We begin the examination of this next example by identifying the actions we consider to be basic. For example: waiting for a page to open, verifying a page, closing a page, or going back from a particular page. We call them basic because they are relevant for practically any page in our applications.

Task definition

We need to implement basic actions for different pages.

Examples of solutions

Imagine this situation: two QA engineers have created steps for waiting for chat and profile pages to open:

Here we spot the following problems:

• The names of the steps are somewhat different so they are going to be difficult to find when we need to use them in new scenarios
• Implementation of the steps differs only in terms of the classes of objects: ChatPage and OwnProfile. Bearing in mind that our applications have many more than two screens, if we create different steps to wait for each one of them to open, it will lead to a lot of duplicated code.

Now let’s see another example that implements the steps for going back from a particular page:

Here is yet another problem: pages contain methods (tap_back, go_back, press_back), which are responsible for one and the same action but which are named differently. This has happened because different people have added them at different places in the repository.

In this way, the shared problem in these examples is code duplication, and this significantly slows down test development. We see duplication, both of the steps themselves, and also of actions implementation on the pages. This all makes it more difficult to reuse steps for basic actions in different tests, given that it is practically impossible to remember all the possible variations of these steps.

How can we get around this? Go from a simple approach to a more complex one. We can create an object of the page class using a string from the step parameter. To this end, we created the page_object_by method.

This is what implementation of the page_object_by method looks like:

We are creating the name of the class based on the name of the page passed to the step. Then, if this class exists, we return an object of this class. If not, we raise an exception.

In this way, all we need to do to wait for any page to open is just to write a single step. We also use a similar approach for all other basic actions. And this significantly simplifies development.

We have also eradicated method duplication for basic actions on pages by implementing generic methods such as press_back.

Implementation of this method can be found on AndroidBumbleBase, IOSBumbleBase, AndroidBadooBase and IOSBadooBase pages, from which the classes of the other pages in our tests are inherited. It should be said that this is not an ideal solution. If this approach is taken, pages that do not have a “back” button can use a method for tapping on this button. It would be better to extract tapping on the “back” button into a separate module such as Navigation::Back, and to include it in all the necessary pages. But we have not used a module of this kind because it would mean having to add it to nearly all the pages in our project — and we opted to make life a bit simpler for ourselves.

Takeaways

So, in the context of this example this is what we recommend:

• Creating generic steps for basic actions
• Creating generic methods for performing basic actions on pages (going back etc.)

In the previous section we recommended creating modules and reusing them on pages. Whereas here we are saying that you need to create methods on pages. We might seem to be contradicting ourselves somewhat but that is not the case.

If you really do have lots of methods on a page, or if you need to reuse code only on some pages, it makes sense to identify modules and to add them to the necessary pages.

However, where an action is relevant for all pages this should be implemented as a method in a class providing the base for all other pages in our tests.

Generic recommendation: where you have lots of similar code, implement generic methods.

This recommendation may seem obvious but we want to stress the importance of applying it. I would remind you that we have about 40 people actively involved in the development of end-to-end tests. It was important for us from the very beginning that we choose the right approach and thereby avoid the need to refactor multiple similar steps and methods later.

Summary

We have considered three examples of solutions to different tasks typical for autotest development. Based on these we offer the following general recommendations:

• Implement generic methods for reusing similar code — both in steps and in methods on pages
• Create verifications at the top level — where you write the body of the test
• Implement simple test objects
• Create components that consist of logically separated modules, for objects with a large number of properties and methods.

It is possible that these recommendations might seem obvious to some. But we would like to draw your attention to the fact that it can be (and should be) applied in different situations.

In Part 2 of the article, we will review another four examples of common test automation tasks, add to our list of recommendations, and will share access to a test project with all the associated practices. Stay tuned!