UI tests on Android. How do we test an app that requires permissions?
My name is Andrey Sumin. I am an Android Developer on the Kaspersky Mobile Dev team. My teammates and I develop the Kaspresso test automation framework.
To work correctly, an app often requires access to specific functions of a mobile device: camera or voice recording, calls and SMS, and so on. An app can gain access to these functions and use them only if the user gives permission.
When writing automated tests for these types of apps, you may encounter certain problems. For example, if your autotest attempts to perform a specific action, the screen might show a permission prompt instead of your expected result. In this case, the permission prompt will be ignored and the test will fail. On the other hand, you may achieve the expected behavior by granting all the necessary permissions when starting tests, but in this case you will not be able to check how the app behaves if the user declines the permission request. In this article, we will show how these problems are resolved by the Kaspresso library.
We prepared an example and published it in our repository on GitHub at https://github.com/KasperskyLab/Kaspresso. Let’s download this project and start the `tutorial` app in one of the latest versions of Android (API 23 or later). After starting the app, tap `Make Call Activity`
A screen will open with two items: an input field and a button. In the input field, you can enter a specific phone number and tap `Make Call` to call this number
Making phone calls is one of the functions that require permission from the user. Therefore, you will see a dialog box prompting you to allow the app to manage calls. This dialog box will contain an `Allow` button and a `Deny` button
If you tap `Allow`, a call will be made to the phone number that you entered in the input field. Please note that if you run the test on a real device, this will start a real phone call that may incur charges.
The next time you open the app, you will not be prompted for permission again because this permission is saved on the device. If you want to revoke permission, you can do this in the settings. To do so, go to the Apps section, find the app and enter the `Permissions` section
Here you can select any permission and change its value from `Allow` to `Deny`, or vice versa.
Another way to revoke permission is to use an adb shell command:
adb shell pm revoke package_name permission_nameFor our app, this command looks as follows:
adb shell pm revoke com.kaspersky.kaspresso.tutorial android.permission.CALL_PHONEAfter this command is run, the app will prompt for permission again the next time a call is attempted.
Creating a test
We will use Kaspresso to test the app, so we must first connect this library to our project. We will add the following dependencies to the build.gradle file:
For details on how to connect Kaspresso to your app, please refer to https://kasperskylab.github.io/Kaspresso/en/Tutorial/Writing_simple_test/#kaspresso_1
When writing tests, we will use Page Objects. If you don’t already know what these are, we recommend that you read the official documentation at https://kasperskylab.github.io/Kaspresso/en/Wiki/Page_object_in_Kaspresso/ and check out our previous article.
In short, a Page Object is a class that contains links to all view elements that we need to interact with.
Create a screen Page Object with the `Make Call` button
To reach this screen, we will need to tap the appropriate button in `MainActivity`, so we add this button in `MainScreen`
We can create a test. For now, let’s simply open the window for making a call, enter a phone number, and tap the button
We run the test. The test was successful.
Depending on whether or not you granted permission, you may see a dialog box prompting you for permission to make the call.
If you are not familiar with the `step` method used above, or if you do not know why it should be used in tests, we advise you to read the lesson from our tutorial at https://kasperskylab.github.io/Kaspresso/en/Tutorial/Steps_and_sections/
So far, we have tested our screen and made sure that we can enter a phone number and tap the button, but we have not yet verified whether a call to the entered number is actually started. To check whether a call is actually started at this time, we can use `AudioManager` as follows:
We can add this check as a separate step:
Prior to starting the test, remove the app from the device or revoke its permissions by running the adb shell command. Also make sure that you are running the test on a device with API version 23 or later
We run the test. The test failed.
This happened because the user was prompted for permission after tapping the button. No one granted this permission, so the next screen was not opened.
Granting permissions using TestRule
There are a few options for resolving this problem. The first option is to use `GrantPermissionRule`. Essentially, this option involves creating a list of permissions that will be automatically granted on the tested device.
To do so, we add a new rule before the test method:
In the `grant` method, we list all required permissions separated by a comma and enclose them in parentheses. In this case, there is only one permission, so we leave it as is. Then the entire test code looks as follows:
Prior to running the test, do not forget to revoke all permissions from the app or remove the app from the device.
Run the test. This test will complete successfully in some cases, but will fail in other cases. Next we’ll find out why.
FlakySafely for assertions
We start a call and then check whether the phone is actually calling. We do so by using the `Assert.assertTrue(…)` method. Sometimes a device manages to dial the number before this check, but sometimes it does not. This is why the test ends with an error in some cases.
Kaspresso lets you use the `flakySafely` mechanism for any check. For more details about this, you can read the tutorial at https://kasperskylab.github.io/Kaspresso/en/Tutorial/FlakySafely/ or check out our previous article.
When this mechanism is used, the same check is run several times within a specific timeout period until it successfully completes. By default, this timeout is equal to 10 seconds. If the check does not return the “true” result within this time period, the test fails. The default timeout is suitable for our test, so we wrap the `Assert.assertTrue` method call in `flakySafely`
Now the test works, but it has a few problems.
First, the device continues the call after the test completes. Let’s add `before` and `after` sections, and end the call in the section that is run after the test. This can be done with the following code:
device.phone.cancelCall(“111”)In this method, we query an instance of the Device class. This class was added to the Kaspresso library. It has a multitude of capabilities, including the ability to change the language, change the device orientation, work with calls and text messages, and much more. For more details about this class, please refer to https://kasperskylab.github.io/Kaspresso/en/Wiki/Working_with_Android_OS
This method works via adb commands, so you need to make sure that your adb server is running prior to starting this test. For more details about this, check out our tutorial at https://kasperskylab.github.io/Kaspresso/ru/Tutorial/Working_with_adb/
Theoretically, you could also use a separate step to end the call and run this step as the last step without moving it to the “after” section. However, this would not be a good solution because if one of the steps ends with an error and the test fails, the device will continue the call and it will never end. The advantage of the “after” section is that the code within this block is executed regardless of the test result.
To avoid duplicating the same number in two locations, let’s move it to a separate variable so that the test code looks as follows:
Now the call is ended after the test is completed.
The second problem is that when using `GrantPermissionRule`, we can test the app only in a state in which the user has granted permission. It is also possible that developers did not foresee a scenario in which the permission request was denied, in which case the result may be unexpected and may even cause the app to crash. We must also test these types of scenarios. However, we cannot use `GrantPermissionRule` for this purpose because it always grants permission and therefore the tests would never reveal how the app behaves when the permission request is denied.
Testing with Device.Permissions
One way to resolve this problem is to use KAutomator to interact with the dialog box after finding all necessary interface elements in advance.
KAutomator is a Kaspresso component that enables interaction with third-party apps and system dialog boxes. For more details about it, please read our tutorial at https://kasperskylab.github.io/Kaspresso/en/Tutorial/UiAutomator/ and check out the official documentation at https://kasperskylab.github.io/Kaspresso/Wiki/Kautomator-wrapper_over_UI_Automator/
In our current scenario, KAutomator is not the best way to resolve this problem. Therefore, a much more convenient option named `Device.Permissions` was added to Kaspresso. This makes it very simple to check dialog boxes for permissions and either grant them or deny them.
For this reason, instead of `Rule`, we will use the `Permissions` object that can be obtained from `Device`. Let’s do this in a separate class so that you can retain both test variants. The class that we are working in now will be renamed to `MakeCallActivityRuleTest`, and we’ll create a new class named `MakeCallActivityDevicePermissionsTest`. The code can be copied from the current test, excluding `GrantPermissionRule`
If we start the test right now, it will fail because we have not granted permissions to make calls. Let’s add another step in which we add the appropriate permission via `device.permissions`. After specifying the object, we can add a dot and see which methods it has:
It provides the capability to check whether the dialog box is displayed, and to deny or grant permission.
This way, we can make sure that the dialog box is displayed and grant permission to make calls.
As we mentioned earlier, the dialog box will be displayed in Android API version 23 or later. Near the end of this article, we will describe how to run these tests in earlier versions
Here we wrote `device.permissions` twice. Let’s condense the code a bit by using the apply function. Let’s also move the `assert` check to the `flakySafely` method. Then the entire test code looks as follows:
Run the test. The test was successful.
Now we can easily write a test to confirm that a call is not made if permission was not granted. To do so, we enter `denyViaDialog` instead of `allowViaDialog`.
We also need to change the checks in the test. Additionally, do not forget to remove the code from the `after` function in the new method because the denied permission means that the call will not be made and we will not need to end the call after the test.
Testing in various API versions
In more recent versions of the Android OS (API 23 or later), the user is prompted for permissions via a dialog box during app operations. In earlier versions, these permission prompts appeared during installation of the app, which then operated under the assumption that the user already consented to all required permissions.
For this reason, if you run a test on devices with an API version earlier than 23, there will be no permission prompt and therefore no need to check the dialog box.
In a test that uses `GrantPermissionRule`, no changes are necessary. The permission is already granted in older versions, so this annotation does not impact the test in any way. However, we do need to make changes to a test that uses `device.permissions` because we explicitly check the dialog box in this scenario.
We have a few options here. First, it would not make sense to check the operation of the app on these devices if permission was denied, so we can simply skip this test. To do so, we can use the `@SuppressSdk` annotation. Then the code of the `checkCallIfPermissionDenied` method is changed to the following:
Now this test will run only in new versions of Android and will be skipped in older versions.
The second way to resolve this problem is to skip specific steps or replace them with other steps depending on the API level. For example, in the `checkSuccessCall` method on older devices, we can skip the step that checks the dialog box by using the following code:
We can leave the rest of the code as is and the test will run successfully on new devices as well as on older ones. The only difference is that the former will include a permission prompt while the latter will not.
The final test code will look as follows:
Summary
In this article, we examined two options for working with Permissions: `GrantPermissionRule` and `device.permissions`.
We also found out that the second option is preferable for a number of reasons:
- The Permissions object provides the capability to check whether a permission prompt dialog box is displayed
- When using Permissions, we can check the behavior of the app not only after permission has been granted but also after permission is denied
- Tests that employ GrantPermissionRule will not work if the permission was previously denied. You will have to either reinstall the app or revoke the previously set permissions using an adb shell command
- If you use an adb shell command to revoke permission during a test, it will work correctly when using the Permissions object but will crash when using GrantPermissionRule
If you are impressed by the capabilities of Kaspresso and plan to use this library in your own projects, please join our community on Telegram at https://t.me/kaspresso_en. Also, please don’t forget to Star us on GitHub at https://github.com/KasperskyLab/Kaspresso, and we’d love to see you among our contributors.
Furthermore, we welcome you to read our previous publications about autotests on Android:
Break free from the instrumented test code — use ADB within the tests
