Native Mobile test automation with Nightwatch and Appium

Daniel Maioni
13 min readJul 22, 2023

Identify elements, write and execute test cases

Requirements

  1. Environment

Given we already have a nightwatch project setup, Appium Server and Appium Inspector installed, and a android emulator installed — check the last post about it — let`s follow the next steps to create a new test case — Check the previous requirements section to know how to do it!

For this, let`s setup Environment and Desired Capabilities at nightwatch.conf.js file

Edit the configure file nightwatch.conf.js

  • at src_folders set the path to the test suites (it could be a single path or multiple paths like src_folders: [‘test’,‘nightwatch/examples’])
  • at test_settings section create a environment item (here we gonna use app.android.emulator.test) and setup the desired capabilities:
module.exports = {

src_folders: ['test']

test_settings: {
....

// new enviroment app.android.emulator.test
'app.android.emulator.test': {
extends: 'app',
// list of capabilities
'desiredCapabilities': {
browserName: null,
platformName: 'android',
'appium:options': {
automationName: 'UiAutomator2',
avd: 'nightwatch-android-11',
app: `${__dirname}/apks/Android.SauceLabs.Mobile.Sample.app.2.7.1.apk`,
// the application to be tested
appPackage: 'com.swaglabsmobileapp',
// the screen to be initialized
appActivity: 'com.swaglabsmobileapp.MainActivity',
chromedriverExecutable: `${__dirname}/chromedriver-mobile/chromedriver`,
newCommandTimeout: 0
}
}

...
},

Note: If you need to find out an application package and activity view, open the desired application and desired application screen, and run the bellow command line:

adb shell dumpsys window windows | grep -i <app name>
OR
adb shell dumpsys window windows | findstr <app name>

adb shell dumpsys window windows | grep -i swaglabs
Window #6 Window{d95474c u0 com.saucelabs.mydemoapp.rn/com.saucelabs.mydemoapp.rn.MainActivity}:

At first line (Window #n): <package>/<activity> = <com.saucelabs.mydemoapp.rn>/<com.saucelabs.mydemoapp.rn.MainActivity>

appPackage: 'com.swaglabsmobileapp',
appActivity: 'com.swaglabsmobileapp.MainActivity',

Now, open the package.json file and add a script command for npm run test:

“scripts”: {
“test”: “nightwatch -c ./nightwatch.conf.js - -env <environment>”
},

{
"scripts": {
"test": "nightwatch -c ./nightwatch.conf.js --env app.android.emulator.test"
},

"devDependencies": {
"@nightwatch/mobile-helper": "^0.1.12",
"appium": "^2.0.0",
"appium-uiautomator2-driver": "^2.29.2",
"appium-xcuitest-driver": "^4.32.21",
"chromedriver": "^114.0.2",
"nightwatch": "^3.0.1"
}
}

Now it`s possible to run all tests with npm run test:

npm run test
OR
npm run <script name>

2. Test Cases

Create a test suite with the BDD describe

describe('Swaglabs Android app test', function() {

});

Add some suite-specific capabilities with this. command:

Add a suite-specific retries option for suite (1) and test cases (2) and set a custom timeout (5000 is the default value if not declared). If a suite fails, it will runs 2 times (first one + 1), and if a test case fails, it will run 3 times (first one + 2).

describe('Swaglabs Android app test', function() {
this.suiteRetries(1);
this.retries(2);
this.timeout(10000);

});

Add a tag to this test suite:

describe('Swaglabs Android app test', function() {
this.suiteRetries(1);
this.retries(2);
this.timeout(10000);

});

Create the test cases for that suite

describe('Swaglabs Android app test', function() {
this.tags = ['swaglabsmobileapp', 'login', 'sanity'];
this.suiteRetries(1);
this.retries(2);
this.timeout(10000);

it('Login test', async function() {

});
});

Initialize app driver (instead of browser)

describe('Swaglabs Android app test', function() {
this.tags = ['swaglabsmobileapp', 'login', 'sanity'];
this.suiteRetries(1);
this.retries(2);
this.timeout(150000);

it('Login test', async function(app) {
app
});

Some examples of settings:

this.unitTest = true; // enable if the current test is a unit/integration test (i.e. no Webdriver session will be created);
this.skipTestcasesOnFail = true // enable if you'd like the browser window to be kept open in case of a failure or error (useful for debugging).
this.disabled = true // enable if you'd like the rest of the test cases/test steps to not be executed in the event of an assertion failure/error
this.timeout(1000) // assertion and element commands timeout
this.retryInterval(100) // Control the polling interval between re-tries

Other examples of test cases at nightwatch repository: https://github.com/nightwatchjs-community/nightwatch-examples

Nightwatch also support the export test syntax, but it`s more limited compared to BDD syntax:

module.exports = {
'@tags': ['login', 'sanity'],
'demo login test': function (browser) {
// test code
}
};

3. Debug Mode

To identify elements, lets start the test case in debug mode, the .debug() works as a breakpoint.

describe('Swaglabs Android app test', function() {
this.tags = ['swaglabsmobileapp', 'login', 'sanity'];
this.suiteRetries(1);
this.retries(2);
this.timeout(150000);

it('Login test', async function(app) {
app
.debug()
});
});

Run the debug mode:

npx nightwatch .<test_path>/<test.js> - -env <environment> - -debug

npx nightwatch ./test/swaglabs-android.js --env app.android.emulator.test --debug

The Debug Mode allows users to pause the test at any point and use a REPL interface (made available in the terminal) to try out the available Nightwatch commands and assertions and see them get executed against the running browser, in real-time.

Typing "browser" gives you all information about commands that can be executed in debug mode, for now, let`s just use it as a breakpoint to connects with Appium Inspector!

Debug Mode

Appium server will start, followed by the android emulator, .exit ends the debug mode (it continues the test execution after the .debug() break point and so finish the test execution).

Note: Sometimes, the port used by Appium Server is busy, you can try to kill it and repeat the process again, if it does not works, reboot your OS:

ps aux | grep appium
kill -9 <process_id>
OR
pkill appium

4. Appium Inspector

Open the Appium Inspector and connect to the emulator

chmod +x Appium-Inspector-linux-2023.7.1.AppImage
./Appium-Inspector-linux-2023.7.1.AppImage

Set the Appium Server settings

Remote Host: localhost
Remote Port: 4723
Remote Path: /

Set the Desired Capabilities — JSON Representation (adapt the absolute path to yours)

{
"appium:browserName": null,
"appium:platformName": "android",
"appium:automationName": "UiAutomator2",
"appium:avd": "nightwatch-android-11",
"appium:app": "/home/popos/Projetos/appium/apks/Android.SauceLabs.Mobile.Sample.app.2.7.1.apk",
"appium:appPackage": "com.swaglabsmobileapp",
"appium:appActivity": "com.swaglabsmobileapp.MainActivity",
"appium:chromedriverExecutable": "/home/popos/Projetos/appium/chromedriver-mobile/chromedriver",
"appium:newCommandTimeout": 0
}
Appium Inspector — Settings Screen

Click Start Session to connect to the Appium Server and Android Emulator

Appium Inspector — Connected to Android Emulator and Identifying elements

With Appium Inspector connected to emulator, navigate to desired screen to check the elements (using the android emulator)

Click into the Refresh button to reload screen image (Refresh Source & Screenshots)

Click into the elements you want at the left side and check how to identify them at the right side of the application.

Appium Inspector — Inspecting elements from SWAGLABS sample application

5. Identify elements

There are many ways to identify a element, by accessibility id, by id, by xpath, by text, by name, by class name, ….

  • input user name — by accessibility id | xpath | attribute | element id | class | text | content-desc
test-Username
//android.widget.EditText[@content-desc="test-Username"]
Value
00000000-0000-0023-ffff-ffff0000002b
android.widget.EditText
Username
test-Username
  • input password — by accessibility id | xpath | attribute | element id | class | text | content-desc
test-Password
//android.widget.EditText[@content-desc="test-Password"]
Value
00000000-0000-0023-ffff-ffff0000002e
android.widget.EditText
Password
test-Password
  • login button — by accessibility id | xpath | element id | class | content-desc
test-LOGIN
//android.view.ViewGroup[@content-desc="test-LOGIN"]
00000000-0000-0023-ffff-ffff0000002f
android.viw.ViewGroup
test-LOGIN

This demo application provide some users to be tested:

  • standard_user
  • locked_out_user
  • problem_user

All them uses the same password

  • secret_sauce

Think about some test scenarios:

  • login with empty username — error message: Username is required
  • login with empty password — error message: Password is required
  • login with invalid username — error message: Username and password do not match any user in this service.
  • login with invalid password — error message: Username and password do not match any user in this service.
  • login with problem user
  • login with blocked user
  • login with valid user

Note: The problem user is used for other scenarios, the login will not be impacted. For this user, the checkout completes, but items remains in the shopping cart, they are not removed; and if you tries to type the first and last name during the checkout, the last name will overwrite the first name. So, lets skip the use of problem_user in the login testing.

More elements to identify

  • error message label — by accessibility id | xpath | element id | class | content-desc
test-Error message
//android.view.ViewGroup[@content-desc="test-Error message"]/android.widget.TextView
00000000-0000-0023-ffff-ffff0000008c
android.widget.TextView
test-Error message

The different error messages content can be identified by text

Username is required
Password is required
Username and password do not match any user in this service.
Sorry, this user has been locked out.
Identifying Error messages

Verify that login was successful by checking the element PRODUCTS

Identify login success — PRODUCTS title
  • label PRODUCTS — by xpath | element id | class | text
//android.view.ViewGroup[@content-desc="test-Cart drop zone"]/android.view.ViewGroup/android.widget.TextView
00000000-0000-0023-ffff-ffff00000096
android.widget.TextView
PRODUCTS

Now we had identified all elements we need to login scenarios, it`s time to write the test case flow: send values to the input fields, click at login button and verify/assert the presence of elements in the screen.

describe('Swaglabs Android app test', function() {
this.tags = ['swaglabsmobileapp', 'login', 'sanity'];
this.suiteRetries(1);
this.retries(2);
this.timeout(150000);

it('Login test', async function(app) {
app
.setValue("xpath","//android.widget.EditText[@content-desc=\"test-Username\"]", "standard_user")
.setValue("xpath","//android.widget.EditText[@content-desc=\"test-Password\"]", "secret_sauce")
.click("xpath","//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
});
});
});

For the PRODUCTS element, let`s also check the text content, so let`s add the xpath + text content validation, using the search option at Appium Inspector to validate the xpath:

Search a element
Element found
describe('Swaglabs Android app test', function() {
this.tags = ['swaglabsmobileapp', 'login', 'sanity'];
this.suiteRetries(1);
this.retries(2);
this.timeout(150000);

it('Login test', async function(app) {
app
.setValue("xpath","//android.widget.EditText[@content-desc=\"test-Username\"]", "standard_user")
.setValue("xpath","//android.widget.EditText[@content-desc=\"test-Password\"]", "secret_sauce")
.click("xpath","//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
.assert.elementPresent("//android.view.ViewGroup[@content-desc=\"test-Cart drop zone\"]/android.view.ViewGroup/android.widget.TextView[@text='PRODUCTS']")
.end();
});
});

6. Run tests

Remove any .debug() step from your test cases.

And run the test script to make sure it`s working:

npm run test

> test
> nightwatch -c ./nightwatch.conf.js --env app.android.emulator.test

Starting Appium Server on port 4723...


[Swaglabs Android app test] Test Suite
────────────────────────────────────────────────────────────────────────────
Using: swaglabsmobileapp on ANDROID (11).


Running Login test:
───────────────────────────────────────────────────────────────────────────────────────────────────
✔ Testing if element <//android.view.ViewGroup[@content-desc="test-Cart drop zone"]/android.view.ViewGroup/android.widget.TextView[@text='PRODUCTS']> is present (1268ms)

✨ PASSED. 1 assertions. (5.972s)
Wrote HTML report file to: /home/popos/Projetos/appium/tests_output/nightwatch-html-report/index.html

We could also start the test directly from package.json, just right click on scripts and select the Run Script option:

test script started in GUI from VSCode

7. Test report

By default, the report folder used to save the report is ./test-output/

But you can change it at nightwatch config settings or pass it with the output flag :

npm run test --output ./tests_output

Settings (./nightwatch/…/lib/settings/defaults.js):

module.exports = {
output_folder: 'tests_output'
}

Open the HTML report at ./test_output/nightwatch-html-report/index.html

Test report

The first positive test case is done.

8. Other scenarios

For log off scenario use the menu on top

  • menu — by accessibility id | xpath | class | content-desc
test-Menu
//android.view.ViewGroup[@content-desc="test-Menu"]
android.view.ViewGroup
test-Menu

For the logout option, we have a problem, when clicking in it, it`s wrong detecting an element under it instead of the logout option (it selects a label of price $29.99), and Appium Inspector doesn`t provide a way to search on app source tab.

So, let`s copy that App Source, paste into any text note application, and search for LOGOUT text

The App Source show us two lines for LOGOUT:

<android.view.ViewGroup index="7" package="com.swaglabsmobileapp" class="android.view.ViewGroup" text="" content-desc="test-LOGOUT" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[55,1399][1025,1497]" displayed="true">
<android.widget.TextView index="0" package="com.swaglabsmobileapp" class="android.widget.TextView" text="LOGOUT" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[55,1399][1025,1461]" displayed="true" />

The first one is the viewGroup (area to be tapped) and the second on the TextView (label text)

Let`s get the ViewGroup instead of the TextView (explanation some steps ahead):

<android.view.ViewGroup 
index="7"
package="com.swaglabsmobileapp"
class="android.view.ViewGroup"
text=""
content-desc="test-LOGOUT"
checkable="false"
checked="false"
clickable="true"
enabled="true"
focusable="true"
focused="false"
long-clickable="false"
password="false"
scrollable="false"
selected="false"
bounds="[55,1399][1025,1497]"
displayed="true">
  • The xpath for it:
//android.view.ViewGroup[@content-desc="test-LOGOUT"]
  • And check it into the search option
Search by XPath — XML Path

It results in the element id: 00000000–0000–00bf-ffff-ffff0000001e

Element id found

Now click into "Find and Select Source"

Get the information about the selected element, also make sure it`s interactive using the tap button (it`s the target icon, if it`s not interactive for sure we could not use that element , and must try to find other solution — in this example the ViewText can`t be target/tapped, so I decided to use the ViewGroup instead)

  • logout menu option — by accessibility id | xpath | element id | index | class | content-desc |
test-LOGOUT
//android.view.ViewGroup[@content-desc="test-LOGOUT"]
00000000-0000-00bf-ffff-ffff0000001e
7
android.view.ViewGroup
test-LOGOUT

With every element needed, let`s finish the other scenarios.

Here let`s use the .useXpath to set xpath as the default selector, and use the .end() to close app every time a test case ends.

It`s possible to replace assert with verify to finish a test case even when it fails.

describe('Swaglabs Android app test', function() {
this.suiteRetries(1);
this.retries(2);
this.timeout(10000);

it('Login with empty user name', async function(app) {
app
.useXpath()
.clearValue("//android.widget.EditText[@content-desc=\"test-Username\"]")
.clearValue("//android.widget.EditText[@content-desc=\"test-Password\"]")
.click("//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
.verify.elementPresent("//android.view.ViewGroup[@content-desc=\"test-Error message\"]/android.widget.TextView[@text='Username is required']")
.end();
});

it('Login with empty password name', async function(app) {
app
.useXpath()
.clearValue("//android.widget.EditText[@content-desc=\"test-Username\"]")
.clearValue("//android.widget.EditText[@content-desc=\"test-Password\"]")
.setValue("//android.widget.EditText[@content-desc=\"test-Username\"]", "standard_user")
.click("//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
.verify.elementPresent("//android.view.ViewGroup[@content-desc=\"test-Error message\"]/android.widget.TextView[@text='Password is required']")
.end();
});

it('Login with invalid user name', async function(app) {
app
.useXpath()
.clearValue("//android.widget.EditText[@content-desc=\"test-Username\"]")
.clearValue("//android.widget.EditText[@content-desc=\"test-Password\"]")
.setValue("//android.widget.EditText[@content-desc=\"test-Username\"]", "invalid_user")
.setValue("//android.widget.EditText[@content-desc=\"test-Password\"]", "secret_sauce")
.click("//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
.verify.elementPresent("//android.view.ViewGroup[@content-desc=\"test-Error message\"]/android.widget.TextView[@text='Username and password do not match any user in this service.']")
.end();
});

it('Login with invalid password', async function(app) {
app
.useXpath()
.clearValue("//android.widget.EditText[@content-desc=\"test-Username\"]")
.clearValue("//android.widget.EditText[@content-desc=\"test-Password\"]")
.setValue("//android.widget.EditText[@content-desc=\"test-Username\"]", "standard_user")
.setValue("//android.widget.EditText[@content-desc=\"test-Password\"]", "secret_invalid")
.click("//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
.verify.elementPresent("//android.view.ViewGroup[@content-desc=\"test-Error message\"]/android.widget.TextView[@text='Username and password do not match any user in this service.']")
.end();
});

it('Login with blocked user', async function(app) {
app
.useXpath()
.clearValue("//android.widget.EditText[@content-desc=\"test-Username\"]")
.clearValue("//android.widget.EditText[@content-desc=\"test-Password\"]")
.setValue("//android.widget.EditText[@content-desc=\"test-Username\"]", "locked_out_user")
.setValue("//android.widget.EditText[@content-desc=\"test-Password\"]", "secret_sauce")
.click("//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
.verify.elementPresent("//android.view.ViewGroup[@content-desc=\"test-Error message\"]/android.widget.TextView[@text='Sorry, this user has been locked out.']")
.end();
});

it('Login test', async function(app) {
app
.useXpath()
.clearValue("//android.widget.EditText[@content-desc=\"test-Username\"]", "invalid_user")
.clearValue("//android.widget.EditText[@content-desc=\"test-Password\"]", "secret_sauce")
.setValue("//android.widget.EditText[@content-desc=\"test-Username\"]", "standard_user")
.setValue("//android.widget.EditText[@content-desc=\"test-Password\"]", "secret_sauce")
.click("//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
.verify.elementPresent("//android.view.ViewGroup[@content-desc=\"test-Cart drop zone\"]/android.view.ViewGroup/android.widget.TextView[@text='PRODUCTS']")
.end();
});

it('Logoff test', async function(app) {
app
.useXpath()
.clearValue("//android.widget.EditText[@content-desc=\"test-Username\"]")
.clearValue("//android.widget.EditText[@content-desc=\"test-Password\"]")
.setValue("//android.widget.EditText[@content-desc=\"test-Username\"]", "standard_user")
.setValue("//android.widget.EditText[@content-desc=\"test-Password\"]", "secret_sauce")
.click("//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
.click("//android.view.ViewGroup[@content-desc=\"test-Menu\"]")
.click("//android.view.ViewGroup[@content-desc=\"test-LOGOUT\"]")
.verify.elementPresent("//android.view.ViewGroup[@content-desc=\"test-LOGIN\"]")
.end();
});
});

And run the tests to make sure everything is fine:

npm run test

Test Suite Execution:



> test
> nightwatch -c ./nightwatch.conf.js --env app.android.emulator.test

Starting Appium Server on port 4723...


[Swaglabs Android app test] Test Suite
────────────────────────────────────────────────────────────────────────────
Using: swaglabsmobileapp on ANDROID (11).


Running Login with empty user name:
───────────────────────────────────────────────────────────────────────────────────────────────────
✔ Testing if element <//android.view.ViewGroup[@content-desc="test-Error message"]/android.widget.TextView[@text='User
name is required']> is present (733ms)
✨ PASSED. 1 assertions. (2.994s)

Running Login with empty password name:
───────────────────────────────────────────────────────────────────────────────────────────────────
Using: swaglabsmobileapp on ANDROID (11).

✔ Testing if element <//android.view.ViewGroup[@content-desc="test-Error message"]/android.widget.TextView[@text='Pass
word is required']> is present (699ms)
✨ PASSED. 1 assertions. (11.237s)

Running Login with invalid user name:
───────────────────────────────────────────────────────────────────────────────────────────────────
Using: swaglabsmobileapp on ANDROID (11).

✔ Testing if element <//android.view.ViewGroup[@content-desc="test-Error message"]/android.widget.TextView[@text='User
name and password do not match any user in this service.']> is present (650ms)
✨ PASSED. 1 assertions. (12.612s)

Running Login with invalid password:
───────────────────────────────────────────────────────────────────────────────────────────────────
Using: swaglabsmobileapp on ANDROID (11).

✔ Testing if element <//android.view.ViewGroup[@content-desc="test-Error message"]/android.widget.TextView[@text='User
name and password do not match any user in this service.']> is present (702ms)
✨ PASSED. 1 assertions. (13.683s)

Running Login with blocked user:
───────────────────────────────────────────────────────────────────────────────────────────────────
Using: swaglabsmobileapp on ANDROID (11).

✔ Testing if element <//android.view.ViewGroup[@content-desc="test-Error message"]/android.widget.TextView[@text='Sorr
y, this user has been locked out.']> is present (207ms)
✨ PASSED. 1 assertions. (13.202s)

Running Login test:
───────────────────────────────────────────────────────────────────────────────────────────────────
Using: swaglabsmobileapp on ANDROID (11).

✔ Testing if element <//android.view.ViewGroup[@content-desc="test-Cart drop zone"]/android.view.ViewGroup/android.wid
get.TextView[@text='PRODUCTS']> is present (1521ms)
✨ PASSED. 1 assertions. (13.772s)

Running Logoff test:
───────────────────────────────────────────────────────────────────────────────────────────────────
Using: swaglabsmobileapp on ANDROID (11).

✔ Testing if element <//android.view.ViewGroup[@content-desc="test-LOGIN"]> is present (1731ms)

✨ PASSED. 1 assertions. (16.827s)

✨ PASSED. 7 total assertions (1m 32s)
Wrote HTML report file to: /home/popos/Projetos/appium/tests_output/nightwatch-html-report/index.html

And here it is the new test report:

Test Suite Report

An example of a test fail during execution:

───────────────────────────────────────────────────────────────────────────────────────────────────

️TEST FAILURE (3m 41s):
- 1 assertions failed; 6 passed

✖ 1) swaglabs-android

– Login with empty user name (8.457s)

→ ✖ NightwatchAssertError
Testing if element <//android.view.ViewGroup[@content-desc="test-Error message"]/android.widget.TextView[@text='Username is required!']> is present in 5000ms - expected "is present" but got: "not present" (5171ms)

Error location:
/home/popos/Projetos/appium/test/swaglabs-android.js:
––––––––––––––––––––––––––––––––––––––––––––––––––––––
19 | .useXpath()
20 | .click('//android.view.ViewGroup[@content-desc="test-LOGIN"]')
21 | .assert.elementPresent("//android.view.ViewGroup[@content-desc=\"test-Error message\"]/android.widget.TextView[@text='Username is required!']"
22 | );
23 | });
––––––––––––––––––––––––––––––––––––––––––––––––––––––
Test Execution Failed

And the fail Report:

Test Report Failed

9. Conclusion

Now you know the basics for Mobile Test Automation using Nightwatch and Appium.

Check it out the Nightwatch documentation for more information at:

Specially the section for Mobile Testing:

You will learn that some commands used for web applications will not works for native mobile, the web one uses browser as alias, and native ones uses app as alias.

Nightwatch mobile test automation

For more information about Browser Mobile testing or Desktop Browser testing instead of Application Native Mobile one, check it out our last post:

Nightwatch logo — a brown owl on a branch

The End.

--

--