Performance Testing iOS apps with Appium and XCode Instruments
Appium comes with an in-built integration to XCode Instruments when it comes to measuring the iOS App’s performance in a real device or a Simulator. Due to few limitations, not all stats are available for recording in a Simulator. In a real device, all the performance counters including GPU & Thermal measurements can be added.
Important Note: You need to have XCode installed on your Mac to do these steps.
Here’s how it's done.
- Use an existing Instrument like ‘Time Profile’, ‘Activity Monitor’ etc., or Create a custom Instrument which is having multiple instruments added to it.
For this blog, I have created a custom instrument by the name ‘BLSPerfTemplate’ which is saved amongst Custom templates. To build them start at a Blank template and add each of the instruments you are interested by using the ‘+’ button on the top right.
In this template, I have added instruments like ‘Activity Monitor’ (to track overall performance monitoring), ‘Allocations’(to track memory allocation), ‘GPU’ (to track FPS), etc.
2. Create util to start and stop performance tracking in App
class TestUtils {
constructor() {
this.TIMEOUT = 60000;
}
async startPerformance(I) {
await I.runOnIOS(() => {
I.executeScript('mobile: startPerfRecord', {
profileName: 'BLSPerfTemplate',
pid: 'current',
timeout: this.TIMEOUT,
});
});
console.log('Started Peformance Measurement');
}
async stopPerformance(I) {
I.runOnIOS(async () => {
let traceOutput = await I.executeScript('mobile: stopPerfRecord', { profileName: 'BLSPerfTemplate' });
//console.log(traceOutput)
let buff = await Buffer.from(traceOutput, 'base64');
console.log(buff);
let fs = require('fs');
await fs.writeFile(
'./perfTrace.zip',
buff,
(err) => {
if (err) throw err;
console.log('File Written');
}
);
});
console.log('Stopped Peformance Measurement');
}
}
module.exports = TestUtils;Important to notice pid: ‘current’, which would force Appium to stat only your iOS app’s process. Else the stats of the entire Device will be tracked by Appium which you are probably not interested in, on top of that the total data collected would be huge.
3. Modify your test case like this
const Login = require('../page_objects').screens.login.LoginPage;
const TestUtils = require('../utils/TestUtils');const USERNAME = codeceptjs.config.get('users').default.username;
const PASSWORD = codeceptjs.config.get('users').default.password;/**@type Login */
let loginPage;
let testUtils;
Feature('LOGIN').tag('Feature=LOGIN');Before(async function ({ I }) {
testUtils = new TestUtils();
testUtils.startPerformance(I);
//Initialize Login Page
loginPage = new Login();
await loginPage.heading.seeElement();
});Scenario('Login Test', async () => {
await loginPage.signIn(USERNAME, PASSWORD);
})
.tag('FUNCTIONAL')
.tag('APP-LOADING')
.tag('LOGIN')
.tag('LOGOUT')
.tag('POSITIVE')
.tag('RELEASE');After(async ({ I }) => {
testUtils.stopPerformance(I);
})
Note: This is CodeceptJS based code example, Please modify it according to your project requirements. Also, in CodeceptJS you can achieve the same thing it using custom recorders to avoid repetition in code. Keeping the audience in mind, don’t want to make this example complex.
4. Start Appium using this option in the background, and also ensure the driver is ‘XCUITestDriver’
./node_modules/.bin/appium --log-timestamp --log-no-colors --allow-insecure=perf_recordNote: You can find an example on setting up iOS Appium tests here
That’s it. Now when you run your test you can find the performance measurements in perfTrace.zip (in the example). Open it to find the .trace file which you can open using XCode by Double Clicking on it.
Do note that results generation is a computationally-intensive process, please ensure your Test runner has high RAM and CPU.
Happy Test Automation!
