Performance testing using real browsers
Quality Engineers (QEs) have taken an interest in performance testing more than ever, and it has become a key milestone activity before a product release. When a performance test activity is planned, testers focus on the system backend, frontend, or even both. When targeting the backend, the straightforward strategy is to employ the protocol-based testing approach and when targeting the frontend, the testers perform a localised browser test using browser extensions.
In today’s article, we’ll look into performance testing using real browsers (aka browser-based performance testing), which is altogether different from the backend and frontend performance testing.
The aim of browser-based performance testing is to generate the same kind of load that is generated by a protocol-based test while doing it through real browsers, similar to how a real user would use a web application in a browser.
Nope, not kidding! But first, let’s clarify some of our terminologies below:
- Frontend performance testing: a localised test to verify the actual loading, rendering, and painting of the webpage document, its assets and styling in a single browser.
- Backend performance testing: a traditional protocol-based test targeting the application API services to see how they behave under a specific user load.
- Performance test using real browsers: a performance test using multiple real browsers at scale that imitates real-world user actions to generate a specific load on the system and see how it behaves. It is also called browser-based performance testing
Frontend performance testing vs Browser-based performance testing
A frontend performance test is usually done to ensure that the end-user experience of an application is ideal. It is an interface-level test that measures round-trip metrics to identify issues like rendering speed, UI lags and sudden shifts in page elements. One of the popular tools to measure frontend performance is Google Lighthouse. If you’d like to know more about using Lighthouse to improve your site, you can look into our previous article here.
Browser-based performance uses real browsers where the test script has instructions to navigate to a page, interact with buttons or other fields, and complete a user flow. All user actions might in-turn trigger API requests from the browser to the application backend. Unlike frontend performance tests, these tests verify both the frontend and backend performances by simulating real user actions using a real browser. Additionally, browser-based tests operate in scale with multiple browsers to generate a sizeable load to test the application backend’s performance.
Some of the tools that support browser-based performance tests are:
- Element by Flood.io
- k6-browser
- LoadView
Let’s look at how to write a performance test script with one of the above tools.
Browser-based load testing with <Element>
Element is a browser-based performance testing tool which has both the open-source and commercial licences. The open-source product hosted on GitHub/NPM allows for local execution of your test script, while the commercial licence allows you to utilise Flood’s SaaS platform to run your test script at scale with a managed infrastructure with premium features like advanced reporting, graphs and analytical data.
Element supports node.js, so you can install and initialise an Element project as follows:
npm install -g element-cli
# initialise a new project
element init bbt-sample
# install node dependencies
cd bbt-sample
npm install
# dry run with generated default script
element runNow let’s rewrite the sample script. Since Element v2, the tool uses the automation library Playwright for element interactions in the browser. So, let’s update the .ts generated by Element with the below code:
import {step, TestSettings, Until, By } from ‘@flood/element’
export const settings: TestSettings = {
waitUntil: 'visible',
};
export default () => {
const URL = 'https://practicetestautomation.com/practice-test-login/';
step('Login', async browser => {
await browser.visit(URL);
// enter username value
await browser.wait(Until.elementIsVisible(By.id('username')));
await browser.type(By.id('username'), 'student');
// enter password value
await browser.wait(Until.elementIsVisible(By.id('password')));
await browser.type(By.id('password'), 'Password123')
// click submit button
await browser.wait(Until.elementIsVisible(By.id('submit')));
await browser.click(By.id('submit'));
// verify successful login
await browser.wait(Until.elementIsVisible(By.visibleText('Logged In Successfully')));
await browser.wait(Until.elementIsVisible(By.linkText('Log out')));
// take screenshot
await browser.takeScreenshot();
});
};To run the test locally, you can execute the below command:
# run test locally
element runAfter running the above command, you should see the test execution in progress and conclude with an output table in your terminal. This is a very simple test which you can update further to run a benchmark/stress test using the Test Settings configuration as mentioned in Element’s guides.
Benefits of browser-based performance testing
The concept and marketing of Browser-based testing hinges on the approach of how the test is designed — from the user’s perspective. The scripts are very similar to how a product owner or client would run a user acceptance test (UAT), and the scripts are just instructions on how to interact with different elements on the website and complete a feature goal.
Browser-based tests are useful to identify compounding issues such as slow loading on frontend and high latency on backend services leading to a degrading usability of the overall application. They are very useful for testing single-page applications with lots of application logic executed in client-side (browser). Since most performance tests are planned with a core features logic or use case, these tests can also be interpreted as an indirect functional test when the application is stressed under load.
Most tools designed for browser-based testing are also cloud-first SaaS products. With a managed infrastructure, there is little to set up or manage from the engineer’s standpoint. The execution can be scaled up to be a massive load test on demand, and with premium features on the cloud platforms, monitoring and reporting are also user-friendly, detailed and efficient.
Oh, and don’t forget, you can run on multiple browsers such as Chrome, Firefox, Edge and Safari.
Drawbacks of browser-based performance testing
Despite the hype of “testing from the user’s perspective”, testing through the browser is not technologically efficient, as browser engines consume more CPU and memory, even if the browsers are initiated in headless mode. Even in the cloud, scaling up your hardware will cost more, and this may be a limiting factor for using browser-based testing tools.
Almost all the SaaS tools with premium features in the market are geared towards moving you to their cloud infrastructure so that you can procure some commercial licences. This may be a tough call for testers who cannot afford those licences or justify their purchase, when established protocol-based testing tools are available for free in open source.
With protocol-based testing, you can also scale up the VU load without worrying too much about CPU utilisation and memory usage, than you can with real browsers. All the scenarios in browser-based testing can be covered in protocol-based testing too, and frontend performance can be adequately tested with Lighthouse too. In reality, protocol-based testing and Lighthouse does a thorough job of testing your system.
As we continuously explore such interesting topics on performance testing, there’s no definitive one-size-fits-all approach to performance testing in a project. You can select a frontend, backend, browser-based, or even the hybrid approach based on your application type and your testing needs.
Hope this article has given you a perspective on performance testing with real browsers. Feel free to drop a comment below, and stay tuned for our next article. Take care!
🧙🏼♀Team Merlin 💛
Application security is not any individual’s problem but a shared responsibility.
