SDET: Quick introduction to Allure reports with Playwright
Hello dear quality lovers,
It’s no secret that Allure stands out as one of the most popular frameworks for test reporting in the world of software development. Allure has captured the attention of developers and QA engineers alike. Its versatility in integrating with numerous testing frameworks, alongside its ability to beautifully visualize test outcomes, makes it an indispensable tool for enhancing the quality assurance process. Personally, I loved it from the first time I saw it, and no matter what programming language or test library I use, I always try to find a way to use Allure. Yes, a bit addictive :)
Today we are going to have a quick look at integrating Allure Reports with another very popular test automation solution today, Playwright. Let’s don’t waste time and jump into the Allure universe
How to install
Playwright quick installation
I assume you already know how to install Playwright. I don’t want to stop here too often. If you don’t, the official documentation has a few small steps on how to do it: https://playwright.dev/docs/intro#installing-playwright.
The next step is to install the allure-playwright library -https://www.npmjs.com/package/allure-playwright
Allure-playwright library installation
From the documentation you can see that you can install it using either “NPM” or “YARN”. I’d prefer to use NPM and run the following command:
npm i -D allure-playwrightIn package.json, you may find the latest stable version of the package dependency has been installed.
Note: In my case, I installed beta because I needed to verify the fixed bug. However, it is better to install the stable version using the command above.
Allure command line installation
Now we need to install Allure command line. This tool will help us generate allure reports from test results and run allure server with nice reports.
npm i -D allure-commandlineNote: You may want to install alure-command line globally if you plan to use it for multiple projects. Please note that if a package is installed both globally and locally within a project, the local version will take precedence when you run the package’s commands from within the project directory.
npm i -g allure-commandlineJava requirement for Allure reports opening
Nobody likes to talk about it, but running the Allure reports on the server requires the installation of Java version 8+. So if you want to run it locally, you probably don’t have a choice.
- Install Java https://www.java.com/en/download/help/download_options.html
- Add Java to environment paths
https://www.java.com/en/download/help/path.html
3. Verify java was installed
java -version
We are done here. Now we need to update/extend the Playwright configuration with Allure Reports.
Extend Playwright configuration
Playwright easily supports multiple reports. All we need to do is add a new one or replace the old one. Depends on your needs :)
What you need to do is add “allure-playwright” as an array element or a single report. Again, it depends on your needs.
That’s it. Very simple. We will look at some additional options later. Now let’s try to run the tests and generate some reports. `
Run tests and reports
Run tests
This is related to Playwright, so I won’t dwell on it for long. We will just run all the tests. You should remember that you may have more than test project configurations in “playwright.config”. So you may only want to run some specific ones.
npx playwright testWhen the test run is complete, you will find a new folder in the Playwright test project root called “allure-results”.
You may already see a spoiler. There is a second folder called “allure-report”. This is a folder that will be created using one of the allure commands.
Generate allure-report
Let’s run the following command to generate reports based on the results.
allure generateNote: If you want to have only latest reports, you may want to use “ — clean” flag.
allure generate --cleanAt the end, your allure-report folder will contain the following data required to run allure server
We are close to seeing our first reports. Let’s go and do it
Run allure server
Command is very simple
allure serveThe system should automatically redirect you to the website with the nice reports.
Nice? Yes, it is. But we can do better. When we open our suites, you may notice that it uses test file paths as suite names.
Let’s think about how we can change it and even more :)
Additional configuration and attributes
Report configuration options
This is not a complete list of the options available, but let’s look at some of them
outputFolder - if you want to change location of the generated test-results
detail - hides/displays hooks steps in report
suiteTitle - if you want to use tesy title instead of allure suite.
Let’s try to disable “suiteTitle” and add custom ones to test ourselves
Allure suite
I have updated the configuration to disable suite titles
Now I’m going to add an “allure.suite” attribute with a suite description
Now, I will do the same as we did before
- Run the tests again
- Will generate reports based on results
- Will open a server with reports
Ones we will navigate to report we my find to things
- Out suite value was applied
- Default suite names values were taken from “test.describe”
This means that you may not need to add an “allure.suite” attribute, just a proper name for the features.
There are many more interesting attributes you might want to try. All the details can be found in the official documentation: https://www.npmjs.com/package/allure-playwright. For example, you might be interested in “allure.step”, “allure.label”, “allure.story”, “allure.link”, “allure.issue”, “allure.attachment”, “allure.parameters”.
Let’s look at two of the most interesting ones: “allure.issue”, “allure.attachment” and “allure.parameters”
Allure issue
You may want to show that your test failed with a specific problem, to show that we know about a bug.
All you need to do is add an attribute “allure.issue” with the description and link to the issue. Like this.
test('Verify page content', async ({ abTestPage }) => {
//Arrange
await allure.issue('Bug description', 'https://github.com/dneprokos/node-rest-services/issues/29');
});
Let’s do a test run, report generation and reopen.
When we open our test run details, we may find a link to the bug in the attachment. If we click on it, it will take us to the link provided
Let’s go to the example of an attachment
Allure attachment
a) Firstly, I would like to point out that screenshots/videos are attached by default if you have enabled this in the configuration.
Let’s look at the screenshot attached to the test failure.
All we need to do is define screenshot taking in “playwright.config”. As an example, we will only run it on failure
Now let’s repeat all the steps to generate reports.
You can find our attached screenshot in the test details. You can expand/collapse it.
This is just the default behaviour. Let’s look at another example where we need to add something custom to the test.
b) I have a test that verifies page names and links.
What I wanna do is to attach JSON array with an objects of name and url path. Let’s try to do it.
- In my implementation, I have a method that returns array of page data with name and urlPath. So, basically we already have a list of the objects
- Everything I need to do, it just convert it to JSON with help of “JSON.stringify”.
- We use “allure.attachment” in order to attach our JSON to the test
const actualPagesData: PageData[] = await mainPage.getPagesData();
await allure.attachment("ATTACH_ACTUAL_PAGES", JSON.stringify(actualPagesData), {
contentType: "application/json",
});
- Once we will run and regenerate a report, we can find the following JSON in the attachment
JFY: If you want to disable the before and after hooks, just set detail to false for the allure-playwright properties. In my case, I just want to see them :)
Allure parameters
Let’s imagine our test accepts parameters. In my case, I will take a simple page that shows what key has been pressed.
- Let’s imagine I want to test some list of the keys we press.
- For each of the test run I will attach specific parameter
const KEYS = ['a', 'Shift', 1, 'Enter', 'Space']
KEYS.forEach(key => {
test(`Press ${key} key and make sure it was pressed`, async ({ page, keyPressesPage }) => {
// Arrange
await allure.parameter("KEY", key.toString());
const expectedValue = `You entered: ${key.toString().toUpperCase()}`;
// Act
await page.keyboard.press(key.toString());
const actualValue = await keyPressesPage.getEnteredKey();
// Assert
expect(actualValue).toBe(expectedValue);
});
});- Once we will run and regenerate a report, we can find out parameter was attached as a label and added on test details
Important: Also parameter takes an third optional parameter with the hidden and excluded options: mode: "hidden" | "masked" - masked hide parameter value to secure sensitive data, and hidden entirely hide parameter from report
excluded: true - excludes parameter from the history
We are almost at the end. But I cannot let you go without a bonus section. It means we are going to run everything in docker containers.
Bonus: Run tests and reports in docker container
You may already know how to install Docker. If not, you should follow this path: https://docs.docker.com/engine/install/.
I also recommend installing docker-compose: https://docs.docker.com/compose/install/
Dockerfile
Next, you need to create a dockerfile with the following commands inside:
# Use the latest Node.js image as the base
FROM node:latest
# Set the working directory to the root of your project
WORKDIR /usr/src/app
# Copy package.json and package-lock.json (or yarn.lock) from the root directory
COPY package*.json ./
# Install application dependencies including Playwright
RUN npm install
# Install dependencies for Playwright tests
RUN npx playwright install --with-deps
# Install Allure command-line tool for report generation
RUN npm install -g allure-commandline
# Install a simple HTTP server to serve the Allure report
RUN npm install -g http-server
# Copy the entire project to the working directory
COPY . .
# Run the build script for your application (if necessary)
RUN npm run build
# Run tests and generate raw Allure results
# Using `|| true` to ensure that the docker build doesn't fail if tests do
RUN npm run test || true
# Generate Allure reports from the raw results
# The results are in the current directory (./allure-results)
RUN allure generate ./allure-results --clean -o ./allure-report || true
# List the contents of the allure-report directory for debugging
RUN ls -al ./allure-report
# Expose port 8080 for the Allure report server
EXPOSE 8080
# Serve the Allure report on container start
# The report is generated in the current directory (./allure-report)
CMD ["http-server", "allure-report", "-p", "8080", "-c-1"]This file will create an image that runs tests, generates allure results and opens report inside of the container.
In my case, I’m also going to add compose file. In this case, it is maybe overwhelm, because we are going to use only one service. Anyway, I’d prefer this way.
Docker-compose preparation
I called my previous file as “Herokuapp-tests-dockerfile”, so I need to specify concreate name to “dockerfile”. Also in my case, local port 8080 was already used, so I decided to map it to port 9000.
version: '3.8'
services:
app:
build:
context: .
dockerfile: Herokuapp-tests-dockerfile
ports:
- "9000:8080"Running docker-compose
Finally we can run docker compose with the following commands.
# Build the Docker images without using the cache
docker-compose -f herokuapp-docker-compose.yml build --no-cache
# Start the services as defined in the Docker Compose file
docker-compose -f herokuapp-docker-compose.yml upFirst command will build our docker container and second will run it from the docker-compose. Once run has finished, you may want to navigate to “localhost: 9000” and see results on allure server.
This one was just a sample template and you may want to modify it according to your needs.
Ending
As I said in the beginning I’m very big fun of Allure reports. In this article we tried to combine/integrate two very popular things: Playwright with Allure Reports. Anyway, it was only high-level overview. I would recommend you to read documentation and try more options: https://www.npmjs.com/package/allure-playwright. In this case, you will have more horizons/options to build reports applicable to your specific needs. Thanks a lot for reading and good luck in your journey.
