SDET: Introduction to the first Playwright project

Kostiantyn Teltov

·

Follow

23 min read

·

Mar 20, 2024

--

4

Listen

Share

I’m sorry, some people have already told me that I like AI-generated images. I would say that I like how they make your life easier. I used to spend a lot of time looking for the right image for my article and sometimes it took a long time. Now it is easier :)

But this article is not about AI-generated images. The idea to write this article came when people started asking me questions about starting a test project with Playwright. Then I realized that even though Playwright is a very obvious and direct library/framework, people still need some guidance in the beginning. That’s why I decided to collect these questions and create a kind of beginner’s guide/template for those who just want to create their own project.

Usually I have reviewed some of the most interesting features in Playwright. In this article, we are not going to cover all the most famous features that exist in Playwright. We will only touch on some basic setups and future enhancements. That should be enough to start your journey. Let’s begin our journey!

Choosing programming languages for Playwright

Yes, this is one of the most frequently asked questions. What programming language should I choose or can I choose any programming language I like?

At this time, Playwright officially supports the following programming languages JavaScript/TypeScript, C#, Java, Python — https://playwright.dev/docs/languages

I don’t want to waste a lot of time here and just tell you that in my opinion it is better to use NODE languages (JavaScript/TypeScript). The reason is non Node versions are limited compare to original one versions. As an example, NODE versions comes with support of it’s own Test Runner.

I already wrote an article before with review of the NET version of Playwright. You may see more details about it here: Playwright with NET

Of course, this is just my personal opinion and would be glad to hear some successful experiences of working with Playwright on other languages.

Initial condition

First what we need to do is to install NODE.js and Visual Studio Code(IDE).

NODE JS

What is it? In simple word, it is just an environment where we run JavaScript code. In a era before NODE, we could only run JS in browsers. But a smart software engineer named Ryan Dahl took a V8 engine (engine of Chromium browser) and put it inside the C++ program. That’s a short version of the story how NODE JS was born.

In order to do it on our local machine we need to install it. Just download and process installation: https://nodejs.org/en

Use CMD to verify if node installed. Use a simple command

node --version

Visual Studio Code

Visual Studio Code is a recommended IDE (Integration Development Environment) for Playwright. Nothing to add here. We just need to install it

I also recommend that you install a Visual Studio Code extension called “Playwright Test for VSCode”. This extension is one of the ways you can run the tests and also gives you the ability to record the tests. Yes, you can record your steps. I think this is one of the killer features in Playwright.

Done? Let’s move to installation…

Installation

• Create a folder with a project name. In my case I called it “playwright-article”.
• Open CMD and run the following command

npm init playwright@latest

• Playwright will ask you a few questions like:

“Do you want to use TypeScript or JavaScript?” — I’d prefer TypeScript to JavaScript because there’s less anarchy.

“Where do you want to put your end-to-end tests?” — The test-runner will recognize this directory as the tests directory. You can always change or expand it in the future.

“Add a GitHub actions workflow?” — I don’t need it right now.

“Install Playwright browsers” — I would recommend installing them from the beginning. At least these browsers are required if you want to record the tests

In simple words I have chosen all default options.

Important: Some of the installation steps or default options maybe changed in time. That’s why I don’t want to concentrate a lot on answers. You may always find full information on official documentation: https://playwright.dev/docs/intro#installing-playwright

You need to wait a little bit until installation in done.

Time to run the tests

The default installation includes sample tests. It means we can run it and see. There are three ways how you can run the tests

Disable Headless mode

But by default, Playwright runs all tests in headless mode. But I think the first time you run it, you want to see all the operations with your eyes.

It is very easy to do. Just go to “playwright.config” and add “headless: false” to “use” section of config or you can do it specifically for the project. As an example, you may want to disable headless only for “chromium” browsers.

Run tests from command line

1. Open CMD in the root of the project. You may already want to use Terminal of the Visual Studio Code.
2. Run the following command

npx playwright test

As you can see, we ran 6 tests. The file “example.spec.ts” contains only 2 tests. Why 6? The answer is simple, you were running 3 projects configured inside the “playwright.config.ts”. You can remove some of them if you don’t need them. We will talk about them later.

There are much more filter options you may you during test run. As an example, you may run only specific project/projects or specific test or specific tags. Please find more information in official documentation.

Note: Additionally you may want to the report configured by default. Do it if you want. You may see a command in a logs.

Run tests from Playwright Extension for VS Code

Here is very simple step. Select “Testing” panel and click “Run Test” arrow button.

Run tests from Playwright UI mode

1. First, you need to start this mode by entering the following command

npx playwright test --ui

2. Once this mode is open, you need to click “Run” arrow button

Note: This mode has more than just test-run fixtures. This is an introductory article, so I won’t describe all the features. All I wanna say you should definitely look at the filters, logs and Action/Before/After states

For more details, you can read the official documentation or the article where I already reviewed this mode

Next, I want to touch a little bit on projects and browsers in the configuration

Projects and browsers

Here is important to say that Playwright supports like browsers installed with Playwright and also branded browsers installed on your machine.

• Playwright installed browsers are actually a builds of the browsers. You should always remember to update these browsers together with playwright. These browsers are: Chromium, WebKit (Safari build) and FireFox
• Branded browsers are just your local browsers. Supported browsers are: Chrome, FireFox, Microsoft Edge.

Additionally it supports mobile emulation. Only emulation, not real devices.

More information about browsers you may find in official documentation

For our example, I will configure 2 projects. One will be running tests using Local Chromium Device and second will use Chromium build.

Let’s go back to our configuration and will left only two projects:

projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'Google Chrome',
      use: { ...devices['Desktop Chrome'], channel: 'chrome' },
    },
  ],

Note: Just added “channel: ‘chrome’ ” property in order to run with local chrome browser.

If you run the tests again, you will see that they run on local Chrome and Chromium build browsers. You may play a little bit with a different browsers configuration. Full list of the supported browsers and device emulation maybe found here.

It is time to ride to our tests

Tests structure

Test structure does not look so scary as you may think.

Test function and fixtures

• On top level, we have a “test” function that was imported from the “@playwright/test’”
• The test function is invoked with two parameters: a descriptive string that names the test case, “get started link”, and an asynchronous callback function that contains the logic for executing the test.
• Inside of the callback function we inject a ‘page’. This one is of the most important elements of the Playwright. With help of this page we perform all our actions. This page is isolated for each of the test.

import { test, expect } from '@playwright/test';

test('get started link', async ({ page }) => {
  await page.goto('https://playwright.dev/');

  // Click the get started link.
  await page.getByRole('link', { name: 'Get started' }).click();

  // Expects page to have a heading with the name of Installation.
  await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
});

Here it is important to remember that besides “page” Playwright also contains other fixtures that can be injected.

page (Page) - Isolated page for this test run.
context (BrowserContext) - Isolated context for this test run. The page fixture belongs to this context as well. Learn how to configure context.
browser (Browser) - Browsers are shared across tests to optimize resources.
browserName (string) - The name of the browser currently running the test. Either chromium, firefox or webkit.
request (APIRequestContext) - Isolated APIRequestContext instance for this test run.

To summarize, you can inject playwright fixture as callback function parameter like this

test('get started link', async ({ page, request }) => {
  await page.goto('https://playwright.dev/');

  // Click the get started link.
  await page.getByRole('link', { name: 'Get started' }).click();

  // Expects page to have a heading with the name of Installation.
  await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
});

Test actions and locators

What is a test itself? It is just a sequence of operations with verifications/assertions! When we talk about tests interacting with UI components, it means that we need to know how to interact with them. To do that, we need to find UI component locators. That’s basically how our tests build.

• First we need to find a locator of the element we want to interact with.

Here is important to know that playwright supports like it’s own locators and also standard XPATH and CSS selectors. Here it will be better to read documentation about it because it is not a topic of this article.

• Then we need to perform an action with this element. We want to click, we want to enter some value to the field or we want to drag and drop.

The most basic actions are:

fill() — Input text
check() — Check checkbox
selectOption() — Select from drop-down
click()/dblclick() — Click button/link
hover()/focus() — hover mouse over element/focus on element
dragTo() — Drag and drop operation
press() — When u need to press a keyboard button
innerText()/allInnerTexts() returns element text/texts

Here it is important to understand that actions are async operations. This means that you must use “await” before calling an action function.

Let’s consider what our test does by lines

1. We await until browser navigates to the “‘https://playwright.dev/'” URL. Important to understand it does not wait until this page is opened.

await page.goto('https://playwright.dev/');

2. Second step is we find our “Get started” link on the page and click on it. This step is exactly about waiting for the link on the page and then just clicking it. Also I decided to update a little bit our code by setting a maximum timeout of 10 seconds (default is 5 but it can be changed in config) to wait for this button. These actions can be useful if you don’t want to update the default timeout and set only for specific case.

await page.getByRole('link', { name: 'Get started' }).click({timeout: 10000 });

3. Our last step is a action that makes our test as test. Yes, I’m talking about assertion. In this step again we have an async operation and we are waiting until our Title with a name “Installation” is visible. It will fail if our locator will not be visible in some time.

await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();

Note: Here is important to remember, Playwright already contains assertion framework inside. Once again, I would recommend to learn about assertions in the official documentation

Test feature

In default example we have two separated tests in one file. What if we want to say our tests belongs to one feature? In this case we need to move our tests under another function with callback “test.describe”

test.describe('Playwright Demo', () => {

  test('has title', async ({ page }) => {
    await page.goto('https://playwright.dev/');

    // Expect a title "to contain" a substring.
    await expect(page).toHaveTitle(/Playwright/);
  });


  test('get started link', async ({ page, request }) => {
    await page.goto('https://playwright.dev/');

    // Click the get started link.
    await page.getByRole('link', { name: 'Get started' }).click({ timeout: 10000 });

    // Expects page to have a heading with the name of Installation.
    await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
  })

});

In one file, we may have a few few fixtures if we want. In test explorer you will see the following hierarchy

Hooks

Sometimes you want to perform some steps before running test itself. Playwright also supports such possibility. We can do it will help of the following hooks:

• test.beforeAll — Runs before all tests of the feature
• test.beforeEach — Runs before each test of the feature
• test.afterEach — Runs after each of the tests
• test.afterAll — Runs after all of the tests

In the example below, I’ve added navigation to “https://playwright.dev/” URL inside of the “test.afterEach” hook. It means, I don’t need to do it inside of the tests. So, I just removed it.

test.describe('Playwright Demo', () => {
  
  test.beforeAll(async ({ page }) => {
    console.log('beforeAll');
  });

  test.beforeEach(async ({ page }) => {
    await page.goto('https://playwright.dev/');
  });

  test.afterEach(async ({ page }) => {
    console.log('afterEach');
  });

  test.afterAll(async ({ page }) => {
    console.log('afterAll');
  });

  test('has title', async ({ page }) => {
    // Expect a title "to contain" a substring.
    await expect(page).toHaveTitle(/Playwright/);
  });

  test('get started link', async ({ page, request }) => {
    // Click the get started link.
    await page.getByRole('link', { name: 'Get started' }).click({ timeout: 10000 });

    // Expects page to have a heading with the name of Installation.
    await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
  })
});

Global Test Setup and Teardown

In the example above, we executed hooks inside the file. This means that if we want to do it in another test file, we will repeat it. In some cases, we may want to do a global setup or/and teardown

First, let’s create a files with setup and teardown logic e.g. “global-setup.ts” and “global-teardown.ts”

Inside of these files will will create simple functions

async function globalSetup() {
    console.log('Global setup completed');
}
  
export default globalSetup;

async function globalTeardown() {
  await stopDockerContainer();
}

export default globalTeardown;

Yes, I have simplified the logic for now. The next step is to use it in our configuration. Let’s open our “playwright.config.ts” file. All we need to do is add our functions to “globalSetup” and “globalTeardown”. Both are part of the config root options.

export default defineConfig({
  testDir: './tests',
  globalSetup: require.resolve('./global-setup'),
  globalTeardown: require.resolve('./global-teardown'),
  ..............
  use: {
    headless: false,
    ................
  },
  projects: [
    ................
  ]
})

Now it will run this logic before your test runs. It maybe useful when you need to prepare some data or start some services.

Additional Test Annotations

There are a few test annotations I also want to quickly highlight

• test.skip — Maybe useful when you want to skip a test or skip by some condition

• test tags — which can be set as within the test name and also using a ‘tag’ property. Think about it like test categories.

Please find more annotations inside of the documentation

Before we jump into the structure of the test project, I want to stop and take a quick look at the test recorder.

Tests recording

This is a killer feature especially for those who want to start. Also, it may be useful if you stack with some logic implementation. You can perform a real user actions and Playwright will record these steps and even some limited assertions for you. Sounds like a dream? Let’s get started!

Start recording

• Open “Tests” tab in Visual Studio Code and click “Record new” button

• It will start a Playwright build browser.

• Everything you need to do is just to perform an actions

1. As an example, I’m going to navigate to the following URL “https://the-internet.herokuapp.com/login”.

2) Enter username, password and click a “Login” button like I do as just regulars user

3) In the end I will verify that page is opened by asserting “Secure Area” text. It will be enought for this sample

4) Finally, I will stop recording button

5) Once I will go back to the Visual Studio Code, I may find a new file with recorded steps

import { test, expect } from '@playwright/test';

test('test', async ({ page }) => {
  await page.goto('https://the-internet.herokuapp.com/login');
  await page.getByLabel('Username').click();
  await page.getByLabel('Username').fill('tomsmith');
  await page.getByLabel('Password').click();
  await page.getByLabel('Password').fill('SuperSecretPassword!');
  await page.getByRole('button', { name: ' Login' }).click();
  await expect(page.locator('h2')).toContainText('Secure Area');
});

Note: There are few unnecessary step were recorded. This was happened because Playwright recorder exactly recorded our actions. These actions were of selecting our fields to enter user name and password. You may removed these steps if you want. Test will work without them

await page.getByLabel('Username').click();
await page.getByLabel('Password').click();

You can play a little more with a recorder. At least now it does not scare you. And we continue our journey. The next step is to consider the project structure in general.

Basic project structure

In many ways, the structure of your test project depends on your needs. Here I will describe the most common parts you might have.

• ‘tests’ folder — you may already familiar with that. This is our test files location. Remember by default playwright will detect and run only tests inside of the root tests directory. You can add more options using ‘testMatch’ inside of the configuration file.

testMatch: ['tests/*tests.ts', 'tests/**/*tests.ts']

• pages — This is our page objects. Probably the most important test design pattern in UI test automation. We will consider small example below
• test-data — If you need to store some predefined test data. Personally I’m not a big fan of the static test data. Anyway, in some cases I may need some predefined files e.g. “.png”
• services — If you need to use some helper services like API or DB requests
• framework — This is very basic logic not directly related to your project. As an example you may have some basic implementations, helpers and other items potentially can be shared to any project.

Let’s take a quick look at each of these.

Framework

Core implementations not related to any of the projects

• Constants
• Environment resolvers
• Basic models
• BasePage
• TestData generators
• Randomizers
• And other…

In the example below you may find a very simplified implementation of the BasePage. In this future, I will have opportunity to use it in a different projects when I build page objects

import { Page } from "@playwright/test";

export class BasePage {
    page: Page

    constructor(page: Page) {
        this.page = page;       
    }

    public async navigateUrl(url: string): Promise<void> {
        await this.page.goto(url);
    }

    public async waitForPageLoad(): Promise<void> {
        await this.page.waitForLoadState('domcontentloaded');
    }
}

Pages

I already mentioned page object is one the most important test design patterns in UI test automation. Let’s quickly look at the basics. We just encapsulate pages logic inside of these classes.

As an example, I moved our form authentication steps to this page object and now I can re-use it.

export default class FormAuthenticationPage extends BasePage {
    constructor(page: Page) {
        super(page);
    }

    async getUsernameAndPassword(): Promise<[string, string]> {
        const nameAndPassword: string[] = await this.page.locator('h4.subheader > em').allInnerTexts();
        const username = nameAndPassword[0];
        const password = nameAndPassword[1];
        return [username, password];
    }

    async performAuthentication(username: string, password: string): Promise<void> {
        await this.page.fill('#username', username);
        await this.page.fill('#password', password);
        await this.page.click('button[type="submit"]');
    }

    async getWelcomeMessage(): Promise<string> {
        return await this.page.textContent('h4.subheader') as string;
    }

    async isLogoutButtonDisplayed(): Promise<boolean> {
        return await this.page.isVisible('a >> text="Logout"');
    }

    async clickLogoutButton(): Promise<void> {
        await this.page.click('a >> text="Logout"');
    }
}

Test-data

Maybe you will use static test data or maybe not. Anyway I decided to mention about this folder. In general it is just container to test data. You may split it inside if you need it. It depends on what you need.

Services

UI tests the most unstable tests in Pyramid Hierarchy. Usually we need to verify some specific area/feature and we are not interested how we did setup or cleanup. In such cases we may need to use some integration layer services like database or API calls. This folder basically collects these services.

In the example below, I have a service that makes API request GetGenreById, verifies that status code is 200 and returns body.

export class GenresApiRequests {
    private baseUrl: string;
    private resourceUrl: string;

    constructor(baseUrl: string) {
        this.baseUrl = baseUrl;
        this.resourceUrl = `${this.baseUrl}/genres`;
    }

    async getGenreById(id: number, bearerToken: string): Promise<GenreApiModel> {
        const context = await request.newContext();

        const urlWithParams = `${this.resourceUrl}/${id}`;

        const response = await context.get(urlWithParams, {
            headers: {
                'accept': 'application/json',
                'authorization': `Bearer ${bearerToken}`
            }
        });

        expect(response.status()).toBe(200);

        const responseBody: GenreApiModel = await response.json();

        return responseBody;
    }

}

Note: You already know, that requests is a part of Playwright fixtures.

Playwright fixture extension

This is one of the features I really like in Playwright. Do you remember the fixture injection we used in Playwright before? No? When we injected a page or a request. The cool thing is that we can extend it and inject whatever we need.

Let’s consider some example. First we will consider page object usage without fixture. We already have an page object for the form authentication and we initialize it inside of our test.

import FormAuthenticationPage  from '../pages/form-authentication-page';
import { test, expect } from '@playwright/test';

test('test', async ({ page }) => {
  //Arrange
  await page.goto('https://the-internet.herokuapp.com/login');
  const formAuthPage = new FormAuthenticationPage(page);

  //Act 
  await formAuthPage.performAuthentication('tomsmith', 'SuperSecretPassword!');

  //Assert
  await expect(page.locator('h2')).toContainText('Secure Area');
});

In general it does not look so bad. But we can do even better. We will make our page part of the Playwright fixture and will have possibility to inject it inside of our callback function.

Let’s do it. I will create a file called “fixture-extention”. Name does not matter.

Inside of this file I will add the following logic

• I need to differentiate new exported test from the default one. So I will import “test as base”. I’m going to extend this base in below
• Next I’m creating a new type called “Pages” and declaring our formAuthenticationPage. In the future, I may want to add more pages declaration inside
• Finally I can extend our base by adding lazy initialization of the our page. Once more, potentially we can have more pages. More than that, you can add some additional logic inside if you need it. As an example, you may want to perform navigation action when you use this page. And don’t forget to export this new test.
• In the end I’m exporting ‘expect’ because I want to use assertion from this extension

import { test as base } from '@playwright/test';
import FormAuthenticationPage from './pages/form-authentication-page';

type Pages = {
    formAuthenticationPage: FormAuthenticationPage
}

export const test = base.extend<Pages>({
    formAuthenticationPage: async ({page}, use) => {
        await use(new FormAuthenticationPage(page));
    },
});

export { expect } from '@playwright/test';

Let’s refactor our test

• I will import our extension instead of default playwright library.
• I will inject “formAuthenticationPage” as callback function parameter
• I don’t need to initialize it. I can use it like it is.

import { test, expect } from '../fixture-extention';

test('test', async ({ formAuthenticationPage, page }) => {
  //Arrange
  await page.goto('https://the-internet.herokuapp.com/login');

  //Act 
  await formAuthenticationPage.performAuthentication('tomsmith', 'SuperSecretPassword!');

  //Assert
  await expect(page.locator('h2')).toContainText('Secure Area');
});

Yes. I’m also tired to write this article. We still have a few important sections . Let’s jump to the configuration chapter

Additional configuration

Cool thing in Playwright it supports a lot of options out of the box. You don’t need to perform additional implementations. You just need to configure it.

First let’s consider additional playwright config options you may be interested in. We can split them on Test Configuration and Test Use options. Next I will recommend to see the next options for both. Please remember it is not final list.

Root config options (Test configuration)

I would recommend to pay attention on the following settings.

testMatch - If you need to discover and run more test directories
timeout - timeout for each tests in milliseconds
retries - The maximum number of retry attempts given to failed tests.
workers - The maximum number of concurrent worker processes to use for parallelizing tests.
reporter - The list of reporters to use. 
globalSetup - Path to the global setup file. This file will be required and run before all the tests.
globalTeardown - Path to the global teardown file. This file will be required and run after all the tests.
globalTimeout - Maximum time in milliseconds the whole test suite can run. Zero timeout (default) disables this behavior.

Test use options

This options can be set both globally or locally for each test project. Local setting will override a global one. I would recommend to pay attention on the following settings.

baseURL - This URL will be used as basic one when you perform an actions like page.goto()
headless - Whether to run browser in headless mode.
viewport - Emulates consistent viewport for each page.
screenshot - Whether to automatically capture a screenshot after each test
video - Whether to record video for each test.
browserName - Name of the browser that runs tests.
timezoneId - Changes the timezone of the context
geolocation - Changes the geolocation of the context
permissions - A list of permissions to grant to all pages in this context. As an example if you want to accept all popup without confirmation
connectOptions - A list of connection options. As an example, when you need to run tests against some grid like BrowserStack and etc.
launchOptions - Options used to launch the browser, as passed to browserType.launch([options]). As an example "slowMo" option helps in debugging or logger options help to configure a log

Nice to mention: Screenshots and video

Cool part of the Playwright, you don’t need to implement screenshot or video recorder. They were already build inside. You just need to enable them with some option inside of the configuration. In in the example below, I enabled them on failure and on first retry.

screenshot: 'only-on-failure',
video: 'on-first-retry'

Nice to mention: Log launch options

Launch options allow you to configure a logger. In the example below, I just used a console log example. If you want to use a custom logger or save logs to a file, you can use this logic instead.

launchOptions: {
            slowMo: 100,
            logger: {
                isEnabled: (name, severity) => name === 'browser',
                log: (name, severity, message, args) => {
                    const severityThreshold = 'info'; // Example threshold
                    const severityLevels = ['verbose', 'info', 'warning', 'error'];
                    if (severityLevels.indexOf(severity) >= severityLevels.indexOf(severityThreshold)) {
                        console.log(`[${name}] ${message}`);
                    }
                },
            },
        }

Combining with DOTENV

The dotenv library is a zero-dependency module in Node.js that loads environment variables from a .env file into process.env. This makes it easier to manage configuration settings and secrets, such as database passwords or API keys, outside of your application’s source code.

And you can easily combine it with your Playwright config

• It is easy to install it with the following command

npm install dotenv --save

• Next, we can create a “.env” file and add some sensitive and not only info inside

#Browser variables
HEADLESS_BROWSER=true

#Parallelism, retries, timeouts
WORKERS=20
RETRY_FAILED=2
MAX_TEST_RUNTIME=60000

#Environment
BASE_URL=https://the-internet.herokuapp.com

// API
MOVIES_API_BASE_URL=http://localhost:5000/api

• We can already read it from the “process.env” like this

const envValue = process.env['BASE_URL'];

• But I would recommend you to create some helper wrapper functions inside of the framework in order to verify, read and convert it

require('dotenv').config();

function getEnv(name: string): string {
    const envValue = process.env[name];
    if (envValue == null)
        throw new Error(`${name} environment variable is not defined.`);
    return envValue;
}

function getEnvParseNumber(name: string): number {
    const envValue = process.env[name];
    if (envValue == null)
        throw new Error(`${name} environment variable is not defined.`);
    return Number.parseInt(envValue);
}

export { getEnv, getEnvParseNumber };

• Here is an example of usage inside of the config file. But you can call anywhere

retries: getEnvParseNumber(EnvironmentParameters.retryFailed)?? 1,
workers: getEnvParseNumber(EnvironmentParameters.workers)?? 10,

We are almost in the end. I also want to briefly touch on reporting and CI.

Reporting and Continues Integration

I’m not going to spend a lot of time here. So let’s start from reporting

Reporting

Playwright Test comes with a few built-in reporters for different needs and ability to provide custom reporters.

• When you install playwright it automatically configured with default reporters. You’ve seen this command in the beginning.
• You can also add another reporters as an example “allure-playwright”. You may find more details in this article

reporter: [
        ['junit', { outputFile: 'test-results/results.xml' }],
        ['allure-playwright', {
            details: true,
            outputFolder: './allure-results',
            suiteTitle: false,
            environmentInfo: {
                FRAMEWORK: 'Playwright',
                HEADLESS: Headless,
                E2E_NODE_VERSION: process.version,
                E2E_OS: process.platform },
        }],
        ['list']
    ],

Please find more reports description in the documentation

Continues integration

Let’s consider high-level case. In order to run tests on CI you need to do:

• To have an Agent/Container/Slave with NODE JS
• (Optional) Potentially you may need to install some packages required for running helper libraries. As an example, if you want to use Allure reports you also need to install Java
• You perform checkout of you repo with “npm install” command
• You perform installation of the playwright browsers with dependencies “npx playwright install — with-deps”

Note: In some cases you may want to use some browsers provider. I have an article with an example how to configure it.

• Next we are running tests with a command “npx playwright test”

Note: Or maybe you want to use some specific filters

• In the end, we can generate and publish reports

Here is an example with GitHub Actions

name: Build and test Herokuapp

on:
  pull_request:
    branches: [ main ]
    paths:
      - '_project-herokuapp-components-tests/**'

jobs:
  build_and_test:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [20.x]

    steps:
    - uses: actions/checkout@v3

    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v3
      with:
        node-version: ${{ matrix.node-version }}

    - name: Install Root Dependencies
      run: npm install

    - name: Run Build
      run: npm run build

    - name: Install Dependencies for Playwright Tests
      run: npx playwright install --with-deps

    - name: Run Playwright Tests
      run: npx playwright test --project="Chromium build-in tests"
      working-directory: _project-herokuapp-components-tests

    - name: Upload Playwright JUnit Test Results
      uses: actions/upload-artifact@v2
      if: always() # ensures that reports are uploaded even if tests fail
      with:
        name: playwright-junit-test-results
        path: _project-herokuapp-components-tests/test-results/results.xml
    
    - name: Publish Unit Test Results
      uses: EnricoMi/publish-unit-test-result-action@v1
      if: always() # ensures that the report is published even if tests fail
      with:
        files: _project-herokuapp-components-tests/test-results/results.xml
        check_name: Test Results # Optional, name of the check

Final words

This is definitely the longest article I have ever written. Honestly I didn’t want to make it so long, but it was important to cover all the basic aspects.

There are other more specific article about playwright available. You may find them more interesting if you need to resolve a concreate problem. This one was for those who just wants to jump into the Playwright train.

Good luck and here is the list of articles:

Thanks for reading!