Automating Testing with GitHub Actions: Continuous Integration (CI) Part 1
In today’s fast-paced world of software development, ensuring top-notch quality through robust testing practices is absolutely crucial. This is the first article in a series focusing on GitHub Actions from the QA Perspective. Today we will focus on Continuous Integration (CI), the superhero of our development process!
Setting Up CI Workflows — The Basics
Let’s dive into the exciting world of setting up CI workflows with GitHub Actions! In case you are familiar with the basics, please skip this section.
The following file does the following:
- Runs automatically after a push on the main branch.
- Sets up our environment on an Ubuntu latest environment provided by GitHub with prerequisites needed for our tests to run. Here we set Node version 20.
- Installs the project dependencies as we manually do on our laptops by running the Yarn install command.
- Executes testing command that will run our tests and produce a report.
name: CI Auto tests on Main Push
on:
push:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4.1.1
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: '20'
- name: Install dependencies
run: yarn install
- name: Run tests
run: yarn cy:run --spec "cypress/e2e/**/*.cy.js" --reporter mochawesomeFor people who haven’t used GitHub Actions before, keep in mind that all you need to do to add this into your repository and see it in action is to:
#cd to your repo root dir
cd <to>/<your>/<repo>
# create your directory
mkdir .github/workflows/
#create ci-auto-tests-on-push.yaml file in your IDE under this repository
git add ci-auto-tests-on-push.yaml
git commit -m "[test] Adding my ci auto trigger on push main branch"
git pushOnce you do this , you will see the tests running on Actions of your Repo in Github. You can explore more on your account! Simple as that!
Supercharge your .yaml file by applying best practices
And now you will say: “Ok, fine, but we need a super strong CI pipeline and we need it now!” Let’s move towards this direction then by applying best practices in a step-by-step guide.
Run Tests on Multiple Operating Systems
Let’s say we have a B2C Web Application Product being used by millions of users all over the globe. How can you ensure the web app works properly on different operating systems? Do not worry, GitHub has the solution for you. The following code will initiate runs on three different environments in parallel:
jobs:
run-ui-tests:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
Parallelising Tests
Let’s say we have an application with a big amount of end to end tests, that take a lot time to run. The tests are tagged with different features, or they can run on different environments. What should we do about it?
Github allows us to run tests in parallel not only on different operating systems, but with different scope of tests. So we can use different configuration files, one for production and one for staging, in order to run on those in parallel.
parallel_tests:
runs-on: ubuntu-latest
needs: build
strategy:
matrix:
config: [cypress.staging.config.js, cypress.production.config.js]
steps:
- name: Run Tests with different Tags
run: |
yarn cy:run --spec "cypress/e2e/**/*.cy.js" --reporter mochawesome --config-file ${{ matrix.config}}If you want to combine it and run in all operating systems as well, you can do:
name: CI Auto tests on Main Push
on:
push:
branches:
- main
jobs:
# Parallelizing Tests with Different Environments and Operating Systems
parallel_tests:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ ubuntu-latest, windows-latest, macos-latest ]
config: [ cypress.staging.config.js, cypress.production.config.js ] # Add more testing tags as needed
steps:
- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: '20'
- name: Checkout Repository
uses: actions/checkout@v2
- name: Install Dependencies
run: yarn install
- name: Debug
run: |
ls -l /home/runner/work/skedda-tests/skedda-tests
- name: Run Tests on different Environment
run: |
yarn cy:run --spec "cypress/e2e/**/*.cy.js" --reporter mochawesome --config-file ${{ matrix.config }}And 💣 Boom! Now you can see 6 jobs, 3 per each operating system and 2 per each different configuration file.
Keep in mind that you can do the same with different environment variables, or different tags, so that you can run subsets of test cases in parallel and reduce your CI execution time!
Create dependencies between the steps
Oh well, adding parallelisation made our run fail. This is because we may need to add dependencies in the steps, to ensure that in concurrent steps, we run step 2 after step 1 execution has finished successfully. Another thing we will add is some timeouts in the steps to ensure they won’t take too long.
Let’s try it out:
name: CI Auto tests on Main Push
on:
push:
branches:
- main
jobs:
parallel_tests:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ ubuntu-latest, windows-latest, macos-latest ]
config: [ cypress.staging.config.js, cypress.production.config.js ] # Add more testing tags as needed
steps:
- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: '20'
- name: Checkout Repository
uses: actions/checkout@v2
- name: dependencies
run: yarn install
timeout-minutes: 5 # Add a timeout of 5 minutes for installing dependencies
- name: debug
run: |
ls -l /home/runner/work/skedda-tests/skedda-tests
timeout-minutes: 2 # Add a timeout of 2 minutes for debugging
if: steps.dependencies.outcome == 'success' #The step will run if the dependencies have run successful
- name: Run Tests on different Environment
run: |
yarn cy:run --spec "cypress/e2e/**/*.cy.js" --reporter mochawesome --config-file ${{ matrix.config }}
timeout-minutes: 10 # Add a timeout of 10 minutes for running tests
if: steps.debug.outcome == 'success' #The step will run if the dependencies have run successfulNow our tests pass! Keep also in mind that if you go deeper in the job structure from Github Actions UI, you will see the exact steps that run:
Depend on other Workflows
Let’s assume now that we have a Github workflow that makes sense to run only if another Github Workflow was successful. One example of this use case would be end to end tests to run only after integration tests have passed. The scenario we will explore here is that we have a deploy-env.yaml file that deploys our web app and we want to depend the tests on the successful execution of that workflow. Here is our file:
name: CI Auto tests on Main Push
on:
push:
branches:
- main
workflow_run:
workflows: ["Deploy Environment"] # Specify the name of the other workflow
types:
- completed
jobs:
parallel_tests:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ ubuntu-latest, windows-latest, macos-latest ]
config: [ cypress.staging.config.js, cypress.production.config.js ] # Add more testing tags as needed
steps:
- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: '20'
- name: Checkout Repository
uses: actions/checkout@v2
- name: Install Dependencies
id: dependencies
run: yarn install
timeout-minutes: 5
- name: Debug
if: steps.dependencies.outcome == 'success'
run: |
ls -l /home/runner/work/skedda-tests/skedda-tests
timeout-minutes: 2
- name: Run Tests on different Environment
run: |
yarn cy:run --spec "cypress/e2e/**/*.cy.js" --reporter mochawesome --config-file ${{ matrix.config }}
timeout-minutes: 10
if: steps.debug.outcome == 'success'In the case that “Deploy Environment” workflow fails, our workflow should not run at all. We don’t want to waste time but we want to fail early. So we can extra add the following line of code:
jobs:
parallel_tests:
if: ${{ github.event.workflow_run.conclusion != 'failure' }} # Run only if the completed job is not a failed job
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ ubuntu-latest, windows-latest, macos-latest ]
config: [ cypress.staging.config.js, cypress.production.config.js ]
steps:
Lessons Learnt
We have seen different techniques that are really useful for a QA engineer, so that you can create a CI pipeline for your tests and do amazing things to level up the Quality in your Product. We will appreciate your 👏 if you did enjoy reading! You can always explore more on the Official Github Documentation.
Stay tuned for Automating Testing with GitHub Actions : Continuous Integration (CI) Part 2 🎁
