Modern Web Testing using Cypress and Cucumber
Need of Automation
- Most of the Agile Delivery teams follow a release cycle of 2 weeks. In order to cope up with the changes and to deliver a bug free application, automation is needed. There is a need to identify bugs earlier in development phase itself to increase the team’s efficiency.
- Most end-to-end scenarios which are identified for testing are lengthy, and automated testing helps in writing more in-depth tests to verify complex use cases. This also covers the lengthy tests that are often avoided during manual testing.
- Many new requirements come up within the Sprint and in order to complete the testing and deliver within the timeline with no / less human errors, automation is needed. For most of the projects, there are multiple environments present which serves different purposes, some environments are client specific as well. In order to achieve Continuous Deployment, automation is necessary.
Solution
In order to fulfill the automation need, Cypress framework integrated with Cucumber is one of the best options. Here the requirements are written in the form of scenarios in feature files that follows Behaviour Driven Development (BDD) syntax.
Introduction to Cypress
- Cypress is a free and open source automation tool, MIT-licensed and written in JavaScript.
- Cypress is a next generation front-end testing tool built for the modern web. With the help of Cypress end-to-end test, integration and unit tests are easy to write and debug.
- Cypress consists of a free, open source, locally installed Test Runner and a Dashboard Service for recording our tests. Cypress has its own folder structure, which gets generated automatically when you open it for the very first time at that specific location. It comes with ready-made recipes that depict how to test common scenes in Cypress.
Cypress Architecture
- Most testing tools (like Selenium) operate by running outside of the browser and executing remote commands across the network. But the Cypress engine directly operates inside the browser. It enables Cypress to listen and modify the browser behavior at run time by manipulating DOM and altering network requests and responses on the fly. This in turn makes the test execution faster and more stable. Thus, it opens the door to new kind of testing along with ultimate control over our application.
- Tests can be easily created for UI and API operations. Cypress directly communicates with API and enables the tests to run and verify the request and response at the run time.
- Cypress works in a modern front-end development stack: Mocha, Chai, Sinon, jQuery. It perfectly matches with the tools, which the developers usually use to write unit tests. E2E Testing for web applications developed using React can also be easily achieved using Cypress.
- Behind Cypress is a Node Server process. Cypress and the Node process constantly communicate, synchronize, and perform tasks on behalf of each other. Having access to both parts (front and back) gives us the ability to respond to the application’s events in real time, while at the same time work outside of the browser for tasks that require a higher privilege.
- Cypress also operates at the network layer by reading and altering web traffic on the fly. This enables Cypress to not only modify everything coming in and out of the browser, but also to change code that may interfere with its ability to automate the browser.
- Cypress ultimately controls the entire automation process from top to bottom, which puts it in the unique position of being able to understand everything happening inside and outside of the browser. This means Cypress is capable of delivering more consistent results than any other testing tool.
- Because Cypress is installed locally on our machine, it can additionally tap into the operating system for automation tasks. This makes performing tasks such as taking screenshots, recording videos, general file system operations and network operations possible.
- Cypress Browser Support: Canary, Chrome, Electron (default), Chromium, Mozilla Firefox browsers (beta support) and Microsoft Edge (Chromium-based) browsers.
Cypress Features
- Time Travel: Cypress takes snapshots as tests run. We can hover over commands in the Command Log to see exactly what happened at each step.
- Debuggability: Cypress code can be debugged directly from browser developer tools as the code is written in JavaScript. It also provides readable errors and stack traces that makes debugging lightning fast.
- Automatic Waiting: Cypress automatically waits for commands and assertions before moving on.
- Spies, Stubs, and Clocks: Using Cypress we can verify and control the behavior of functions, server responses, or timers.
- Screenshots and Videos: Screenshots are taken automatically on failure, or videos of the entire test suite when run from the CLI.
- Cross Browser Testing: Runs tests within Firefox and Chrome-family browsers (including Edge and Electron) locally and optimally in a Continuous Integration pipeline.
In addition to these, it has changed the way how the test automation framework development used to happen before its introduction. The below image depicts it very beautifully:
Following are some references for Cypress assertions, inbuilt commands and custom commands which are commonly used across projects :
- https://docs.cypress.io/guides/core-concepts/introduction-to-cypress.html#Cypress-Can-Be-Simple-Sometimes
- https://docs.cypress.io/guides/references/assertions.html#Chai
- https://docs.cypress.io/guides/core-concepts/retry-ability.html#Commands-vs-assertions
- https://docs.cypress.io/api/cypress-api/custom-commands.html#Syntax
- https://docs.cypress.io/api/commands/stub.html#Syntax
Cypress Assertions
- Assertions are the validation steps that determine whether the specified step of the automated test case succeeded or not. In actual, assertions validate the desired state of your elements, objects, or Application Under Test (AUT). E.g. Assertions enable us to validate scenarios such as whether an element is visible or has a particular attribute, CSS class, or state.
- It is always a recommended practice that all the automated test cases should have assertion steps. Otherwise, it will not be feasible to validate whether the application has reached the expected state or not.
- Cypress itself bundles assertions provided by Chai, Sinon, and jQuery libraries.
We can broadly classify all of these assertions into two segments based on the subject on which we can invoke them: Implicit and Explicit Assertions.
Implicit Assertions
- When the assertion applies to the object provided by the parent chained command, its called an Implicit assertion.
- Additionally, this category of assertions generally includes commands such as “.should()” and “.and().”
- As these commands don’t stand independently and always depends on the previously chained parent command, they automatically inherit and acts on the object yielded by the previous command.
- For example :
cy.get('#loginButton').should('have.class', 'active').and('have.attr', 'href', '/login');
As
.should('have.class')does not change the subject,.and('have.attr')is executed against the same element, it is handy to use these Implicit assertions when you need to assert multiple things against a single subject quickly.
Generally, we use Implicit assertions when we want to:
- Assert multiple validations about the same subject.
- Alter the subject before making the assertions on the subject.
Explicit Assertions
- When there is a need to pass an explicit subject for the assertion, it falls under the category of Explicit assertion.
- This category of assertions contains the commands such as “expect()” and “assert(),” which allow you to pass an explicit subject/object.
- For example :
const obj = { foo: 'bar' }
expect(obj).to.equal(obj)
expect(obj).to.deep.equal({ foo: 'bar' })Generally, we use Explicit assertions when we want to:
- Perform some custom logic before making the assertions on the given subject.
- Perform multiple assertions against the same subject.
Common Cypress Assertions
Cypress provides a wide range of assertions which can be very handy during UI automation. Some of the most widely used Cypress assertions are:
1) Length: Validates the number of elements returned by the previously chained command. E.g.: cy.get('.demo-frame > ul > li').should('have.length',20);
2) Class: Validates whether the element does have or doesn’t have the mentioned class. E.g.: cy.get('form').find('input').should('not.have.class', 'disabled');
3) Text Content: Validates element to have a specified text. E.g.: cy.get('a').parent('span').should('contain', 'Testing');
4) Visibility: Validates whether the element is visible or not. E.g.: cy.get('button').should('be.visible');
5) Existence: Validates whether the element exists in the DOM. E.g.: cy.get('#loader').should('not.exist');
Cypress Hooks
Cypress also provides the constructs which are also known as Cypress Hooks which help in performing a particular set of actions just before/after each test case or before/after all the test cases in the test suite.
before(): It runs once before starting the execution of first tests specified using “it” or “specify” in the “describe” or “context” block.after(): It runs once after completion of all the tests specified using “it” or “specify” in the “describe” or “context” block.beforeEach(): It runs before starting the execution of each of the tests specified using “it” or “specify” in the “describe” or “context” block.afterEach(): It runs after finishing the execution of each of the tests specified using “it” or “specify” in the “describe” or “context” block.
Cypress Test Runner
- Cypress runs tests in a unique interactive runner that allows us to see commands as they execute while also viewing the application under test.
- The left hand side of the Test Runner is a visual representation of our test suite. Each test block is properly nested and each test, when clicked, displays every Cypress command and assertion executed within the test’s block as well as any command or assertion executed in relevant before, beforeEach, afterEach, and after hooks.
- Clicking on commands : Each command, assertion, or error, when clicked on, displays extra information in the dev tools console. Clicking also ‘pins’ the Application Under Test (righthand side) to its previous state when the command executed.
- Hovering on commands : Each command and assertion, when hovered over, restores the Application Under Test (right hand side) to the state it was in when that command executed. This allows us to ‘time-travel’ back to previous states of your application when testing.
- The test runner also provides an option to select different browsers from a dropdown. In this way we can perform cross browser testing in our local, provided all the browsers are installed.
Introduction to BDD and Cucumber
- Test Scenarios are developed directly from the user stories. In order to cover the complete user stories and the tests around it, automated tests are written in BDD format i.e. Cucumber steps. These steps are written in the Gherkin language, so they would be, or should be readable by everyone.
- In agile projecs requirements keep on changing frequently with new features getting introduced in the application so BDD fits very well with these changes.
- Given — When — Then approach gives liberty to the testers to use the same steps as many times we want in the feature file which gradually helps in saving time for the automation.
Components of BDD Framework
- Feature File is the entry point to the Cucumber tests of your framework. It is a file where you will write your tests or acceptance criteria in Descriptive Gherkin language (Like English). A feature file can include one or many scenarios that are in the form of Given-When-Then format.
- Step Definition is a small piece of code with a design pattern attached to it. An annotation followed by the pattern that is mentioned in the feature file links the Step Definition to all the matching Steps.
For more details -
Cypress Installation
- Run :
npm init(This will create a package.json file)
Make sure that you have already run npm init or have a node_modules folder or package.json file in the root of your project to ensure cypress is installed in the correct directory.
- Download using npm: Use below command to download Cypress using the node package manager. Moreover, we have to execute this command in the same directory, which means inside the project folder itself that we mentioned while doing npm init above.
- Run :
npm install cypress --save-dev(This will update the package.json and package-lock.json file with the required dependencies and also a node_modules folder will be created) - Run :
npx cypress open(This will launch the Test Runner for the first time and a default folder structure will be generated)
For more details -
https://docs.cypress.io/guides/getting-started/installing-cypress.html
Project Structure
Following is a snapshot of the default folder structure of Cypress framework :
- Fixtures are used as external pieces of static data that can be used by your tests. Fixture files are located in cypress/fixtures by default but can be configured to another directory.
For more details :
https://www.toolsqa.com/cypress/fixtures-in-cypress/
- Feature files & Step Definitions are located in cypress/integration by default but can be configured to another directory.
- By default, Cypress will automatically include the plugins file cypress/plugins/index.js before every single spec file it runs.
- The plugins file is a special file that executes in Node before the project is loaded, before the browser launches, and during your test execution. While the Cypress tests execute in the browser, the plugins file runs in the background Node process, giving your tests the ability to access the file system and the rest of the operating system by calling the cy.task() command.
- Support folder consists of files with utility methods (custom commands) for UI and API related operations. The custom commands can be created to give a different behavior to the existing in-built Cypress command or can be used to create a new command altogether. A custom command in Cypress is nothing more than a regular Cypress command. The only difference is it is defined by the user, as opposed to the default commands that Cypress supplies out of the box. Custom commands are beneficial for automating a workflow you repeat in your tests over and over.
- By default, Cypress will automatically include the support file cypress/support/index.js.
- Unlike Selenium, Cypress only supports CSS selectors( very similar to JQuery Selectors). Cypress does support Xpath selectors as well. However, that does not come by default. In other words, we need cypress-xpath external plugin to assist this selector.
To install cypress-xpath plugin, run the command :
npm install -D cypress-xpathThen we need to add the following in cypress/support/index.js :
require('cypress-xpath')
- Adding own custom commands :
Cypress provides an API for creating a new custom command or overwriting existing commands to change their implementation. The built-in Cypress commands also use the API. It can be represented syntactically as follows:
// Add a new commandCypress.Commands.add(name, callbackFn)
Cypress.Commands.add(name, options, callbackFn)
For more details :
https://www.toolsqa.com/cypress/custom-commands-in-cypress/
package.jsonconsists of all the required dependencies. All npm packages contain a file in the project root, called package.json, which holds various metadata and libraries relevant to the project. This file gives information to npm that allows it to identify the project as well as handle the project’s dependencies. It is similar topom.xmlfrom Maven and build.gradle in Gradle. Assuming that you have already installed a node, we must first create a package.json file.
Configurations in Cypress
- As per the standard definition, the config files keep some initial settings and configurations for the program or the framework. Consequently, following the same pattern, Cypress also provides specific configurations that initially have particular default values, and on a need basis, these values can override by the users.
- Configurations specify some key-values which can be used throughout the test framework and will guide the behavior of the framework based on their default or altered values.
- The global configurations provide all the default configurations which Cypress considers while executing the tests. Additionally, these configurations are majorly related to timeouts, environment variables, node version, the folder path from where it is reading all the integration tests, fixture, commands, etc.
- The first method provided by Cypress to change the values of the default configurations is by specifying the needed values of those configurations in the config files. The default config file used by Cypress is cypress.json. Therefore, whatever values we want to change, we can specify those key-value pairs in the cypress.json file.
- Suppose, we want to update the defaultCommandTimeout configuration and want to change it to 10 secs from its default 4 secs. Consequently, it is achievable by specifying the below values in the cypress.json file:
{
"defaultCommandTimeout": 10000
}Integration of Cypress with Cucumber Preprocessor
The first step in the integration of Cucumber with Cypress is installing the dependency of Cucumber-preprocessor in the framework.
For installing the Cucumber-preprocessor node module for Cypress, we can use the following command:
npm install --save-dev cypress-cucumber-preprocessor
When we execute this command, it will produce the sample output, as shown below :
This module will be added under “devDependencies” in package.json file in the project.
Now, once the Cucumber-preprocessor installation is successful, the next step is to make some configuration changes, so as Cypress can start using the Cucumber and enables users to write BDD tests.
- Add it to the plugins under cypress/plugins/index.js:
const cucumber = require('cypress-cucumber-preprocessor').defaultmodule.exports = (on, config) => {
on('file:preprocessor', cucumber())
}
- Add support for feature files to your Cypress configuration cypress.json :
{
"testFiles": "**/*.feature"
}Some key points to note while writing test cases in BDD format (feature files) :
1) Import statement : We have to import Given-When-Then annotations from cypress-cucumber-preprocessor. The reason being these annotations provide the mapping between steps in feature files and the methods in the step definition files.
2) While writing the step definition for the statements in the Cucumber feature file, the English text in the feature file and the step definition file should be the same. It assists the Regex in identifying which step matches to which method implementation.
3) For data that was in inverted commas in the feature file, we are saving it as a {string}. Then later, we will pass it as a parameter like we have used Username, Email, etc.
For more details :
Reporting with Mochawesome
- Mochawesome is a custom reporter for use with the JavaScript testing framework, Mocha.
- It runs on Node.js (>=10) and works in conjunction with mochawesome-report-generator to generate a standalone HTML/CSS report to help visualize your test runs.
- marge (mochawesome-report-generator) is the counterpart to mochawesome, a custom reporter for use with the JavaScript testing framework, mocha. Marge takes the JSON output from mochawesome and generates a full-fledged HTML/CSS report that helps visualize your test suites.
Installing Mochawesome :
Run:
npm install mochawesome
- In cypress.json config file add the following :
Advantages Of using Mochawesome Report Generator
Here are the significant advantages of Mochawesome in comparison to the other Mocha reporting tools:
- Mochawesome report is designed in a mobile-friendly and straightforward way. It uses chartJS for the visualization of test reports.
- It supports display of different types of hooks —
before(), beforeEach(), after(),afterAll()and so on. - Failure at a particular line of code is directly visible in the report itself due to the inline code review feature in Mochawesome.
- It has support for stack trace logs and failure messages.
- You can add extra context information to tests like console logs, environment details, custom messages to tests as an attachment. This gives the additional hand of information about the test cases in the report.
- Since the reports are generated with standalone HTML/CSS, they can be shared offline with .HTML extension.
- Following is a snapshot of a sample Mochawesome report generated after test execution :
Value Creation and Stability
- When compared to other automation frameworks, Cypress is more universal. That’s because it is written in JavaScript and based on Mocha and Chai. It also uses Node.js when running in browsers.
- JavaScript is the main language used by developers to develop websites. Hence, testing is created in the same language. With Cypress, you can run cross-browser testing.
- Cypress can execute tests in multiple browsers like Chrome, Firefox, Edge and Chromium.
- Cypress provides a response time of less than 20ms. The framework automatically runs tests after completion of another. This eliminates downtime and the need to manually trigger the next test which expedites the whole execution.
- It’s very straightforward to reuse tests for other use cases. The advantage is that we can reuse the same framework for other projects.
- Cypress can be integrated with CI/CD tools like GitLab, Jenkins, CircleCI etc. For more details : https://docs.cypress.io/guides/continuous-integration/introduction#What-is-Continuous-Integration
