SoftAssertion in Cypress
Enhance Test Robustness with Soft Assertions in Cypress.
Problem Statement:
When writing automated tests in Cypress, it’s common to encounter situations where multiple assertions must be made within single or multiple tests. However, if one assertion fails, the test will immediately stop and the subsequent assertions will not be executed. This makes it difficult to identify all the issues within a test, as only the first failure will be reported. For example, consider the following Cypress test:
describe("Scenario-1 with native assertions", () => {
it('should validate multiple assertions using native assertions', () => {
cy.visit('https://www.saucedemo.com/');
cy.get('#user-name').type('standard_user');
cy.get('#password').type('secret_sauce');
cy.get('#login-button').click();
cy.get('#react-burger-menu-btn').click();
//error assertion code
cy.get('a#inventory_sidebar_link').should('have.text', 'All Item'); //it should be "All Items"
cy.get('button#react-burger-cross-btn').click();
cy.get('button#add-to-cart-sauce-labs-backpack').should('have.text', 'Add to cart');
cy.get('.app_logo').should('contain', 'Swag Labs');
cy.get('div#inventory_container').last().should('have.class', 'inventory_container');
cy.get('.inventory_item').should('have.length', 6);
cy.get('img[alt="Sauce Labs Backpack"]').should('have.attr', 'src', '/static/media/sauce-backpack-1200x1500.0a0b85a3.jpg');
});
});In the above example, I have implemented multiple assertions. If the first assertion fails, the subsequent assertions will not be executed, and only the error message for the first assertion will be reported. This makes it challenging to identify all the issues within the test, as the test will stop executing after the first failure.
Output:
Solution Statement:
To overcome this problem, I have added a softAssert & assertAll custom command, which allows multiple assertions to be executed within a single test or multiple tests without stopping the test on the first failure. The softAssert command can be chained to the Cypress object, followed by the assertion method. For example:
// should('have.text')
cy.get('a#inventory_sidebar_link').then($el => {
cy.softAssert($el.text(), 'All Items', 'The text should be "All Items"');
});If any of the assertions fail, the test will continue to execute, and all the failures will be reported at the end of the test along with their corresponding error messages. For example, consider the following Cypress test that uses the softAssert & assertAll custom command:
describe("Scenario-2 with SoftAssertions", () => {
after("This will store all the errors for all the describe blocks", function () {
cy.assertAll(); // This we need to add for throwing the errors
});
it('should validate multiple assertions using softAssert', () => {
cy.visit('https://www.saucedemo.com/');
cy.get('#user-name').type('standard_user');
cy.get('#password').type('secret_sauce');
cy.get('#login-button').click();
cy.get('#react-burger-menu-btn').click();
cy.get('a#inventory_sidebar_link').then($el => {
cy.softAssert($el.text(), 'All Items', 'The text should be "All Items"');
});
cy.get('button#react-burger-cross-btn').click();
cy.get('button#add-to-cart-sauce-labs-backpack').then($el => {
cy.softAssert($el.text(), 'Add to cart', 'The text should be "Add to cart"');
});
// should('contain')
cy.get('.app_logo').then($el => {
cy.softAssert($el.text().includes('Swag Labs'), true, 'The element should contain Swag Labs');
});
// should('have.class')
cy.get('div#inventory_container').then($el => {
cy.softAssert($el.hasClass('inventory_container'), true, 'The element should have the class "inventory_container"');
});
// should('not.have.class')
cy.get('div#inventory_container').then($el => {
cy.softAssert(!$el.hasClass('inventory'), true, 'The element should not have the class "inventory"');
});
// should('have.length')
cy.get('.inventory_item').then($el => {
cy.softAssert($el.length, 6, 'The length should be 6');
});
// should('have.attr')
cy.get('img[alt="Sauce Labs Backpack"]').then($el => {
cy.softAssert($el.attr('src'), '/static/media/sauce-backpack-1200x1500.0a0b85a3.jpg', 'attribute should be present');
});
});
});
In the above example, the softAssert method is used instead of the should method, which allows multiple assertions to be executed without stopping the test on the first failure. If any of the assertions fail, the test will continue to execute, and all the failures will be reported at the end of the test along with their corresponding error messages.
Enable softAssert
To enable softAssert in Cypress, you need to add the below 2 custom commands in the commands.ts file.
let itBlockErrors = {}; // Define an object to store errors for each 'it' block title globally
let totalFailedAssertionsByDescribe = {}; // Keep track of the total number of assertion failures for each "describe" block
Cypress.Commands.add('softAssert', { prevSubject: false }, (actualValue, expectedValue, message) => {
return cy.wrap(null, { timeout: Cypress.config('defaultCommandTimeout') }).then(() => {
try {
expect(actualValue).to.equal(expectedValue, message);
} catch (err) {
const itBlockTitle = Cypress.currentTest.title;
const describeBlockTitle = Cypress.currentTest.titlePath[0];
// Initialize the count for the "describe" block if it doesn't exist
totalFailedAssertionsByDescribe[describeBlockTitle] = totalFailedAssertionsByDescribe[describeBlockTitle] || 0;
totalFailedAssertionsByDescribe[describeBlockTitle]++;
if (!itBlockErrors[itBlockTitle]) {
itBlockErrors[itBlockTitle] = [];
}
itBlockErrors[itBlockTitle].push({ message, error: err });
}
});
});
Cypress.Commands.add('assertAll', () => {
const errors = itBlockErrors;
itBlockErrors = {};
if (Object.keys(errors).length > 0) {
const errorMessages = Object.entries(errors).map(([title, entries], index) => {
const errorMessage = (entries as Array<{ error: Error }>).map(({ error }) => (
`${"=> "+error.message}`
)).join('\n\n');
return `${index + 1}. Test Title: ${title}\n${errorMessage}`;
});
const errorMessage = Object.entries(totalFailedAssertionsByDescribe).map(([describe, count]) => {
return `Total assertion failures in "${describe}": ${count}`;
}).join('\n');
throw new Error(`Soft assertion failed: Total it block failed (${Object.keys(errors).length})\n${errorMessages.join('\n')}\n\n${errorMessage}`);
}
});Add the chainable interface under index.d.ts file.
declare namespace Cypress {
interface Chainable<Subject> {
softAssert(actualValue: any, expectedValue: any, message: string): Chainable<any>;
assertAll(): Chainable<any>;
}
}Multiple softAssertion errors
We can implement this feature with multiple it blocks as well as multiple describe blocks in the same spec file or in different spec files.
Please check out the link for Some Commands Example.
Conclusion:
In conclusion, the softAssert method in Cypress allows multiple assertions to be executed within a single test without stopping the test on the first failure. This makes it easier to identify all the issues within a test, as all the failures will be reported at the end of the test along with their corresponding error messages.
By using the softAssert method, you can write more robust tests that are less likely to fail due to small issues. This can help you catch bugs early on in the development process and reduce the time it takes to identify and fix issues.
Limitations
- This code will only work till Cypress version 10.3.0 because of some new features introduced by Cypress.
- Last it block only becomes red even if any previous it block is having any error.
- We cannot chain anything after using the softAssertion command.
Please let me know in the comment section any new ideas and implementation of this for the newer version of Cypress.
