3 and 1/2 Reasons Why Your Tests Should Be Stateless

Save time and reduce headaches with self-encapsulated tests

Ā·

5 min read

Featured on Hashnode
3 and 1/2 Reasons Why Your Tests Should Be Stateless

This lego person lives in a bubble and your tests should too! (šŸ“·: Sufian Hanif)

Definition

ā€œA stateless process or application can be understood in isolation. There is no stored knowledge of or reference to past transactions. Each transaction is made as if from scratch for the first timeā€¦ā€ ā€”ā€ŠRed Hat (source)

Living in a Bubble šŸ«§

Debugging obscure test failures is about as fun as hiking Mt. Kilimanjaro with a rock in your shoe. If youā€™ve ever written a unit, integration, or end-to-end (E2E) test you have likely experienced extreme boredom stepping through incalculable lines of test code, chasing your tail trying to hunt down unknown errors. Countless hours of frustration can be saved by writing self-encapsulated, self-configuring tests instead of relying on applications to magically exist in the desired state.

1. Standalone Tests Are Easier To Debug

The problem is in the test. You donā€™t have to snoop around other tests, your applicationā€™s test user account, or check your database to ensure the correct information exists and is available for testing. Knowing exactly where to look for the solution to your test failure greatly accelerates the rate at which you are able to resolve test issues.

2. Self-Encapsulated Tests Can Be Run in Parallel

Writing tests that can only operate in a specific order is a serious anti-pattern. For longer running tests such as Integration and E2E tests, it becomes particularly important to run tests in parallel to reduce test suite runtimes. Additionally, itā€™s vitally important that you have the ability to launch long-running tests independently of the test suite.

3. Stateless Tests Can Be Run in Multiple Environments

Relying on the preexisting application state in your tests means relying on undocumented memories of the testā€™s author. Tests that rely on the application being configured a specific way prior to the start of the test suite often canā€™t run in multiple environments such as on a CI/CD machine or in a new hireā€™s IDE. Itā€™s a tedious endeavor into drudgery setting up new environments for running tests manuallyā€Šā€”ā€Šdonā€™t do it!

Bonus: Stateless Tests Double As Documentation

Well-written tests are the best form of technical documentation. Tests describe actions that can be performed on a system, as well as the conditions under which those actions can be performed. When a test includes all the necessary set-up itā€™s significantly more transparent to the observer how the system is supposed to work.

The Problem

In this scenario, a developer on our team has written a suite comprised of two Cypress E2E tests. The first test adds a user to a table containing a list of users. The second test filters the table and ensures that a user is displayed in the list after the filter has been applied.

The developer writes the test suite as follows:

Notice that the second test is filtering based on a user that was added by the first test. The second test canā€˜t be executed independently or in parallel. A failure in the first test might trigger a false positive failure in the second test even if the functionality in the second test is working properly.

The Remedy

Writing tests that donā€™t contain dependencies on the external state is often straightforward. There are a variety of options to improve the sample test above. The easiest solution is to simply move the call to cy.addUser() to the beforeEach block, as you can see below:

There are a few drawbacks to the solution above. The first issue is that weā€™ve introduced duplication in our tests and both tests written above are testing the cy.addUser(user) command. The second issue is that setting up tests using user actions is slow and should be avoided unless you are specifically testing the user action.

To speed up our test and reduce duplication we can re-write the test to use an API client to configure the state of the system directly. You can define a Cypress task cy.task('add:user', user) and leverage it to add the user at test time via an API call, shown below:

The second solution is closer to optimal, but do we need to add the user via an API to filter the table? Adding the user via a task is still prone to failures, and isnā€™t necessary to test filtering the table. In our hypothetical application, the usersā€™ table is filtered on the client and we can improve the test by mocking the state of the system via a network intercept. Hereā€™s the code:

Final Thoughts

Like most things, the topics described above should be taken with a grain of salt. In an ideal scenario, all tests would be able to run completely standalone and have no dependencies on their environment being configured in a specific manner. However, the real world is messy and sometimes it isnā€™t feasible to make every test responsible for the entirety of its state.

In an example scenario, it might be undesirable to create a new user every time you run a test. A reasonable alternative might be to create a new user at the beginning of the entire test suite or use a user that already exists in the system. In such a scenario, your tests should handle the necessary configuration of the account and not depend on any existing configuration to be present at the beginning of the test run.

Thanks for reading!

Want to connect?

If you found the information in this tutorial valuable, follow me on Twitter.

Ā