The more our project and test suite grow, the longer the feedback loop becomes. Fortunately, there are techniques available to speed up our build time. One of such techniques is parallelizing our tests. Instead of running our tests in sequence, we can run them in parallel to save time. The parallelization may not work for all kinds of tests, and hence we'll learn with this article how to only parallelize our unit tests with JUnit 5 and Maven. The upcoming technique is framework independent, and we can apply it to any Java project (Spring Boot, Quarkus, Micronaut, Jakarta EE, etc.) that uses JUnit 5 (JUnit Jupiter, to be precise) and Maven.
Upfront Requirement: Separation of Tests
Before we jump right into the required configuration setup for parallelizing our unit tests, we first have to split our tests into at least two categories.
The reason for this is to allow a separate parallelization strategy (or no parallelization at all) depending on the test category.
While there are many different types of tests in the literature, one can endlessly discuss what's the correct name and category. Sticking to two basic test categories is usually sufficient: unit and integration tests.
Where to draw the line between unit and integration tests is yet another discussion. In general, if a test meets the following criteria, we can usually refer to it as a unit test:
- A small unit (method or class) is tested in isolation
- The collaborators of the class under test are replaced with a fake/stub
- The test doesn't depend on infrastructure components like a database
- The test executes fast
- We can parallelize the test as there are no side effects from other tests
While this list of requirements is not exhaustive, it's a first good indicator of what a unit test is. Any test that doesn't fit in this category will be labeled an integration test.
When it comes to labeling and separating our tests, we have multiple options when using JUnit and Maven.
First, JUnit 5 lets us tag @Tag("integration-test") (former JUnit 4 categories) our test class to label them. However, when adding a new test to our project, we may forget to add these tags, and in general, it requires a little bit more maintenance effort on our end.
A more pragmatic approach is to use the convenient defaults of two Maven plugins that are involved in our testing lifecycle: the Maven Surefire and Maven Failsafe Plugin.
By default, the Maven Surefire plugin will run any test that has the postfix *Test (e.g., CustomerServiceTest). The Maven Failsafe Plugin, on the other hand, only executes tests with the postfix *IT (for integration test). We can even override these naming strategies and come up with our own postfix.
Sticking to the defaults, if we add the postfix *Test only to our unit tests classes and *IT for our integration tests, we already have a separation.
Both plugins run separately at a different build phase of the Maven default lifecycle. The Maven Surefire plugin executes our unit tests in the test phase while the Failsafe plugin gets active in the verify phase. For more information on both plugins and to understand how Maven is involved in testing Java applications, head over to this article.
Upfront Requirement: Independent Unit Tests
Another requirement we have to conform to is the independence of our unit tests. As soon as we've split up our test suite into unit and integration tests by naming them differently, we have to ensure our unit test can run in parallel.
For this to work, there shouldn't be an implicit order that dictates the success or failure of our unit tests. Our unit tests should pass or fail independently of the order they were invoked.
As our unit tests ran in sequence before, we may not have noticed any violation of this requirment. It's very likely that some tests fail this requirement, especially the longer our project exists.
The parallelization acts as a litmus test for the independence of our unit test. If we see random test failures, we know there's something for us to work on before we can fully benefit from the test parallelization.
As the independence of our tests is a best practice, it's we should revisit any test that fails to meet this requirement. Even if we decide not to parallelize them, fixing these tests is still worth the effort as they may fail randomly in the future. This makes our build less deterministic and results in frustrated developers that try to fix a critical bug while working under pressure.
Java JUnit 5 Test Example
For the upcoming unit test parallelization example with JUnit 5 and Maven, we're using the following sample unit test:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | class StringFormatterTest { private StringFormatter cut = new StringFormatter(); @BeforeEach void artificialDelay() throws Exception { // delaying the test execution to see the parallelization efforts Thread.sleep(1000); } @Test void shouldUppercaseLowercaseString() { String input = "duke"; String result = cut.format(input); assertEquals("DUKE", result); } // two more similar tests } |
The actual implementation that is being tested by this unit test is secondary. Our StringFormatterTest test class contains three unit tests that we artificially slow down with a Thread.sleep(). This will help us see an actual difference once we enable the parallelization of our unit tests.
Running the three tests of this class in sequence takes three seconds:
| 1 2 3 4 5 6 7 8 9 | [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running de.rieckpil.blog.StringFormatterTest [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.056 s - in de.rieckpil.blog.StringFormatterTest [INFO] [INFO] Results: [INFO] [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 |
This is our benchmark to compare the result of running the tests in parallel.
Next to this unit test, we have the following sample integration test. The test is a copy of the unit test with the postfix IT:
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | class StringFormatterIT { private StringFormatter cut = new StringFormatter(); @BeforeEach void artificialDelay() throws Exception { // delaying the test execution to see the parallelization efforts Thread.sleep(1000); } @Test void shouldUppercaseLowercaseString() { String input = "duke"; String result = cut.format(input); assertEquals("DUKE", result); } // two more similar tests } |
While this is not a real integration test, it acts as a placeholder test. It helps us see that our upcoming configuration change only takes effect for the unit tests.
Running all tests for this sample project takes six seconds as all tests are executed in sequence.
Parallelize Java Unit Tests with JUnit 5 and Maven
Now it's time to parallelize our Java unit tests with JUnit 5 and Maven.
As JUnit runs our tests in sequence by default, we have to override this configuration only for our unit tests. A global JUnit 5 config file to define the parallelization won't do the job as this would also trigger parallelization for our integration tests.
When using Maven and the Maven Surefire Plugin, we can add custom configurations (environment variables, system properties, etc.) specifically for the tests that are executed by the Surefire plugin. We can use this technique to pass the relevant JUnit 5 configuration parameters to start parallelizing our unit tests:
| 1 2 3 4 5 6 7 8 9 10 11 12 | <plugin> <artifactId>maven-surefire-plugin</artifactId> <version>3.0.0-M7</version> <configuration> <properties> <configurationParameters> junit.jupiter.execution.parallel.enabled = true junit.jupiter.execution.parallel.mode.default = concurrent </configurationParameters> </properties> </configuration> </plugin> |
Using the configuration above to run all tests concurrently, we get the following result after running our unit tests with ./mvnw test:
| 1 2 3 4 5 6 7 8 9 | [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running de.rieckpil.blog.StringFormatterTest [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.031 s - in de.rieckpil.blog.StringFormatterTest [INFO] [INFO] Results: [INFO] [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 |
The total test execution time (see the time elapsed) went down from three seconds to one second as now all three tests run in parallel. While this time improvement may seem negligible in this example, imagine the same 300% speed improvement for a bigger project.
As we don't override the parallelization strategy, JUnit will fall back to the default dynamic parallization and parallelize by the number of available processors/cores. Per default, JUnit uses one thread per core.
Hence, if we run the tests on a machine with only two cores, the build time will be two seconds as only two tests can run in parallel. That's why we might see build time differences when comparing our local build time with our build agent (e.g., GitHub Actions, Jenkins).
We can override the degree to which parallelize by using either a custom, fixed, or dynamic strategy (factor x available cores):
| 1 2 3 4 5 6 7 8 | <properties> <configurationParameters> junit.jupiter.execution.parallel.enabled = true junit.jupiter.execution.parallel.mode.default = concurrent junit.jupiter.execution.parallel.config.strategy = fixed junit.jupiter.execution.parallel.config.fixed.parallelism = 2 </configurationParameters> </properties> |
Overriding the default parallelism configuration with the fixed configuration above, JUnit 5 will now use two threads to run our tests. This results in an overall test execution time of 2 seconds as we have three tests to execute.
For more configuration options for the test parallelization, head over to the JUnit 5 documentation.
Running Our Java Integration Tests In Sequence
The previous Maven Surefire Plugin configuration only propagates the JUnit 5 configuration for the unit tests.
Hence for the Maven Failsafe Plugin that executes our integration tests, we can run the tests in sequence by not specifying any parallelism config:
| 1 2 3 4 5 6 7 8 9 10 11 12 | <plugin> <artifactId>maven-failsafe-plugin</artifactId> <version>3.0.0-M7</version> <executions> <execution> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> </execution> </executions> </plugin> |
The configurationParameters from the Surefire Plugin are not shared, and hence we can isolate the configuration for both test executions.
If our integration test suite allows for parallel test execution, we can even configure different parallelism and concurrent execution. We may want to only parallelize on an integration test class level and run the test methods in sequence. This can be achieved with the following configuration:
| 1 2 3 | junit.jupiter.execution.parallel.enabled = true junit.jupiter.execution.parallel.mode.default = same_thread junit.jupiter.execution.parallel.mode.classes.default = concurrent |
As soon as both our unit and integration tests share the same parallelism config, we may rather prefer a single junit-platform.properties file to avoid the duplicated config for two Maven plugins. This file has to be at the root of our classpath, and hence we can store it within src/test/resources.
Conclusion of Parallelizing Java Unit Tests with JUnit 5 and Maven
Parallelizing our Java unit tests with JUnit 5 and Maven is a simple technique to speed up our Maven build.
The parallelization feature of JUnit 5 allows for a fine-grained parallelization configuration. By separating our tests into two categories and by using two different Maven plugins, we can isolate the parallelization setup.
Depending on how many existing unit tests we already have, parallelizing them may take some initial setup effort. Not all tests may have been written to be run in parallel in any order. Fixing them is worth the effort.
A sample JUnit 5 Maven project with this parallelization setup is available on GitHub.
Joyful testing,
Philip
