Xcode Parallel Test Execution of XCUITest is a Joke
How to run 100s of XCUITests under 15 min?
By Anton Prokuda, Sr. iOS Developer
Running XCTest on Apple platforms is a fundamental aspect of app development, ensuring that your code functions as expected across a variety of scenarios. However, as your test suite grows, executing tests in a timely manner becomes increasingly challenging. Apple’s native parallel test runner in Xcode may leave you feeling frustrated due to its limitations, especially when you have more than a thousand XCUItests to run. But fear not, we have a game-changing solution that will drastically improve your testing efficiency: Sift.
The Agonizing Wait
If you’re like me, you know the pain of waiting for your XCUItests suite to complete its marathon run. Picture this: you have over a thousand XCUItests, and running them all using Xcode’s native parallel runner feels like watching paint dry. What should ideally take a couple of hours ends up taking a staggering 38 hours! It’s frustrating, right?
Let’s face it, slow test runs can be a real buzzkill for qa and developers. Waiting for tests to finish is not just frustrating; it’s a colossal waste of time and productivity. While XCUITests and Xcode have served us well, the built-in parallelization options sometimes fall short, leaving us twiddling our thumbs while we wait for the test suite to complete.
The Game-Changing Open-Source Solution: Sift
But here’s the good news — I’ve built a game-changing solution that will make your testing process a breeze: Sift. The best part? It’s completely FREE and open-sourced! In this article, I’ll walk you through setting up Sift step by step, complete with copy-and-paste commands, to supercharge your testing process. Say goodbye to slow test runs and hello to efficient parallelization!
What Is Sift?
Before we dive into the nitty-gritty of setting up Sift, let’s understand what makes it a must-have tool for iOS and macOS developers. Sift is a revolutionary, open-source tool designed to maximize the efficiency of XCTest runs on Apple platforms. It offers a seamless way to parallelize your XCTest suites, significantly reducing the time it takes to complete your tests.
Benefits of Using Sift
Lightning-Fast Parallelization: Sift dramatically accelerates your XCTest suite by intelligently distributing test cases across multiple simulators and devices, enabling parallel execution.
Improved Productivity: With Sift, you’ll never have to waste time waiting for lengthy test runs. Get back to coding and building your app with confidence.
Detailed Reporting: Sift provides detailed test execution reports, making it easy to identify and resolve issues quickly.
Easy Integration: Setting up Sift is a breeze, and we’ll walk you through the process step by step.
Open Source and Free: Sift is a gift to the development community, available at no cost. Contribute, improve, and benefit from a thriving community of users.
Setting Up Sift: A Step-By-Step Guide
Now that you’re excited about what Sift can do for your XCTest suite let’s get down to business and set it up. Follow these steps to supercharge your testing process:
1. Clone the Repository
Before we dive into Sift, let’s grab its code from GitHub. Open your terminal and use this command:
git clone https://github.com/engenious-inc/sift-iosThis will fetch the Sift repository, and you’ll be ready for the next step.
2. Building Sift
Now that you’ve cloned the repository, navigate to the directory where Sift is located:
cd sift-iosWith Sift’s directory as your current location, build the code using this command:
swift build -c releaseThis will compile Sift and generate a release build at the specified path: .build/arm64-apple-macosx/release/Sift or .build/x86_64-apple-macosx/release/Sift
cd .build/arm64-apple-macosx/release or cd .build/x86_64-apple-macosx/release/3. Setting Up Sift
With Sift built, it’s time to configure it. Run the following command to kick off the setup process:
./sift setupSift’s setup command will guide you through essential configurations. Here’s what you’ll need to provide:
.xctestrun File Path: This file is generated by Xcode when you build for testing. Trigger a test build in your project by pressing Command+Shift+U. The .xctestrun file can typically be found in the DerivedData directory, like this:
/Users/YOUR_USERNAME/Library/Developer/Xcode/DerivedData/MyTestProject-…/Build/Products/MyTestProject_UITest_iphonesimulator… .xctestrun
Test Results Directory: Specify where you want test results to be aggregated and stored.
Retry Mechanism: Configure how Sift handles test failures by specifying the number of retries.
Tests Bucket: Define how many tests should be bundled together for execution.
Test Execution Timeout: Set the maximum duration allowed for a batch of tests to run.
Execution Node Configuration: Specify details about the machines on which tests will be executed.
Device/Simulator UDID Configuration: Specify which devices or simulators on the execution node should run the tests.
Config Storage: Sift will generate a configuration file; specify where you want to store it or just hit Enter to save it in the same directory with Sift.
*If you provide incorrect data or want to change some options, you can open the config.json and manually make any necessary changes.
4. Running Tests with Sift
With the configuration done, Sift setup will provide you with the exact command to run it. It will look something like this:
/Users/USERNAME/sift/Sift run — config “/Users/USERNAME/sift/config.json”
You can use this command to execute your tests with Sift.
Sift in action:
In conclusion, by following this guide and harnessing the power of Sift, you’ll drastically improve your XCTest execution efficiency on Apple platforms. No more waiting around for hours. It’s time to embrace rapid and scalable testing. Happy coding!
Contact me: Discord <Anton Prokuda>
