Xcode Cloud: Feedback from leboncoin
By Pierre Abi-aad (Lead Engineer iOS)
The leboncoin iOS community is made up of 28 developers working on a modular technical stack. The main functionality areas are organised in repositories. Each of them contains an Xcode project which will generate a framework.
Our CI/CD is completely in-house, with 20 Mac minis on the hardware side and Zuul / Fastlane for the software.
Upgrades and patch requirements are dealt with on an ongoing basis by a number of infrastructure developers.
- iOS updates
- Xcode updates
- macOS updates
- Unit testing / Integration testing that may fail
- Maintenance outage
This architecture supports two objectives:
- To help developers work as easily and as quickly as possible.
- To be able to deliver an App Store ready application four times a day for POs & QAs to test.
When we heard about Xcode Cloud, we were over the moon. The prospect of complete hardware management and job scheduling by Apple is very good news.
Being able to launch our different builds/testing very quickly on multiple versions of Xcode is an obvious advantage.
So we had to try out this new service.
We got access to the beta version belatedly, in November 2021, but once we got the golden ticket, what were we to do with it?
As already mentioned, our architecture is quite complex (another article may come back to this in detail) and it is difficult for us to devote all our resources to a beta version of a service that may not meet our requirements right away. So we had to find another way to explore the terrain.
A few enthusiastic iOS developers had been meeting for a few weeks via BBLs to build a tooling application for our community on macOS: this was the perfect playground!
A project from scratch, where we could decide on the architecture we wanted to set up without being bogged down by refactoring.
Plus, on the CI/CD side, the project needed to not fit into a roadmap that would have monopolised the infrastructure.
Can you see where this is going?
Xcode Cloud appeared to be the ideal solution to start automating certain tasks on our project.
The project
Application
The application is split into two panels:
- In the left one we display the entry points to our different features
- If an entry point is selected, its function is displayed in the right-hand panel.
Architecture
Unlike the leboncoin iOS stack, we went with a single repository to avoid a complex chain of dependencies that would be overkill for a project like this.
The breakdown is well defined, however: the main application is almost empty shell that just contains all the functions. These are split into Swift packages, which are then imported.
A Common package is used to house code that can be reused in other packages:
- System Design
- Extensions
- Repositories
- Use cases
The code is hosted on our corporate GitHub and we use Trunk Based Development.
Finally, participants must submit pull requests to get validation from a peer and also from the CI.
This is the aspect that we are interested in.
How does Xcode Cloud work
You can configure your project and manage your CI / CD directly within Xcode. The documentation for Xcode Cloud is comprehensive and very clear: this is an excellent point.
Workflows
Workflows are the entry point for everything you want to automate.
Each workflow can be configured to a fairly detailed level, and the interface is quite well thought out.
- What input launches it?
- Creation of a pull request.
- Creation / update of a branch.
- Creation of a tag.
- Recurrent scheduling.
- What internal actions are carried out:
- The app build.
- Performance of unit tests.
- Performance of a build analysis.
- Archive
- What end of workflow actions are performed:
- TestFlight testing.
- Notification of success / failure by email and/or Slack.
An important thing to bear in mind: each action you decide to perform will trigger the (parallel) execution of the entire pipeline (clone / scripts / xcodebuild …).
Scripts
Between certain key steps (post clone pre build post build ) we can run bash scripts to trigger specific actions.
This is the ideal place for:
- Installing external dependencies (brew package for the linter for example).
- Notifying your different discussion channels in the event of a problem.
- Manipulate your project.
Apple gives access to several environment variables for the current workflow. These variables are very useful if you want to apply different scripts depending on the workflow.
There are some downsides, however:
- External dependencies will be installed for each new pipeline. This results in a significant increase in throughput times.
- A
post save artifactswould be a real plus for handling test results/logs/deliverables.
Managing Environment Variables
In addition to those already provided by Apple, you can define your own environment variables for each of your workflows.
Icing on the cake: you can specify whether their value should be secret or not.
For our application, we use this to inject the API key from an external SDK so that it is never added to our code base. Once the project clone has been completed, we modify our Info.plist to give it the key’s value.
Many other uses can be helpful:
- Debug mode
- Environment URLs
- Slack channel ID
- Credentials for external connections
Which workflows, and when?
The machine resources allocated to a workflow can quickly lead to several consequences:
- Additional costs: for the moment the service is in beta and free but what about in future?
- Waiting time for developers, which causes frustration
The different scenarii need to be broken down as small as possible and only run when necessary.
Unit Tests
The breakdown into Swift packages naturally implies individual responsibility, and means unit teting can be carried out intelligently.
For each pull request submitted on GitHub we only launch test workflows realted to the modified packages. For each execution condition (except scheduling) you can include conditions for changes to files / folders.
If neither the shell application classes nor our xcworkspace/xcodeproj or test folders are modified, we do not need to launch a workflow on them.
Finally, all the tests are only run during our nightlies.
Sanity Checks
As CI validation of the pull requests is only conditional on unit tests taking place in distinct packages, it is quite possible that modifying a public interface could crash the compilation of the whole project.
To overcome this problem but avoid overloading PR validation time, we opted for a more pragmatic solution: where the PRs merge on our main branch we launch a workflow that executes a build and all unit tests.
We also created a similar workflow but it runs at midnight every day. What’s the advantage in that compared to the previous workflow? None at the moment, but several avenues can be explored:
- UI test runs.
- Running tests on different Xcode / macOS versions.
Release to Developers
For this part we did not use TestFlight at all, even though it is included in the solution. We need to activate sandbox mode, which Apple naturally does not allow for builds at their destination
The publication of a new release is done without changing the (marketing and technical) build number in a specific commit. When we are ready, we tag a specific commit.
The tag will be named with the release number (x.y.z). Once the tag is created, a workflow starts automatically to archive the application.
The post clone script modifies the build numbers:
- Name of the tag for the marketing version
- Raw date for the technical version
Areas for improvement
We have identified a few points that could improve the experience (if you have information that we may have missed, please feel free to pass it on to us):
- Being able to declare the different workflows elsewhere.
- We can do this on the AppStoreConnect web interface but it stays the same as for Xcode.
- Being able to write everything in yaml files and provide them as entry points to a project would make workflow management easier.
- These files could be versioned by us, which would very useful if a problem occurs.
- Having a base template that we could then define.
- For Apple to pre-install the basic external dependencies, perhaps with granular management of the different versions (swiftlint, carthage, etc …) to avoid the install waiting times with each action.
- More script steps.
- Extend their environment variables:
- Artifact URLs
- Coverage code (useful for blocking PRs)
- We had some downtime or internal errors:
- Add a retry mechanism on dependency resolutions that can sometimes fail.
- Have more alerts on the state of health of the services from Apple.
Our Opinion
Considering this beta version is only 6 months old, Apple has really delivered a pretty advanced product that works almost all the time.
The leboncoin application, which is architecturally complex, would need an enormous amount of computing time under Xcode Cloud. This solution is therefore not for us as yet.
This beta version is mainly aimed at independent developers or startups. Since these users do not necessarily have the resources or the time to build a complete CI, they could easily go for this turnkey solution.
We just can’t wait to see what Apple has in store for us at the WWDC 2022!
