Creating an XCFramework

Posted byheckjPosted inGeekstuff, iPhone, mac, Ranting and Reflections, swiftTags:, , , ,

In the past couple of years, I’ve had the occasion to want to make an XCFramework – a bundle that’s used by Apple platforms to encapsulate binary frameworks or libraries – a couple of times. In both cases, the reason wasn’t that I didn’t want to ship the source, but because the source was from a language that isn’t directly supported by Xcode: Rust. The goal of both of these efforts was basically the same – to expose and use library code written in Rust from the Swift language.

Import bits to know about creating an XCFramework

As of this article (2023 for future-me), the best path for exposing a Rust library to Swift if through a C-based FFI interface. Swift knows how to talk to C libraries – both static and dynamic libraries. To use these libraries in an iOS or Mac OS app, what appears to me to be the “best path” is to use this C FFI interface path, and extend that low-level C based API interface with idiomatic Swift. This article aims to walk through some of the specifics of getting from the static library space into Swift. The details of what goes into making an XCFramework is rather sparse, or perhaps more appropriately “terse” through Apple’s documentation. The relevant documents there (definitely worth reading), include:

The key pieces to know when doing tackling this are embedded in the core of the article: Creating a multiplatform binary framework bundle:

  1. For a single library, use the xcodebuild -create-xcframework command with the -library option. There’s also a -framework option, but reserve that for when you need to expose multiple static, or dynamic, libraries as a binary deliverable.
  2. Avoid using dynamic libraries in when you want to support iOS and iOS simulator, as only macOS supports the dynamic linking for these using a Framework structure. Instead, use static libraries.
  3. Use the lipo command to merge libraries when you’re building for x86 and arm architectures, but otherwise DO NOT combine the static libraries for the different platforms. You’ll want, instead, to have separate binaries for each platform you’re targeting.
  4. These days, the iOS simulator libraries need to support BOTH x86 and arm64 architectures, so yep – that’s where you use lipo to merge those two into a single “fat” static library – at least if you’re targeting the iOS simulator on macOS. Same goes for supporting the macOS platform itself.
  5. Get to know the codes called “triples” that represent the platforms you’re targeting. In the world of Rust development, three Apple platforms are “supported” without having to resort to nightly development: iOS, iOS simulator, and macOS. The “triples” are strings (yep – no type system here to double-check your work). Triple is ostensibly to support “CPU”, “Vendor”, and “platform” – but like any fairly dynamic type thing, it’s been extended a bit to support “platform variants”.

The triple codes you’ll likely want to care about, and their platforms:

  • x86_64-apple-ios – the original iOS Simulator on an Intel Mac
  • aarch64-apple-ios-sim – the iOS simulator on an M1/arm based Mac.
  • aarch64-apple-ios – iOS and iPadOS (both are only arm architectures)
  • aarch64-apple-darwin – M1/arm based Macs
  • x86_64-apple-darwin – Intel based Macs

Building libraries for platform and architecture combinations

If you’re following along wanting to generate Rust libraries, then you’ll want to make sure you tell Rust that you want to be able to compile to those various targets. The following commands enable the targets (iOS, iOS simulator, and macOS) for the Rust compiler:

rustup target add x86_64-apple-ios
rustup target add aarch64-apple-ios
rustup target add aarch64-apple-darwin
rustup target add x86_64-apple-darwin
rustup target add aarch64-apple-ios-sim

There’s also the variant triple’s aarch64-apple-ios-macabi and x86_64-apple-ios-macabi that aim to represent Mac Catalyst, but those aren’t easily built with the released, supported version of Rust today. Likewise, there’s work to do the same for watchOS and tvOS. All of those, as far as I know, are only available in “Rust nightlies”, so I’m going to skip over them. That final string extension of -sim or -macabi on the triple is relevant when it comes to XCFrameworks – the command that builds the XCFramework understands that detail and encodes it into the XCFramework’s Info.plist as a platform variant key.

When you go to compile your library, specify the target you want to generate. If you’re using cargo from the Rust ecosystem, the –target option is the key to what you need. For example, the command cargo build --target aarch64-apple-ios --package myPackage --release tells Rust to compile your package for the aarch64-apple-ios target, and with a release build.

Another suggestion I’ve seen is to include that same target as the CFLAGS environment variable. I’m not sure where and how this does anything additional, but given how “interesting” compilers are about picking up environment variables for compilation, I’ve taken the example to heart (fair warning – I’m admitted I’m giving you potentially poor advice, blindly). The pattern I’ve seen used there: export CFLAGS_x86_64_apple_ios="-target aarch64-apple-ios" before invoking cargo build command. (For reference, I spotted this in This Week in Glean from January 2022, providing some examples of how they do this kind of Rust library inclusion.)

If you’re coming from another language, I’ll leave it to you – but there is likely equivalent command options for specifying a “target triple” for the code you’re compiling, and possibly additional compiler flags to specify.

The end result of all this should be a static library – typically a “.a” file – dropped in a directory somewhere. DEFINITELY DOUBLE CHECK that you’ve compiled it for the architectures you expect. Apple’s documentation suggests you use the file command in the terminal to inspect the architecture of your static library, but I’ve found that detail to be pretty useless. In macOS 13.2, file returns the message current ar archive. and no detail about the architecture. Instead, use the command lipo -info with the path to your library to make sure you’ve got the right architecture(s) in there. A couple of examples:

$ lipo -info target/aarch64-apple-ios/release/libuniffi_yniffi.a

Non-fat file: target/aarch64-apple-ios/release/libuniffi_yniffi.a is architecture: arm64

$ lipo -info target/x86_64-apple-ios/release/libuniffi_yniffi.a

Non-fat file: target/x86_64-apple-ios/release/libuniffi_yniffi.a is architecture: x86_64

For those platforms where you need to support more than one architecture (IOS simulator and macOS), you’ll want to take those single-platform static library binaries and merge them together – correctly. The key here is, again, the lipo command. The form of the command is:

lipo -create path_to_first_architecture_static_binary \
path_to_second_architecture_static_binary \
-output path_to_new_location_with_combined_binary

In practical terms, I usually generate a new directory in which I’ll hold these “fat” static libraries – one for each of those platforms: iOS-simulator and apple-darwin.

Once that’s merged together, use lipo -info to double check your work:

$ lipo -info target/ios-simulator/release/libuniffi_yniffi.a

Architectures in the fat file: target/ios-simulator/release/libuniffi_yniffi.a are: x86_64 arm64

At this point, we only need a few more pieces to be able to assemble the final XCFramework.

Gathering the headers and defining a module map

The first thing you’ll need to find is the C header file that matches the static library you just built. Different libraries and frameworks have these in different places, but you’ll need at least one to expose the library over to Swift.

In the case of the project I’ve been helping with, we’re using the Mozilla library UniFFI to generate both the header, and a bit of associated Swift code to make consumption a bit nicer. In its case, it generates the C header for us from the rust library code and declarations that we made. (UniFFI is probably worth its own blog article, it’s been really nice to work with.)

The other thing you critically need is a modulemap file. A module map is a text file that defines what, from header files, should be exposed to Swift – and the name of the module that’s being exposed. This file is not only critical, but unfortunately a bit arcane as well. The documentation for the structure of a modulemap is in the Clang Modules documentation under module map language. If the modulemap file is missing, everything appears to work – but when you attempt to import the module into Swift, you’ll get the error “no such module“, but very little additional detail to go on to understand WHY its not available.

If you’re exposing a static binary with a single header, the “easy” path is to expose everything in that header – functions, types, etc – through to Swift. The module map file for that goal looks something like the following:

module yniffiFFI {
    header "yniffiFFI.h"
    export *
}

In the example above, the module that is being exposed to Swift is yniffiFFI. It references a single header file yniffiFFI.h, expected to be in the same directory as the modulemap.

In the case of UniFFI, the tools helpfully generate a modulemap file, and names it after the name of the Rust library. The UniFFI documentation for module shows an example of providing an explicit link to the modulemap file via a command line compilation. When you’re creating an XCFramework, however, you can’t specify the name of the module map. The XCFramework and compiler expects the name of the file to be the far more generic (and default) module.modulemap. So if you’re using UniFFI, make sure you rename the file to module.modulemap before you provide it to a command assembling an XCFramework.

Aside: It took me a week and half to hunt down that the module wasn’t being picked up when it was named anything other than module.modulemap. Don’t do that to yourself – rename the file.

Building the XCFramework

To create the XCFramework on Apple platforms, the only “supported” route is to use the xcodebuild -create-xcframework command. The other option is to hand-assemble the directory structure of the XCFramework bundle and create the appropriate Info.plist manifest within it. Unfortunately, Apple doesn’t provide any real documentation of the structure(s) expected in an XCFramework. The script that Mozilla uses when assembling their Rust libraries for iOS follows the hand-assembly option, and uses a larger “framework” (vs library) structure as well.

To learn about the -create-xcframework command, use the following command to show the details of how to use xcodebuild to create an XCFramework:

$ xcodebuild -create-xcframework -help

which (in my version) shows:

OVERVIEW: Utility for packaging multiple build configurations of a given library or framework into a single xcframework.
USAGE:
xcodebuild -create-xcframework -framework <path> [-framework <path>...] -output <path>
xcodebuild -create-xcframework -library <path> [-headers <path>] [-library <path> [-headers <path>]...] -output <path>
OPTIONS:
-archive <path>                 Adds a framework or library from the archive at the given <path>. Use with -framework or -library.
-framework <path|name>          Adds a framework from the given <path>.
                                When used with -archive, this should be the name of the framework instead of the full path.
-library <path|name>            Adds a static or dynamic library from the given <path>.
                                When used with -archive, this should be the name of the library instead of the full path.
-headers <path>                 Adds the headers from the given <path>. Only applicable with -library.
-debug-symbols <path>           Adds the debug symbols (dSYMs or bcsymbolmaps) from the given <path>. Can be applied multiple times. Must be used with -framework or -library.
-output <path>                  The <path> to write the xcframework to.
-allow-internal-distribution    Specifies that the created xcframework contains information not suitable for public distribution.
-help                           Show this help content.

For each platform that you’re including in your XCFramework, you’ll want to include a pair of options to the command line with -library pointing to the static binary file (the one you made into a fat static binary file is multiple architectures need to be supported), and a -headers that points to a directory which contains the module.modulemap file and the header file(s) that it references. I code this in a shell script, so this part of the script ends up looking something like:

xcodebuild -create-xcframework \
    -library "./${BUILD_FOLDER}/ios-simulator/release/${LIB_NAME}" \
    -headers "./${BUILD_FOLDER}/includes" \
    -library "./$BUILD_FOLDER/aarch64-apple-ios/release/$LIB_NAME" \
    -headers "./${BUILD_FOLDER}/includes" \
    -output "./${XCFRAMEWORK_FOLDER}"

where the variable XCFRAMEWORK_FOLDER is a directory named for the module that I specified in module.modulemap followed by .xcframework. Using the yniffiFFI module example above, the resulting XCFramework name is yniffiFFI.xcframework.

Structure of an XCFramework

The structure of the library-focused XCFramework is fortunately pretty simple. You can use a command on the terminal such as tree to dump it out:

$ tree yniffiFFI.xcframework
yniffiFFI.xcframework
├── Info.plist
├── ios-arm64
│   ├── Headers
│   │   ├── module.modulemap
│   │   └── yniffiFFI.h
│   └── libuniffi_yniffi.a
└── ios-arm64_x86_64-simulator
    ├── Headers
    │   ├── module.modulemap
    │   └── yniffiFFI.h
    └── libuniffi_yniffi.a
5 directories, 7 files

Each platform-triplet that is supported will have its own directory under the xcframework. In the example above, the framework has support for ios-arm64 (iOS and iPadOS) and the combined static binary file for the iOS simulator. Alongside each of the platform directories is a manifest file: Info.plist.

The module that’s exported within module.modulemap is expected to match the name of the XCFramework. The name of that module is not, however, encoded into the Info.plist manifest.

You can dump the manifest using the plutil command with the -p option in the terminal. In my example: plutil -p yniffiFFI.xcframework/Info.plist results in:

{
  "AvailableLibraries" => [
    0 => {
      "HeadersPath" => "Headers"
      "LibraryIdentifier" => "ios-arm64_x86_64-simulator"
      "LibraryPath" => "libuniffi_yniffi.a"
      "SupportedArchitectures" => [
        0 => "arm64"
        1 => "x86_64"
      ]
      "SupportedPlatform" => "ios"
      "SupportedPlatformVariant" => "simulator"
    }
    1 => {
      "HeadersPath" => "Headers"
      "LibraryIdentifier" => "ios-arm64"
      "LibraryPath" => "libuniffi_yniffi.a"
      "SupportedArchitectures" => [
        0 => "arm64"
      ]
      "SupportedPlatform" => "ios"
    }
  ]
  "CFBundlePackageType" => "XFWK"
  "XCFrameworkFormatVersion" => "1.0"
}

You can expect the top level keys to always have the keys CFBundlePackageType, XCFrameworkFormatVersion, and AvailableLibraries. AvailableLibraries is an array of dictionaries, each of which has the following keys:

  • HeaderPath – a string that represents the directory path to the include files that are shared with this binary
  • LibraryIdentifier – a string that represents the directory name that identifies the platform “triplet” within the XCFramework
  • LibraryPath – a string that represents the name of the binary being exposed
  • SupportedPlatform – a string that represents the the core supported platform
  • SupportedArchitectures – an array of architectures that this static binary supports, such as arm64 and/or x86_64.

And optionally the following key

  • SupportedPlatformVariant – a string that represents the variant of the platform, such as simulator.

This structure is matched by open-source code in SwiftPM: XCFrameworkMetadata.swift, which was interesting to see how it was used in SwiftPM, but ultimately not very helpful for any debugging – as swift build only builds for the current platform (which is what invokes swiftpm), and when I was debugging, I was trying to sort out an iOS only framework.

If you look at those values, they should match exactly to the tree structure and file names you see in the .xcframework directory. If something is awry – a directory misnamed for example – the XCFramework will simply appear to not be loaded.

Debugging the creation of an XCFramework

I hate to say it, but this is a right pain in the ass. There’s no real tooling available to “vet” if an XCFramework is correct, although depending on how it’s screwed up – you might get some semi-useful messages from development tools like Xcode.

If you’re using swift and Swift Package Manger on the command line to build using a binary target (and referencing an XCFramework), then the failures are mostly – intentionally – silent. The reason is that the build systems (Xcode, SwiftPM, etc) do some work to find, sometimes download, an XCFramework and expose it somewhere in your local operating system. Then it adds the paths to the binary, and the headers, to the swift compiler.

Does not contain the expected binary

If you get the message “... does not contain the expected binary artifact“, (a error that I got from Xcode while “learning the ropes”), then the issue is that the name of the XCFramework doesn’t match the name of the module that’s exposed within it. The naming convention is pretty darned tight – and definitely case sensitive. If you renamed the xcframework directory without doing all the work to rename its internals – you’ll be hitting this.

In addition to the name of exported module needing to be reflected in the name of the XCFramework, it also should be the name of the binary target in Package.swift.

Oh – and if you’re using a compressed binary target (such as a compress zip file hosted somewhere), absolutely make sure you compress the xcframework using ditto with the option --keepParent. This same error can crop up if you fail to include the –keepParent option because as Xcode unpacks the XCFramework, it’ll expand itself into a different name, and you’ll be right back at the same “does not contain the expected binary” error message.

No such module …

If you’re attempting to load a module from an XCFramework and you’re getting “no such module” as the error, then something has gone awry with the loading of the modules. There’s unfortunately very little you can do to see WHAT has been loaded. If you’re in Xcode, then the most convenient way to see what’s available to import is to use the code completion capability, and start typing “import …” into a Swift file, and see what shows up in the editor.

Another suggestion, if you’re building for your local platform with swift package manager, was to use the swift build --verbose command. You can then repeat the whole of one of the underlying commands that it doesn’t with an additional option -Rmodule-loading, which prints out an additional debugging message from the compiler that let’s you know what’s being loaded – or at least from where. You’ll see messages akin to the following:

<unknown>:0: remark: loaded module at /Users/heckj/src/y-uniffi/lib/.build/arm64-apple-macosx/debug/ModuleCache/2C5HQS9YK727C/SwiftShims-2DA6NLEWJC11R.pcm
<unknown>:0: remark: loaded module at /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/usr/lib/swift/Swift.swiftmodule/arm64e-apple-macos.swiftinterface
<unknown>:0: remark: loaded module at /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/usr/lib/swift/_StringProcessing.swiftmodule/arm64e-apple-macos.swiftinterface
<unknown>:0: remark: loaded module at /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk/usr/lib/swift/_Concurrency.swiftmodule/arm64e-apple-macos.swiftinterface

These are all the locations which are actively getting loaded through the module cache. The notable thing isn’t what’s there – but what’s not there. In the event the XCFramework isn’t loaded, then you won’t see it – or it’s expanded version location – listed.

Aside: Exploring the swift module cache

If you’re really curious and want to see what’s in those files, you can directly look at anything with the .swiftinterface extension as a plain text file. That’ll show you not only the module name, but compilation flags and quite a lot of detail of the swift modules. For binary files, they’ll be exposed as .pcm files, which are a binary format. You can look at those by using the swift compiler through the swiftc -dump-pcm command. It dumps a LOT of detail, so I recommend piping the results through a pager, such as less:

swiftc -dump-pcm /Users/heckj/src/y-uniffi/lib/.build/arm64-apple-macosx/debug/ModuleCache/2C5HQS9YK727C/SwiftShims-2DA6NLEWJC11R.pcm | less

It’ll share all the details of the options of HOW the PCM was compiled (options, targets, etc) as well as include links to the inputs that generated it – including system module map files and headers. While interesting – and critical for a compiler and what it needs to do – I didn’t find it explicitly helpful in debugging why an XCFramework wasn’t loaded, so I’m including this detail only for completeness.

I’m going to guess the most common reason is that the XCFramework doesn’t contain the target that you’re attempting to build for. For example, swift build on macOS builds for the architecture and platform of your local machine – nothing else. In my case, with an M1 Mac, it looks and builds for the architecture arm64-apple-macosx11.0. If you failed to include any binaries for the target arm64-apple-darwin, this is exactly the reason.

Probably your best bet is to investigate the structure and architecture of the binaries included in the XCFramework – as well as look for the module.modulemap file and verify the module you’re expecting is there – and matches the name of the XCFramework. The commands tree and lipo -info are invaluable, both of which I showed above.

Compressing and using a remote XCFramework

If you’re using the XCFramework locally and directly, then you can reference it by location on your local machine. The binaryTarget stanza in a Package.swift file looks something like:

.binaryTarget(
    name: "yniffiFFI",
    path: "./yniffiFFI.xcframework"
),

The name for this target should match the name of the module being exposed, and the name of the XCFramework as well.

But a common way to want to use this – the whole “distributed binaries” point of binaryTarget – is to use an external, hosted version. You can use a service such as GitHub or GitLab to host binary results with a tag that work well for this sort of thing.

In order to do this, you’ll need to do two things:

  1. Compress the XCFramework into a .zip file.
  2. Compute a signature to use when referencing that .zip file.

Both of these are well defined in Distributing binary frameworks as Swift packages. You can use ditto to compress an XCFramework. As noted earlier, make sure to use the --keepParent option. As an example:

ditto -c -k --sequesterRsrc --keepParent "$XCFRAMEWORK_FOLDER" "$XCFRAMEWORK_FOLDER.zip"

Once the XCFramework is compressed, compute the checksum. The Apple documentation offers the command:

swift package compute-checksum path/to/MyFramework.zip

But the checksum is a SHA256 digest of the zip, so you can also use openssl to get the same result:

openssl dgst -sha256 "$XCFRAMEWORK_FOLDER.zip"

Use the resulting string where you want to define a remote binary target within a Package.swift:

.binaryTarget(
    name: "yniffiFFI",
    url: "https://github.com/heckj/yniffi/releases/download/0.0.1/YniffiFFI.xcframework.zip",
    checksum: "098a5bc1f62dd2efa3b316daa14e08cb584515c9ae866cc78e0ad4c3154ab6f2"
),

To be clear, the example above will not function as is – I made up the values. So feel free to copy it, but don’t expect that to work as is. It’s an example for you to use, replacing your own values.

Published by heckj

Developer, author, and life-long student. Writes online at https://rhonabwy.com/.