Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Back-deploy concurrency #39342

Merged
merged 63 commits into from Sep 24, 2021

Conversation

DougGregor
Copy link
Member

Back-deployment support for Swift concurrency.

DougGregor and others added 30 commits September 15, 2021 09:19
This assertion does not account for the fact that a particular
requirement can be imported both as async and with a completion
handler, and therefore one of those two views won't have a witness.
The assertion is no longer catching anything useful, so remove it.

Addresses rdar://77684000.
…nd out of the top level build-script file.

This shrinks the number of lines in build-script by 50% and just makes it easier
to read. It will also enable me to refactor parts of it into pieces without
touching the main build-script file. This is really library code... build-script
itself doesn't care about how BuildScriptInvocation is actually implemented
under the hood.

(cherry picked from commit c86e3af)
Fixes SR-14701.

(cherry picked from commit 526b0ef)
…opt into using the just built swift toolchain.

(cherry picked from commit c7001f9)
In a back deployment scenario, this will provide a place where one could provide
function implementations that are not available in the relevant stdlib.

This is just setting up for future work and isn't doing anything interesting
beyond wiring it up/making sure that it is wired up correctly with tests.

(cherry picked from commit 1bc94bf)
Add a frontend-only flag `-enable-experimental-back-deploy-concurrency`
to be used to stage in the back deployment of concurrency. At present,
all it does is lower the availability minimums for use of concurrency
features.

(cherry picked from commit 76c4ac8)
Introduce an additional build product to build-script to build
back-deployable concurrency libraries. These libraries would need to
be embedded in apps deployed prior to macOS 12/iOS 15 to support
concurrency.

The built-script option `--back-deploy-concurrency` can be provided to
build these back-deployment libraries. They are built in addition to
the normal concurrency libraries, as a separate product that installs
into `lib/swift-5.5/<platform>` within the toolchain. The macro
`SWIFT_CONCURRENCY_BACK_DEPLOYMENT` is set when building the
concurrency library, so that we can adapt the implementation to older
OS's.

(cherry picked from commit 3f749dc)
…yment.

When enabling the build of the back-deployable concurrency library via
the build-script option `--back-deploy-concurrency`, also build the
compiler and (main) concurrency library to support older deployment targets.

Building the compiler for older deployment targets is effectively the
same as implicitly passing
`-Xfrontend -enable-experimental-back-deploy-concurrency`. That option
should probably go away.

Building the primary _Concurrency library for back-deployment means
setting the "SwiftStdlib 5.5" availability back to the earlier
deployment targets. This should have no effect on how the _Concurrency
library binary is built, but it does ensure that the right
availability annotations are in the _Concurrency module.

(cherry picked from commit f2995a3)
…ncy.

When build-script is given `--back-deploy-concurrency`, also use that
to build other parts of Swift with the back-deployed versions:

* The compiler allows async and actors to be defined with the
back-deployed availability, e.g., the same as `-Xfrontend
-enable-experimental-back-deploy-concurrency`. (The latter will go
away soon)
* The standard library unit testing framework and distributed actors
library are build with the older OS versions.
* The tests use the older OS versions, with some adjustments to make
them agnostic to the back-deployment setting.

(cherry picked from commit ebd7b49)
Allow the installation of the back-deployment shared libraries for
concurrency via build-script.

(cherry picked from commit 6d6c2e6)
… name

When linking against the back-deployed _Concurrency library, ensure that
clients record an `@rpath`-relative dependency.

(cherry picked from commit 75a5831)
When testing the runtime in the OS (via the lit parameter
`use_os_stdlib`), add the path to the back-deployed concurrency
libraries into `DYLD_LIBRARY_PATH` for the test run. This should allow
us to run our regression tests on older OS's.

(cherry picked from commit 0573f1e)
Remove the option that explicitly enables concurrency back-deployment,
and instead always enable its support in the compiler. Remove the use
of the extraneous CMake option as well.

(cherry picked from commit 5999743)
Swift 5.5 does not implement support for structural types that are
Sendable, which breaks type checking of the newer _Concurrency module.
Work around the issue with an `#if` in the offending inline code.

Fixes rdar://82602353.

(cherry picked from commit 95d84d6)
…erface

Swift 5.5 didn't support back deployment of concurrency features, so a
Swift 5.5 compiler processing the _Concurrency .swiftinterface will
produce errors for each `async` function and actor with pre-macOS
12/iOS 15 availability. Emit `-disable-availability-checking` into the
generated `.swiftinterface` files to allow Swift 5.5 to continue to
build them.

Finishes rdar://82602353.

(cherry picked from commit a6c88ee)
…gurations.

`swift_async_extendedFramePointerFlags` is used in back-deployed
concurrency code as a mask to set the extended async frame info bit on
OS versions that understand it. Set this mask to zero in the
back-deployed libraries.

(cherry picked from commit 03956e0)
When back-deploying concurrency support, do not use the standard
substitutions for _Concurrency-defined types (such as `Task`) in type
metadata because older Swift runtimes will not be able to demangle
them. Instead, use the full mangled names so the runtime can still
demangle them appropriately.

Addresses rdar://82931890.

(cherry picked from commit 7f21176)
Prior to this, build-script products that build in the host-specific
directories for cross-compilation would get the wrong path, and
therefore wouldn't get picked up in toolchain builds.

(cherry picked from commit 0068722)
When back-deploying, create global-actor-qualified function types via a
separate entrypoint
(`swift_getFunctionTypeMetadataGlobalActorBackDeploy`) in the
compatibility library, which checks whether it is running with a
new-enough runtime to use `swift_getFunctionTypeMetadataGlobalActor`.
Failing that, it calls into a separate copy of the implementation that
exists only in the back-deployed concurrency library.

Fixes rdar://79153988.

(cherry picked from commit 5b027ca)
@swift-ci
Copy link
Collaborator

Build failed
Swift Test OS X Platform
Git Sha - cfd87f4

@DougGregor
Copy link
Member Author

@swift-ci please test

@DougGregor
Copy link
Member Author

@swift-ci please build toolchain macOS

@swift-ci
Copy link
Collaborator

Build failed
Swift Test Linux Platform
Git Sha - 6d876bb700042a742300089b9ae47fb52af3c1ac

@swift-ci
Copy link
Collaborator

Build failed
Swift Test OS X Platform
Git Sha - 6d876bb700042a742300089b9ae47fb52af3c1ac

@DougGregor
Copy link
Member Author

This is actually wonderful news:

Test Result (5 failures / -6)
Swift(macosx-x86_64).BuildSystem.infer_dumps_deps_if_verbose_build.test
Swift(macosx-x86_64).BuildSystem.infer_implies_install_all.test
Swift(macosx-x86_64).Python.build_swift.swift
Swift(macosx-x86_64).Python.python_lint.swift
Swift(macosx-x86_64).Python.swift_build_support.swift

It's just silly pedantic Python failures that don't matter.

gottesmm and others added 9 commits September 23, 2021 20:32
… passing -DCMAKE_AR to cmake.

I am currently working on being able to build cmake build-script projects as
always being cross compiled by using toolchain files (1). CMake expects in means
that I need to be able to specify CMAKE_AR in said toolchain.

(1) We still build for the host machine (of course), but instead of building it
normally with cmake, we build it as if we are cross compiling meaning we are
cross compiling for the host on the host. This makes it so that the host build
is no longer different from cross compilation builds making Swift's cross
compilation build more robust since it will be tested and less mysterious.

(cherry picked from commit 6f4ef7c)
… when neccessary

When we deploy to a minimum target that is not known to support extended
frame information the function prolog of Swift async functions will
contain a reference to swift_async_extendedFramePointerFlags. This
reference needs to be weak like any async symbols.

rdar://83412550
(cherry picked from commit 16a5d69)
(cherry picked from commit d52d4d5)
(cherry picked from commit aa3e7fb)
@DougGregor
Copy link
Member Author

@swift-ci please test

@DougGregor
Copy link
Member Author

@swift-ci please build toolchain macOS

@DougGregor
Copy link
Member Author

@swift-ci please test Windows

@swift-ci
Copy link
Collaborator

macOS Toolchain
Download Toolchain
Git Sha - dab326a

Install command
tar -zxf swift-PR-39342-1133-osx.tar.gz --directory ~/

@DougGregor
Copy link
Member Author

Alright, there's a bit more to come here to fix some known bugs, but this is a good baseline to land. :shipit:

@DougGregor DougGregor merged commit b03cde6 into apple:release/5.5 Sep 24, 2021
@DougGregor DougGregor deleted the back-deploy-concurrency-5.5 branch September 24, 2021 17:00
@unnamedd
Copy link

Woohoo!!! Great work done here @DougGregor!

oozoofrog pushed a commit to oozoofrog/SwiftDemangle that referenced this pull request Oct 10, 2021
oozoofrog pushed a commit to oozoofrog/SwiftDemangle that referenced this pull request Oct 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet