iTranslated by AI
Introduction to Swift Compiler Development
Background
Many years have passed since the Swift compiler became open source, and with continuous improvements, participating in compiler development has become more accessible. On the other hand, perhaps due to the development team being busy, reported issues often remain unaddressed for long periods. Consequently, there is an increasing need for Swift users to submit patches themselves.
Target Audience
In this article, I will explain the know-how and procedures for general Swift users to start developing the Swift compiler. By doing so, readers will be able to submit patches themselves, accelerating the improvement of the Swift compiler. Ultimately, all Swift users will benefit from these improvements.
Please note that this guide assumes a Mac with reasonable specs. Apple Silicon is recommended and will be the basis for this article. If you are using an Intel-based Mac, please adapt the instructions accordingly.
Setting Up Your Local Environment
First, let's set up your local machine environment.
The basic information is covered in the manual, but I will explain it here once more.
To start, install the necessary software.
Xcode
Please install Xcode. Generally, the latest stable version is preferred. In very rare cases, the latest Beta version may be required. According to the manual, the required version is apparently listed on the CI webpage.
If you have multiple versions of Xcode installed on your machine, be careful not to forget the setting to specify which version of Xcode to use in the CLI environment.
You can configure this from the menu bar by going to Xcode > Settings... to open the settings dialog, then selecting it from the Command Line Tools: selection box in the Locations tab within the Locations page.
Homebrew
Let's install Homebrew to include other necessary tools. If you have never used it before, make sure to follow the steps for adding it to your path during the setup process.
Git
Let's install Git. The version bundled with macOS should be fine, but it's safer to install the latest version via Homebrew.
brew install git
CMake
CMake is a cross-platform meta-build tool. By configuring a project with CMake, it becomes possible to build projects—primarily C++—on various operating systems. Building with CMake allows for generating platform-native IDE projects, such as Xcode projects on macOS or Visual Studio projects on Windows, and it can also output CLI build configurations like Makefiles or Ninja files on Linux. Afterward, these tools are used to build the actual C++ project. However, as mentioned later, you don't need to worry about anything other than Ninja generation.
Let's install the latest version via Homebrew.
brew install cmake
Ninja
Ninja is a high-speed build tool. In Swift compiler development, we primarily use it for builds. The aforementioned CMake outputs build configurations for Ninja.
Let's install the latest version via Homebrew.
brew install ninja
Python
Python is used as the scripting language in the Swift compiler project, and it is required for both checking out the source and building. The aforementioned CMake is invoked via these Python scripts. While the version bundled with macOS might work, it's safer to use a newer version to avoid potential compatibility issues. Using Homebrew for the Python installation is the simplest approach.
brew install python
Flake8
Flake8 is a Python linter. It is integrated into the automated tests, so it's a good idea to install it. Use pip, Python's standard package manager, to install it.
python3 -m pip install --break-system-packages flake8 flake8-import-order
However, please note that specifying --break-system-packages is at your own risk.
This completes the machine environment preparation.
Checkout
Once the machine environment is ready, it's finally time to obtain the source code.
Preparing the Project Directory
Before checking out the source code, let's prepare the directory. The Swift compiler project consists of multiple Git repositories arranged flatly in the same directory. First, create an empty directory to house these.
mkdir swiftlang
You can name the project directory whatever you like, but for the following explanation, I'll assume it's called swiftlang.
Checking out the Swift Repository
First, check out the Swift repository. Simply check it out using Git into the directory you just prepared.
cd swiftlang
git clone https://github.com/swiftlang/swift
This will take a very long time. Let's take a break.
Checking out Dependency Repositories
The Swift repository has several dependency repositories. These are not managed using a standard mechanism like Git submodules, but rather with a dedicated script included in the Swift repository.
These will be deployed in the project directory, side-by-side with the swift directory.
Check them out using the following command:
cd swiftlang/swift
utils/update-checkout --clone
The --clone option instructs the script to newly obtain dependency repositories that haven't been checked out yet. Therefore, it is mandatory for the first run like this. When you only need to update the dependencies in the future while continuing development, it won't be necessary. However, dependency repositories are added quite frequently, and if you miss them and fail to obtain them, it can cause trouble, so it's safer to always specify it.
This step probably took quite a while as well.
It might seem like the checkout is finished, but an important step follows.
Finding a Stable Version
Through the steps so far, you have checked out the latest version. However, the bleeding-edge Swift project is frequently broken. Since the correspondence between specific commits in the main Swift repository and those in its dependency repositories is not fixed, it breaks quite naturally. Consequently, the version you just checked out likely won't build. Therefore, let's first find a stable version that can be built.
The Swift project is built daily by CI, and snapshot tags are created whenever a build succeeds. These tags are applied across all dependency repositories, which also fixes the combination of commits.
Snapshot tags follow a format like swift-DEVELOPMENT-SNAPSHOT-yyyy-MM-dd-a. The a part is likely a digit that increments to b, c, and so on if multiple snapshots are released on the same day, but I have only ever seen a.
Use Git to list the tags and look for one with the most recent date.
git tag -l
As you will see when you look at them, it is common for snapshots not to be released for about two weeks at a time.
Also, be careful not to confuse these with tags that include language versions, such as swift-6.1-DEVELOPMENT-SNAPSHOT-yyyy-MM-dd-a. These are snapshots for specific branches cut for release plans; they don't incorporate the latest changes and are generally not needed for regular compiler development.
Once you have decided on a tag to check out, proceed to the next step. In the following examples, we will assume swift-DEVELOPMENT-SNAPSHOT-2024-12-13-a was selected.
Checking out the Snapshot
To check out a snapshot, use the aforementioned script again. This will also update the Swift repository itself.
cd swiftlang/swift
utils/update-checkout --clone --tag swift-DEVELOPMENT-SNAPSHOT-2024-12-13-a
Specify the snapshot tag you selected earlier using the --tag option.
This completes the checkout process.
Building
Now it's finally time to build.
To perform the build, run the dedicated script. It will run CMake to generate the Ninja build configuration, followed by Ninja itself to execute the build.
Start the build with the following command:
cd swiftlang/swift
utils/build-script --skip-build-benchmarks \
--swift-darwin-supported-archs "$(uname -m)" \
--release-debuginfo --swift-disable-dead-stripping \
--bootstrapping=hosttools
This will take a very long time. Your CPU will run at full capacity, the machine will heat up, and the fans will spin loudly. Exciting, isn't it?
The command has many options and may look complicated, but these are the ones listed in the manual. You can simply copy and paste them, as sticking to these is generally sufficient and there's no need to memorize them.
One important part to keep in mind is the --release-debuginfo specification. This determines the compilation mode for the Swift compiler itself, and it stands for a release build with debug information. It's a bit like an oxymoron (like the "thornless thorn" beetle), but essentially, it enables optimizations for high performance while still allowing you to attach a debugger for troubleshooting—a "best of both worlds" approach between debug and release modes. This mode is also referred to as RelWithDebInfo in other contexts.
While it offers the best of both worlds, optimizations can sometimes interfere with the debugger's functionality. For maximum focus on debugging, you can specify --debug instead for a pure debug mode. However, note that this will result in slower performance.
Additionally, there are many other options for controlling the build, such as building only the LLVM portion in release mode. If you're interested, you can check the help by specifying the -h option.
Once the build is finished, a build directory will be created in your project directory.
Its structure will look like this:
cd swiftlang
tree -L 2 build
build
└── Ninja-RelWithDebInfoAssert
├── cmark-macosx-arm64
├── earlyswiftdriver-macosx-arm64
├── llvm-macosx-arm64
└── swift-macosx-arm64
The first-level directory represents the build tool and compilation mode specified. In this example, it shows that the build tool was Ninja and the mode was release with debug information. If these specifications change, a different directory will be created, allowing intermediate build files and caches to be isolated and reused as appropriate. This first-level directory is essentially the build directory and will appear in various contexts. The root build directory serves as a storage directory for build directories.
The second-level directories divide the subprojects and are suffixed with the CPU architecture. The toolchain package for distributing Swift includes not only the Swift compiler but also Clang (a C compiler) and other components. Since Clang and others originate from the LLVM project, they are generated in the llvm-macos-arm64 directory, while Swift project products like the Swift compiler are contained in the swift-macosx-arm64 directory. On an Intel machine, these names will be slightly different.
If the Build Fails
Unfortunately, you might run into a build failure. Even if you think you’ve set everything up correctly, issues can arise due to your local environment or occasional incompatibilities with specific macOS versions.
In such cases, first delete the build directory mentioned above. Then, it’s best to start over from the step of finding and checking out a stable version. You might want to try a version that is about a month old.
If you are unsure about your local environment and want to find the most stable version possible, consider choosing a release tag instead of a daily snapshot. Release tags are attached to versions shipped to the general public and follow a format like swift-6.0.3-RELEASE.
Working with Xcode
Generating an Xcode Project
Once the build has succeeded, let's set up an environment for editing the code. You can develop the Swift compiler using Xcode.
I mentioned earlier that CMake can output Xcode projects in addition to Ninja configurations. Until recently, using that feature was the recommended approach. However, a completely new method has recently emerged that eliminates the need to use CMake for this purpose. It is a dedicated tool called swift-xcodegen that generates projects based on Ninja build configurations. It was introduced in November 2024 via the following PR:
swift-xcodegen is implemented as a standalone Swift package. Since it’s written in Swift, you might find it interesting to look at its source code. As it is a new tool, it is currently under active development.
To generate a project, run the following command. This is also described in the manual.
cd swiftlang/swift
utils/generate-xcode ../build/Ninja-RelWithDebInfoAssert
Specify the build directory as an argument to the command.
The generated Xcode project will be placed in the project directory under the name Swift.xcodeproj. Let's open it immediately.
cd swiftlang
open Swift.xcodeproj
Xcode Project Structure
When you open the Xcode project, it looks like this:
The file tree on the left roughly corresponds to the actual file structure. There are many targets and many schemes generated. The scheme selector is located at the far left of the bar in the top center of the Xcode window.
When working, select one of the provided schemes and build or run using standard Xcode operations. The core compiler itself is the swift-frontend scheme. For now, select it and try building with ⌘ + B. Since it internally calls the underlying Ninja, the build cache should work, and the build should complete quickly if everything is set up correctly.
Note that while the compiler core is swift-frontend, the swift command you typically use is different. The swift command is actually swift-driver. The driver receives command-line arguments and converts them into calls to the frontend command. Depending on the input, the driver may invoke the frontend multiple times. In other words, the frontend holds the compiler functionality, so it's the behavior of the frontend that you'll mostly be changing in compiler development.
Preparing a Working Directory
In compiler development, you'll need to feed source code into the compiler as input, so it's a good idea to set up a working directory. Let's create a work directory in your project directory.
cd swiftlang
mkdir work
Let's prepare a source file.
cd work
cat > a.swift << EOF
print("hello")
EOF
Execution Settings
Let's configure swift-frontend so it can run in the working directory. Select Edit Scheme... from the Xcode scheme menu.
In the left pane of the dialog, select the Run page and open the Options tab. Edit the Working Directory item there. Check the Use custom working directory: box and specify the full path to your working directory in the text field below.
Next, open the Arguments tab. Set -dump-ast a.swift in the Arguments Passed On Launch section.
Close the dialog to save the settings, then run with ⌘ + R. If the AST is output to the Xcode console, it's working correctly.
-dump-ast is one of the frontend's operating modes; it parses the source code, performs type checking, and outputs the result.
Trying Out Debugging
Let's try debugging in Xcode. Set a breakpoint at the beginning of the function evaluator::SideEffect TypeCheckSourceFileRequest::evaluate(Evaluator &eval, SourceFile *SF) const in TypeChecker.cpp.
Run it. Since -dump-ast performs type checking, it will hit this breakpoint.
Execution will pause, and the stack trace and variable viewer will work.
Let's try using LLDB. Enter e SF->dump() into the console. The dump method will execute, and the AST of the source file will be displayed in the console.
Since this is the file about to undergo type checking, you can confirm that the types haven't been resolved yet.
In the Swift compiler code, many types have a dump method implemented, which is very useful for debugging like this.
I've explained the basic development workflow.
Operations During Development
In addition to code editing and debugging, various other operations will be necessary. I will explain how to perform them.
Adding and Deleting Files
Adding or deleting source files is quite a challenge. Operating within the Xcode project likely won't work correctly. For C++ source files, you need to edit CMakeLists.txt, which is a CMake definition file. After editing, you need to re-run CMake, so you'll need to run utils/build-script again. Furthermore, you also need to re-run utils/generate-xcode. It's quite tedious, so it's best to avoid it if possible.
Rebuilding
There are several ways to rebuild. First, you can build as usual within Xcode. Re-running utils/build-script is also an option. If there are no changes to the file structure, you can also rebuild by directly executing Ninja from the CLI as follows:
cd swiftlang/swift
ninja -C ../build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64 bin/swift-frontend
Specify the -C option followed by the project directory within the build directory. Then, after a space, specify the build product. Ninja will resolve dependencies based on its definitions and perform the necessary build steps.
By the way, if you look at the Build Phases settings of the ninja-build-swift-frontend target in the Xcode project, you'll see that it is configured to execute the above Ninja command as a Run Script.
Cleaning
If rebuilding stops working correctly, you should clean and start over.
As a method for cleaning, you can specify the --clean option in utils/build-script. In that case, keep all other options configured exactly the same.
However, a more reliable and simpler method is to just delete the build directory from the file system. All intermediate build files are consolidated there, so it will wipe everything clean.
Testing
Once you've finished editing the code, you should run automated tests to ensure that existing functionality hasn't been broken. Currently, this cannot be done from the Xcode project, so you must use the CLI.
Test cases are written as individual Swift source files within the test directory.
It is convenient to use a dedicated script to run the tests. It is used as follows:
cd swiftlang/swift
utils/run-test --build-dir ../build/Ninja-RelWithDebInfoAssert test/Parse
Specify the build directory with the --build-dir option, followed by the test cases you want to run. In this example, since the test/Parse directory is specified, all test cases under it will be executed. This example is also described in the manual.
You can specify individual files in even greater detail, or conversely, specify only test to run everything. It's a good idea to do this before submitting a patch.
There is also a directory called validation-test. This is a test suite for verifying a wider range of operations. It can be specified in the run-test script just like test. It takes much longer to run, but it's better to execute it if you're unsure about the scope of impact of your changes.
As a side note, rather than running tests after completing your patch, it might be better to run them once before you start working. This is because even if the build succeeds, some tests might not pass in your local environment from the start. In that case, you won't have to worry as much if those pre-failing tests still fail after applying your patch, whereas not knowing they were already failing can lead to unnecessary trouble.
Submitting a Patch
Once you have finished your work, it's time to submit it. Following the general etiquette for open-source projects on GitHub should be sufficient. Writing in English is often the biggest hurdle, but I've found it much easier recently by using ChatGPT.
CI
Once your review and CI (Continuous Integration) pass, a maintainer will merge your changes. However, this CI process is also somewhat unique.
You can trigger the CI by mentioning a bot in the comments section of a GitHub pull request, but only users recognized as committers have the authority to do this. As a beginner contributor, you won't be able to start the CI yourself, so you'll need to wait for a maintainer to handle it or ask a friend who is a committer.
I mentioned earlier that the main branch often fails to build, and for the same reason, CI often fails as well. This is because the CI simply checks out the latest environment for dependency repositories other than the main Swift repository.
Basically, your patch needs to pass in three environments: Ubuntu, macOS, and Windows. If you look at these status pages, you'll see that they fail quite often.
So, even if an error occurs in the CI after you submit your patch, stay calm and check the logs. It's often the case that your changes are unrelated to the failure. Monitor the status pages and try re-triggering the CI when there are many successes and things seem to be in good shape.
Aiming to Become a Committer
You can apply for committer status once about five of your good patches have been accepted. It might be a good idea to aim for this initially.
Themes to Work On
So far, I have explained how to participate in compiler development. However, I should also explain what kind of themes you should work on in the first place.
Immediate Changes to the Swift Language are Not Allowed
First and most importantly, additions of features or specification changes to the Swift language itself are not allowed. Such changes must go through a process called Swift Evolution, which requires the submission of proposal documents and review in the forums. Therefore, please note that such changes will not be merged directly even if you submit them.
Bug Fixes
An easy entry point is fixing compiler bugs. Since there are about 7,000 issues accumulated, find a bug you're interested in and fix it.
In rare cases, it can be ambiguous whether something is a language specification change or a bug fix. In such cases, if you submit a patch as a bug fix for the time being, it is easier to exchange opinions during the review process.
Improving the Compiler Developer Experience
As we have seen, the Swift repository contains many tools and scripts for compiler development in addition to the compiler source itself. If you find any issues while developing, it would be a good idea to try improving them.
Improving Code Quality
Even if there are no bug fixes or behavioral changes, improving code quality is also worthwhile. The codebase is massive and the pace of development is fast, so you might find parts that aren't necessarily finished to the highest standard. Therefore, simply reading the code in parts you like is a good approach.
Performance Improvements
Improving the execution performance of the compiler is also good. However, regarding this point, the existing code is written at a quite high level. It might be difficult to find areas for improvement.
Common Concerns
Lastly, I will touch upon some common concerns.
I Don't Know C++
You might think you can't work on it because you don't know C++. However, recently, parts of the compiler source are being written in Swift. It might be a good idea to approach those areas.
I Don't Know Compilers
Even so, you might feel like you don't understand anything because you lack knowledge about compilers. In that case, you might want to join a Swift Compiler Study Group. There, you can consult with knowledgeable people.
Discussion