Skip to content

Commit f1caa8d

Browse files
alexcrichtonmark-i-m
alexcrichton
authored and
mark-i-m
committed
Add some documentation about updating LLVM
1 parent 022d3f4 commit f1caa8d

File tree

2 files changed

+150
-0
lines changed

2 files changed

+150
-0
lines changed

‎src/SUMMARY.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@
6565
- [miri const evaluator](./miri.md)
6666
- [Parameter Environments](./param_env.md)
6767
- [Code Generation](./codegen.md)
68+
- [Updating LLVM](./codegen/updating-llvm.md)
6869
- [Emitting Diagnostics](./diag.md)
6970

7071
---

‎src/codegen/updating-llvm.md

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
# Updating LLVM
2+
3+
The Rust compiler uses LLVM as its primary codegen backend today, and naturally
4+
we want to at least occasionally update this dependency! Currently we do not
5+
have a strict policy about when to update LLVM or what it can be updated to, but
6+
a few guidelines are applied:
7+
8+
* We try to always support the latest released version of LLVM
9+
* We try to support the "last few" versions of LLVM (how many is changing over
10+
time)
11+
* We allow moving to arbitrary commits during development.
12+
* Strongly prefer to upstream all patches to LLVM before including them in
13+
rustc.
14+
15+
This policy may change over time (or may actually start to exist as a formal
16+
policy!), but for now these are rough guidelines!
17+
18+
## Why update LLVM?
19+
20+
There are two primary reasons nowadays that we want to update LLVM in one way or
21+
another:
22+
23+
* First, a bug could have been fixed! Often we find bugs in the compiler and fix
24+
them upstream in LLVM. We'll want to pull fixes back to the compiler itself as
25+
they're merged upstream.
26+
27+
* Second, a new feature may be avaiable in LLVM that we want to use in rustc,
28+
but we don't want to wait for a full LLVM release to test it out.
29+
30+
Each of these reasons has a different strategy for updating LLVM, and we'll go
31+
over both in detail here.
32+
33+
## Bugfix Updates
34+
35+
For updates of LLVM that typically just update a bug, we cherry-pick the bugfix
36+
to the branch we're already using. The steps for this are:
37+
38+
1. Make sure the bugfix is in upstream LLVM.
39+
2. Identify the branch that rustc is currently using. The `src/llvm` submodule
40+
is always pinned to a branch of the
41+
[rust-lang/llvm](https://github.com/rust-lang/llvm) repository.
42+
3. Fork the rust-lang/llvm repository
43+
4. Check out the appropriate branch (typically named `rust-llvm-release-*`)
44+
5. Cherry-pick the upstream commit onto the branch
45+
6. Push this branch to your fork
46+
7. Send a Pull Request to rust-lang/llvm to the same branch as before
47+
8. Wait for the PR to be merged
48+
9. Send a PR to rust-lang/rust updating the `src/llvm` submodule with your bugfix
49+
10. Wait for PR to be merged
50+
51+
The tl;dr; is that we can cherry-pick bugfixes at any time and pull them back
52+
into the rust-lang/llvm branch that we're using, and getting it into the
53+
compiler is just updating the submodule via a PR!
54+
55+
Example PRs look like:
56+
[#56313](https://github.com/rust-lang/rust/pull/56313)
57+
58+
## Feature updates
59+
60+
> Note that this is all information as applies to the current day in age. This
61+
> process for updating LLVM changes with practically all LLVM updates, so this
62+
> may be out of date!
63+
64+
Unlike bugfixes, updating to pick up a new feature of LLVM typically requires a
65+
lot more work. This is where we can't reasonably cherry-pick commits backwards
66+
so we need to do a full update. There's a lot of stuff to do here, so let's go
67+
through each in detail.
68+
69+
1. Create new branches in all repositories for this update. Branches should be
70+
named `rust-llvm-release-X-Y-Z-vA` where `X.Y.Z` is the LLVM version and `A`
71+
is just increasing based on if there's previous branches of this name. All
72+
repositories here should be branched at the same time from the upstream LLVM
73+
projects, we currently use https://github.com/llvm-mirror repositories. The
74+
list of repositories that need a new branch are:
75+
76+
* rust-lang/llvm
77+
* rust-lang/compiler-rt
78+
* rust-lang/lld
79+
* rust-lang-nursery/lldb
80+
* rust-lang-nursery/clang
81+
82+
2. Apply Rust-specific patches to LLVM repositories. All features and bugfixes
83+
are upstream, but there's often some weird build-related patches that don't
84+
make sense to upstream which we have on our repositories. These patches are
85+
typically the latest patches on the branch. All repositories, except `clang`,
86+
currently have Rust-specific patches.
87+
88+
3. Update the `compiler-rt` submodule in the
89+
`rust-lang-nursery/compiler-builtins` repository. Push this update to a
90+
`rust-llvm-release-*` branch of the `compiler-builtins` repository.
91+
92+
4. Prepare a commit to rust-lang/rust
93+
94+
* Update `src/llvm`
95+
* Update `src/tools/lld`
96+
* Update `src/tools/lldb`
97+
* Update `src/tools/clang`
98+
* Update `src/libcompiler_builtins
99+
* Edit `src/rustllvm/llvm-rebuild-trigger` to update its contents
100+
101+
5. Build your commit. Make sure you've committed the previous changes to ensure
102+
submodule updates aren't reverted. Some commands you should execute are:
103+
104+
* `./x.py build src/llvm` - test that LLVM still builds
105+
* `./x.py build src/tools/lld` - same for LLD
106+
* `./x.py build` - build the rest of rustc
107+
108+
You'll likely need to update `src/rustllvm/*.cpp` to compile with updated
109+
LLVM bindings. Note that you should use `#ifdef` and such to ensure that the
110+
bindings still compile on older LLVM versions.
111+
112+
6. Test for regressions across other platforms. LLVM often has at least one bug
113+
for non-tier-1 architectures, so it's good to do some more testing before
114+
sending this to bors! If you're low on resources you can send the PR as-is
115+
now to bors, though, and it'll get tested anyway.
116+
117+
Ideally, build LLVM and test it on a few platforms:
118+
119+
* Linux
120+
* OSX
121+
* Windows
122+
123+
and afterwards run some docker containers that CI also does:
124+
125+
* `./src/ci/docker/run.sh wasm32-unknown`
126+
* `./src/ci/docker/run.sh arm-android`
127+
* `./src/ci/docker/run.sh dist-various-1`
128+
* `./src/ci/docker/run.sh dist-various-2`
129+
* `./src/ci/docker/run.sh armhf-gnu`
130+
131+
7. Send a PR! Hopefully it's smooth sailing from here :).
132+
133+
For prior art, previous LLVM updates look like
134+
[#55835](https://github.com/rust-lang/rust/pull/55835)
135+
[#47828](https://github.com/rust-lang/rust/pull/47828)
136+
137+
### Caveats and gotchas
138+
139+
Ideally the above instructions are pretty smooth, but here's some caveats to
140+
keep in mind while going through them:
141+
142+
* LLVM bugs are hard to find, don't hesitate to ask for help! Bisection is
143+
definitely your friend here (yes LLVM takes forever to build, yet bisection is
144+
still your friend)
145+
* Updating LLDB has some Rust-specific patches currently that aren't upstream.
146+
If you have difficulty @tromey can likely help out.
147+
* If you've got general questions, @alexcrichton can help you out.
148+
* Creating branches is a privileged operation on GitHub, so you'll need someone
149+
with write access to create the branches for you most likely.

0 commit comments

Comments
 (0)