Skip to content

[llvm][docs] Refresh Arm builtins cross compile guide #150966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 31, 2025
Merged

[llvm][docs] Refresh Arm builtins cross compile guide #150966

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
50be7ce
[llvm][docs] Refresh Arm builtins cross compile guide
DavidSpickett Jul 28, 2025
08c4652
missing word
DavidSpickett Jul 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
227 changes: 112 additions & 115 deletions llvm/docs/HowToCrossCompileBuiltinsOnArm.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,117 +14,113 @@ targets are welcome.

The instructions in this document depend on libraries and programs external to
LLVM, there are many ways to install and configure these dependencies so you
may need to adapt the instructions here to fit your own local situation.
may need to adapt the instructions here to fit your own situation.

Prerequisites
=============

In this use case we'll be using cmake on a Debian-based Linux system,
cross-compiling from an x86_64 host to a hard-float Armv7-A target. We'll be
In this use case we will be using cmake on a Debian-based Linux system,
cross-compiling from an x86_64 host to a hard-float Armv7-A target. We will be
using as many of the LLVM tools as we can, but it is possible to use GNU
equivalents.

* ``A build of LLVM/clang for the llvm-tools and llvm-config``
* ``A clang executable with support for the ARM target``
* ``compiler-rt sources``
* ``The qemu-arm user mode emulator``
* ``An arm-linux-gnueabihf sysroot``
You will need:
* A build of LLVM for the llvm-tools and ``llvm-config``.
* A clang executable with support for the ``ARM`` target.
* compiler-rt sources.
* The ``qemu-arm`` user mode emulator.
* An ``arm-linux-gnueabihf`` sysroot.

In this example we will be using ninja.
In this example we will be using ``ninja`` as the build tool.

See https://compiler-rt.llvm.org/ for more information about the dependencies
See https://compiler-rt.llvm.org/ for information about the dependencies
on clang and LLVM.

See https://llvm.org/docs/GettingStarted.html for information about obtaining
the source for LLVM and compiler-rt. Note that the getting started guide
places compiler-rt in the projects subdirectory, but this is not essential and
if you are using the BaremetalARM.cmake cache for v6-M, v7-M and v7-EM then
compiler-rt must be placed in the runtimes directory.
the source for LLVM and compiler-rt.

``qemu-arm`` should be available as a package for your Linux distribution.

The most complicated of the prerequisites to satisfy is the arm-linux-gnueabihf
The most complicated of the prerequisites to satisfy is the ``arm-linux-gnueabihf``
sysroot. In theory it is possible to use the Linux distributions multiarch
support to fulfill the dependencies for building but unfortunately due to
/usr/local/include being added some host includes are selected. The easiest way
to supply a sysroot is to download the arm-linux-gnueabihf toolchain. This can
be found at:
* https://developer.arm.com/open-source/gnu-toolchain/gnu-a/downloads for gcc 8 and above
* https://releases.linaro.org/components/toolchain/binaries/ for gcc 4.9 to 7.3
``/usr/local/include`` being added some host includes are selected.

The easiest way to supply a sysroot is to download an ``arm-linux-gnueabihf``
toolchain from https://developer.arm.com/open-source/gnu-toolchain/gnu-a/downloads.

Building compiler-rt builtins for Arm
=====================================

We will be doing a standalone build of compiler-rt using the following cmake
options.

* ``path/to/compiler-rt``
* ``-G Ninja``
* ``-DCMAKE_AR=/path/to/llvm-ar``
* ``-DCMAKE_ASM_COMPILER_TARGET="arm-linux-gnueabihf"``
* ``-DCMAKE_ASM_FLAGS="build-c-flags"``
* ``-DCMAKE_C_COMPILER=/path/to/clang``
* ``-DCMAKE_C_COMPILER_TARGET="arm-linux-gnueabihf"``
* ``-DCMAKE_C_FLAGS="build-c-flags"``
* ``-DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld"``
* ``-DCMAKE_NM=/path/to/llvm-nm``
* ``-DCMAKE_RANLIB=/path/to/llvm-ranlib``
* ``-DCOMPILER_RT_BUILD_BUILTINS=ON``
* ``-DCOMPILER_RT_BUILD_LIBFUZZER=OFF``
* ``-DCOMPILER_RT_BUILD_MEMPROF=OFF``
* ``-DCOMPILER_RT_BUILD_PROFILE=OFF``
* ``-DCOMPILER_RT_BUILD_SANITIZERS=OFF``
* ``-DCOMPILER_RT_BUILD_XRAY=OFF``
* ``-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON``
* ``-DLLVM_CONFIG_PATH=/path/to/llvm-config``
options::

cmake path/to/compiler-rt \
-G Ninja \
-DCMAKE_AR=/path/to/llvm-ar \
-DCMAKE_ASM_COMPILER_TARGET="arm-linux-gnueabihf" \
-DCMAKE_ASM_FLAGS="build-c-flags" \
-DCMAKE_C_COMPILER=/path/to/clang \
-DCMAKE_C_COMPILER_TARGET="arm-linux-gnueabihf" \
-DCMAKE_C_FLAGS="build-c-flags" \
-DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" \
-DCMAKE_NM=/path/to/llvm-nm \
-DCMAKE_RANLIB=/path/to/llvm-ranlib \
-DCOMPILER_RT_BUILD_BUILTINS=ON \
-DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
-DCOMPILER_RT_BUILD_MEMPROF=OFF \
-DCOMPILER_RT_BUILD_PROFILE=OFF \
-DCOMPILER_RT_BUILD_SANITIZERS=OFF \
-DCOMPILER_RT_BUILD_XRAY=OFF \
-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON \
-DLLVM_CONFIG_PATH=/path/to/llvm-config

The ``build-c-flags`` need to be sufficient to pass the C-make compiler check,
compile compiler-rt, and if you are running the tests, compile and link the
tests. When cross-compiling with clang we will need to pass sufficient
information to generate code for the Arm architecture we are targeting. We will
need to select the Arm target, select the Armv7-A architecture and choose
between using Arm or Thumb.
instructions. For example:
information to generate code for the Arm architecture we are targeting.

* ``--target=arm-linux-gnueabihf``
* ``-march=armv7a``
* ``-mthumb``
We will need to select:
* The Arm target and Armv7-A architecture with ``--target=arm-linux-gnueabihf -march=armv7a``.
* Whether to generate Arm (the default) or Thumb instructions (``-mthumb``).

When using a GCC arm-linux-gnueabihf toolchain the following flags are
When using a GCC ``arm-linux-gnueabihf`` toolchain the following flags are
needed to pick up the includes and libraries:

* ``--gcc-toolchain=/path/to/dir/toolchain``
* ``--sysroot=/path/to/toolchain/arm-linux-gnueabihf/libc``
* ``--gcc-toolchain=/path/to/dir/toolchain``
* ``--sysroot=/path/to/toolchain/arm-linux-gnueabihf/libc``

In this example we will be adding all of the command line options to both
``CMAKE_C_FLAGS`` and ``CMAKE_ASM_FLAGS``. There are cmake flags to pass some of
these options individually which can be used to simplify the ``build-c-flags``:
these options individually which can be used to simplify the ``build-c-flags``::

* ``-DCMAKE_C_COMPILER_TARGET="arm-linux-gnueabihf"``
* ``-DCMAKE_ASM_COMPILER_TARGET="arm-linux-gnueabihf"``
* ``-DCMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN=/path/to/dir/toolchain``
* ``-DCMAKE_SYSROOT=/path/to/dir/toolchain/arm-linux-gnueabihf/libc``
-DCMAKE_C_COMPILER_TARGET="arm-linux-gnueabihf"
-DCMAKE_ASM_COMPILER_TARGET="arm-linux-gnueabihf"
-DCMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN=/path/to/dir/toolchain
-DCMAKE_SYSROOT=/path/to/dir/toolchain/arm-linux-gnueabihf/libc

Once cmake has completed the builtins can be built with ``ninja builtins``

Testing compiler-rt builtins using qemu-arm
===========================================

To test the builtins library we need to add a few more cmake flags to enable
testing and set up the compiler and flags for test case. We must also tell
cmake that we wish to run the tests on ``qemu-arm``.
cmake that we wish to run the tests on ``qemu-arm``::

* ``-DCOMPILER_RT_EMULATOR="qemu-arm -L /path/to/armhf/sysroot``
* ``-DCOMPILER_RT_INCLUDE_TESTS=ON``
* ``-DCOMPILER_RT_TEST_COMPILER="/path/to/clang"``
* ``-DCOMPILER_RT_TEST_COMPILER_CFLAGS="test-c-flags"``
-DCOMPILER_RT_EMULATOR="qemu-arm -L /path/to/armhf/sysroot"
-DCOMPILER_RT_INCLUDE_TESTS=ON
-DCOMPILER_RT_TEST_COMPILER="/path/to/clang"
-DCOMPILER_RT_TEST_COMPILER_CFLAGS="test-c-flags"

The ``/path/to/armhf/sysroot`` should be the same as the one passed to
``--sysroot`` in the "build-c-flags".
``--sysroot`` in the ``build-c-flags``.

The "test-c-flags" need to include the target, architecture, gcc-toolchain,
sysroot and arm/thumb state. The additional cmake defines such as
The ``test-c-flags`` need to include the target, architecture, gcc-toolchain,
sysroot and Arm/Thumb state. The additional cmake defines such as
``CMAKE_C_COMPILER_EXTERNAL_TOOLCHAIN`` do not apply when building the tests. If
you have put all of these in "build-c-flags" then these can be repeated. If you
wish to use lld to link the tests then add ``"-fuse-ld=lld``.
you have put all of these in ``build-c-flags`` then these can be repeated. If you
wish to use lld to link the tests then add ``-fuse-ld=lld``.

Once cmake has completed the tests can be built and run using
``ninja check-builtins``
Expand All @@ -142,19 +138,21 @@ This stage can often fail at link time if the ``--sysroot=`` and
``CMAKE_C_FLAGS`` and ``CMAKE_C_COMPILER_TARGET`` flags.

It can be useful to build a simple example outside of cmake with your toolchain
to make sure it is working. For example: ``clang --target=arm-linux-gnueabi -march=armv7a --gcc-toolchain=/path/to/gcc-toolchain --sysroot=/path/to/gcc-toolchain/arm-linux-gnueabihf/libc helloworld.c``
to make sure it is working. For example::

clang --target=arm-linux-gnueabi -march=armv7a --gcc-toolchain=/path/to/gcc-toolchain --sysroot=/path/to/gcc-toolchain/arm-linux-gnueabihf/libc helloworld.c

Clang uses the host header files
--------------------------------
On debian based systems it is possible to install multiarch support for
arm-linux-gnueabi and arm-linux-gnueabihf. In many cases clang can successfully
``arm-linux-gnueabi`` and ``arm-linux-gnueabihf``. In many cases clang can successfully
use this multiarch support when ``--gcc-toolchain=`` and ``--sysroot=`` are not supplied.
Unfortunately clang adds ``/usr/local/include`` before
``/usr/include/arm-linux-gnueabihf`` leading to errors when compiling the hosts
header files.

The multiarch support is not sufficient to build the builtins you will need to
use a separate arm-linux-gnueabihf toolchain.
use a separate ``arm-linux-gnueabihf`` toolchain.

No target passed to clang
-------------------------
Expand All @@ -164,12 +162,13 @@ as ``error: unknown directive .syntax unified``.

You can check the clang invocation in the error message to see if there is no
``--target`` or if it is set incorrectly. The cause is usually
``CMAKE_ASM_FLAGS`` not containing ``--target`` or ``CMAKE_ASM_COMPILER_TARGET`` not being present.
``CMAKE_ASM_FLAGS`` not containing ``--target`` or ``CMAKE_ASM_COMPILER_TARGET``
not being present.

Arm architecture not given
--------------------------
The ``--target=arm-linux-gnueabihf`` will default to arm architecture v4t which
cannot assemble the barrier instructions used in the synch_and_fetch source
The ``--target=arm-linux-gnueabihf`` will default to Arm architecture v4t which
cannot assemble the barrier instructions used in the ``synch_and_fetch`` source
files.

The cause is usually a missing ``-march=armv7a`` from the ``CMAKE_ASM_FLAGS``.
Expand Down Expand Up @@ -202,7 +201,7 @@ may need extra c-flags such as ``-mfloat-abi=softfp`` for use of floating-point
instructions, and ``-mfloat-abi=soft -mfpu=none`` for software floating-point
emulation.

You will need to use an arm-linux-gnueabi GNU toolchain for soft-float.
You will need to use an ``arm-linux-gnueabi`` GNU toolchain for soft-float.

AArch64 Target
--------------
Expand All @@ -220,8 +219,12 @@ Armv6-m, Armv7-m and Armv7E-M targets
To build and test the libraries using a similar method to Armv7-A is possible
but more difficult. The main problems are:

* There isn't a ``qemu-arm`` user-mode emulator for bare-metal systems. The ``qemu-system-arm`` can be used but this is significantly more difficult to setup.
* The targets to compile compiler-rt have the suffix -none-eabi. This uses the BareMetal driver in clang and by default won't find the libraries needed to pass the cmake compiler check.
* There is not a ``qemu-arm`` user-mode emulator for bare-metal systems.
``qemu-system-arm`` can be used but this is significantly more difficult
to setup.
* The targets to compile compiler-rt have the suffix ``-none-eabi``. This uses
the BareMetal driver in clang and by default will not find the libraries
needed to pass the cmake compiler check.

As the Armv6-M, Armv7-M and Armv7E-M builds of compiler-rt only use instructions
that are supported on Armv7-A we can still get most of the value of running the
Expand All @@ -233,32 +236,30 @@ builtins use instructions that are supported on Armv7-A but not Armv6-M,
Armv7-M and Armv7E-M.

To get the cmake compile test to pass you will need to pass the libraries
needed to successfully link the cmake test via ``CMAKE_CFLAGS``. It is
strongly recommended that you use version 3.6 or above of cmake so you can use
``CMAKE_TRY_COMPILE_TARGET=STATIC_LIBRARY`` to skip the link step.

* ``-DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY``
* ``-DCOMPILER_RT_OS_DIR="baremetal"``
* ``-DCOMPILER_RT_BUILD_BUILTINS=ON``
* ``-DCOMPILER_RT_BUILD_SANITIZERS=OFF``
* ``-DCOMPILER_RT_BUILD_XRAY=OFF``
* ``-DCOMPILER_RT_BUILD_LIBFUZZER=OFF``
* ``-DCOMPILER_RT_BUILD_PROFILE=OFF``
* ``-DCMAKE_C_COMPILER=${host_install_dir}/bin/clang``
* ``-DCMAKE_C_COMPILER_TARGET="your *-none-eabi target"``
* ``-DCMAKE_ASM_COMPILER_TARGET="your *-none-eabi target"``
* ``-DCMAKE_AR=/path/to/llvm-ar``
* ``-DCMAKE_NM=/path/to/llvm-nm``
* ``-DCMAKE_RANLIB=/path/to/llvm-ranlib``
* ``-DCOMPILER_RT_BAREMETAL_BUILD=ON``
* ``-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON``
* ``-DLLVM_CONFIG_PATH=/path/to/llvm-config``
* ``-DCMAKE_C_FLAGS="build-c-flags"``
* ``-DCMAKE_ASM_FLAGS="build-c-flags"``
* ``-DCOMPILER_RT_EMULATOR="qemu-arm -L /path/to/armv7-A/sysroot"``
* ``-DCOMPILER_RT_INCLUDE_TESTS=ON``
* ``-DCOMPILER_RT_TEST_COMPILER="/path/to/clang"``
* ``-DCOMPILER_RT_TEST_COMPILER_CFLAGS="test-c-flags"``
needed to successfully link the cmake test via ``CMAKE_CFLAGS``::

-DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY \
-DCOMPILER_RT_OS_DIR="baremetal" \
-DCOMPILER_RT_BUILD_BUILTINS=ON \
-DCOMPILER_RT_BUILD_SANITIZERS=OFF \
-DCOMPILER_RT_BUILD_XRAY=OFF \
-DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
-DCOMPILER_RT_BUILD_PROFILE=OFF \
-DCMAKE_C_COMPILER=${host_install_dir}/bin/clang \
-DCMAKE_C_COMPILER_TARGET="your *-none-eabi target" \
-DCMAKE_ASM_COMPILER_TARGET="your *-none-eabi target" \
-DCMAKE_AR=/path/to/llvm-ar \
-DCMAKE_NM=/path/to/llvm-nm \
-DCMAKE_RANLIB=/path/to/llvm-ranlib \
-DCOMPILER_RT_BAREMETAL_BUILD=ON \
-DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON \
-DLLVM_CONFIG_PATH=/path/to/llvm-config \
-DCMAKE_C_FLAGS="build-c-flags" \
-DCMAKE_ASM_FLAGS="build-c-flags" \
-DCOMPILER_RT_EMULATOR="qemu-arm -L /path/to/armv7-A/sysroot" \
-DCOMPILER_RT_INCLUDE_TESTS=ON \
-DCOMPILER_RT_TEST_COMPILER="/path/to/clang" \
-DCOMPILER_RT_TEST_COMPILER_CFLAGS="test-c-flags"

The Armv6-M builtins will use the soft-float ABI. When compiling the tests for
Armv7-A we must include ``"-mthumb -mfloat-abi=soft -mfpu=none"`` in the
Expand All @@ -267,25 +268,21 @@ test-c-flags. We must use an Armv7-A soft-float abi sysroot for ``qemu-arm``.
Depending on the linker used for the test cases you may encounter BuildAttribute
mismatches between the M-profile objects from compiler-rt and the A-profile
objects from the test. The lld linker does not check the profile
BuildAttribute so it can be used to link the tests by adding -fuse-ld=lld to the
BuildAttribute so it can be used to link the tests by adding ``-fuse-ld=lld`` to the
``COMPILER_RT_TEST_COMPILER_CFLAGS``.

Alternative using a cmake cache
-------------------------------
If you wish to build, but not test compiler-rt for Armv6-M, Armv7-M or Armv7E-M
the easiest way is to use the BaremetalARM.cmake recipe in clang/cmake/caches.

You will need a bare metal sysroot such as that provided by the GNU ARM
Embedded toolchain.

The libraries can be built with the cmake options:
the easiest way is to use the ``BaremetalARM.cmake`` recipe in ``clang/cmake/caches``.

* ``-DBAREMETAL_ARMV6M_SYSROOT=/path/to/bare/metal/toolchain/arm-none-eabi``
* ``-DBAREMETAL_ARMV7M_SYSROOT=/path/to/bare/metal/toolchain/arm-none-eabi``
* ``-DBAREMETAL_ARMV7EM_SYSROOT=/path/to/bare/metal/toolchain/arm-none-eabi``
* ``-C /path/to/llvm/source/tools/clang/cmake/caches/BaremetalARM.cmake``
* ``/path/to/llvm``
You will need a bare metal sysroot such as that provided by the GNU ARM Embedded
toolchain.

**Note** that for the recipe to work the compiler-rt source must be checked out
into the directory llvm/runtimes. You will also need clang and lld checked out.
The libraries can be built with the cmake options::

-DBAREMETAL_ARMV6M_SYSROOT=/path/to/bare/metal/toolchain/arm-none-eabi \
-DBAREMETAL_ARMV7M_SYSROOT=/path/to/bare/metal/toolchain/arm-none-eabi \
-DBAREMETAL_ARMV7EM_SYSROOT=/path/to/bare/metal/toolchain/arm-none-eabi \
-C /path/to/llvm/source/tools/clang/cmake/caches/BaremetalARM.cmake \
/path/to/llvm
Loading