How to setup Windows 10 on Arm for llvm development

This describes how to setup a Windows 10 on Arm laptop to build llvm. Windows 10 on Arm supports emulation of 32-bit x86 programs. This allows us to use those 32-bit x86 version on Windows 10 on Arm but support for x64 versions of same executables is not available in Windows 10.

We have moved regular LLVM development to Windows 11 while some information on this page is still valid overall, please follow https://linaro.atlassian.net/l/cp/6SWAH1pt page in order to setup your Windows 11 machines for LLVM development.

Adding to the PATH

For all the steps below you’ll need to put things on the PATH, if an installer doesn’t do it for you. To do that open the start menu, type “environment” and open the link to “System Properties”. There click “Environment Variables” to see a GUI for editing them.

Note that we configure RunAsRob (the service that runs buildbot) to run as “tcwg”, not “System” user, so environment changes will also apply to the buildbot.

Note: Just like in Unix, adding to the path doesn’t refresh active terminals, and there is no way to refresh an active terminal. So you need to open a fresh terminal after changing environment variables.

After following this guide your PATH should have these extra entries:

  • C:\Users\tcwg\AppData\Local\Programs\Python\Python39-32\Scripts\

  • C:\Users\tcwg\AppData\Local\Programs\Python\Python39-32\

  • C:\Program Files (x86)\Git\usr\bin

    • This is for mingw utilities. Git adds itself to System PATH

  • C:\Users\tcwg\source\ninja

  • C:\Program Files\LLVM\bin

  • C:\Program Files (x86)\CMake\bin

For building release packages we also install 7-Zip https://www.7-zip.org/download.html and NSIS https://nsis.sourceforge.io/Download

  • C:\Program Files (x86)\7-Zip

  • C:\Program Files (x86)\NSIS

Install Visual Studio Build Tools

Note: Microsoft Visual Studio 2022 has been released but the installer is a 64-bit application requiring ARM64 emulation support. Arm64 emulation is generally available for Windows 11 but only a preview build is available for windows 10.

Build tools is the command line portion of visual studio, download it from https://visualstudio.microsoft.com/downloads/ . (“Tools for Visual Studio 2019” → “Build Tools for Visual Studio 2019”)

The Visual Studio installer has many options but generally, you want various things to do with C++ desktop development for ARM64. (we do not need ARM components, ARM means Arm 32 bit in this context)

If you have build issues later, come back to the installer and add anything that seems relevant. Here’s the list of what is minimally needed, check this against “Individual Components” in your “Installation Details” panel:

  • MSVC v142 - VS 2019 C++ x64/x86 build tools (latest)

    • This is needed to make vsdevcmd.bat correctly setup LIB/LIBPATH/INCLUDE variables for x86->arm64 cross-compilation. All the lib/header files are actually present, but vsdevcmd.bat doesn’t add them without x86-hosted tools.

    • Installing this may become optional in the future.

  • MSVC v142 - VS 2019 C++ ARM64 build tools (latest)

    • Without these vsdevcmd.bat doesn’t setup x86->arm64 cross-compilation.

  • C++ ATL for latest v142 build tools (ARM64)

    • LLVM needs ATL libraries for processing debug info.

  • Windows 10 SDK

    • Versions 18362 and 19041 are known to work.

    • Version 20348 is known to cause LLVM build failure.

      • error: use of undeclared identifier '__umulh'

    • This is required for correct cross-compilation setup. (not sure if SDK libraries are actually used)

Note: there’s probably a choice in the installer for where to install to. If you do that modify any instructions as needed. The default that I got was “C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\Tools\”.

(worst case, you can always search C:\ for a file, it isn’t as slow as you’d expect)

Install MSVC Redist libraries

Search filesystem for vc_redist.arm64.exe, and you’ll most likely find it in C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Redist\MSVC\v142\ .

Install these libraries to get msvcp140.dll, vcruntime140.dll, concrt140.dll and other libraries. Without these DLLs MSVC-built application will not run; in LLVM build this manifests itself as llvm-tblgen.exe failing.

For reasons unknown, these libraries are not installed by default as part of Windows or Visual Studio.

Install latest llvm for Windows on Arm

Go to https://github.com/llvm/llvm-project/releases/tag/llvmorg-14.0.6 and download the Windows on Arm (“woa64”) installer. Run it and if it asks to add llvm to the path say yes. If it doesn’t or you forget that you can always add the install directory to PATH yourself.

Install CMake

Recent versions of VS ship cmake as x86_64 binary (in VS circa-2020.08 cmake was x86_32 binary). Instead install the i386 build from https://cmake.org/files/ and add that to the PATH.

LLVM 12 or 13, with CMake 3.17 is known to work.

LLVM 13 with CMake 3.21 or 3.22 is known not to work.

Install Python

If you run “python” it takes you to the Windows Store which allegedly will give you a (native?) copy of Python3. As per usual with the Windows Store, it didn’t do anything when I clicked install.

So instead, you can go to Python.org and get a 32 bit x86 build of the latest Python3. Remember to:

  • Tell the installer to add python variables to environment (so that cmake can find python3).

Note: lldb’s Python integration requires a native Python install, see the section below about lldb dependencies.

Install Git

Go to https://git-scm.com/download/win and get the latest 32 bit x86 installer. There is likely a copy of git the VS Build Tools install, but we reccomend installing a separate copy so that you also get the tools git for Windows is packaged with. These tools are used for testing llvm.

llvm-lit.py: <...>\llvm-project\llvm\utils\lit\lit\llvm\config.py:46: note: using lit tools: C:\Program Files (x86)\Git\usr\bin

(you can get these tools by installing MSYS2 instead, but git for Windows is based on that so the result is the same)

Build Ninja

VS Build Tools does come with a ninja but the default one doesn’t run on WoA. You should build from source (https://github.com/ninja-build/ninja ) using the cmake build method.

(https://github.com/ninja-build/ninja/releases does provide prebuilt releases but at this time the Windows variant is x86_64 only)

Check out LLVM

The git for Windows install will default to converting line endings to windows style. This applies to any file git thinks is ASCII, which includes some archive files used for llvm tests. As stated in https://llvm.org/docs/GettingStarted.html#checkout-llvm-from-git , use the following to override this behaviour:

git clone --config core.autocrlf=false https://github.com/llvm/llvm-project.git

Testing the Install

First, open a plain terminal “Command Prompt” (ignore the cross prompts shortcuts you might find in the start menu).

Then run VsDevCmd.bat to setup the environment.

"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\Tools\VsDevCmd.bat" -host_arch=x86 -arch=arm64 <...> "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\Tools\VsDevCmd.bat" -test <...> set PATH=C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\DIA SDK\bin\arm64;%PATH%

If the second test command fails, your build probably won’t work. If figuring out why it failed is difficult, go ahead and do a build anyway. The compiler’s errors will probably be more informative.

Note: We setup the Visual Studio env first because it adds some copies of cmake and ninja that we don’t want. By doing it first, then checking cmake and ninja, we know if our preferred versions are being used.

Note: We have to add the arm64 DIA (debug information access) DLL onto the path otherwise it will find the x86_32 version in BuildTools\Common7\IDE.

Now check that cmake and ninja can run at all.

Doing a Build

In the same command prompt as the previous step, set your compiler(s) to be the clang-cl.exe we installed earlier.

Then make a folder next to your llvm checkout and from that folder:

Some specifics:

  • We need the “-fms-compatibility-version" flag to have clang-cl pretend to be a newer MSVC, especially if that clang-cl is a prior release to the clang you’re compiling. If this value is too low than cmake will tell you what it should be, just set it to that.

  • Known issue with cmake 3.21.1, it builds all try_compile/try_run as debug even if you select release mode. This is why we set “-DCMAKE_TRY_COMPILE_CONFIGURATION=Release”. Not doing so causes a try_run to fail to get error message strings, so lit defaults to Linux strings and many tests will fail.

  • We set LLVM_DEFAULT_TARGET_TRIPLE manually because the prompt we use is an x86 32 bit host prompt. There is no arm64 to arm64 prompt, so cmake detects the host/default target triple “correctly” but it’s not what we really want.

Then build as usual with ninja.

Here is where you might get errors about includes and linker directories and so on. Best way to solve this is to go back to the Visual Studio Installer and add anything that looks related.

You will find things online about adding certain dirs to your path but be careful because you could just be adding x86 libs. Installing ARM64 specific variants is almost always what you want.

If you’re doing a debug build and you see linker errors about missing libs, see if the file names end with “d”. E.g. “foo.lib” would be “food.lib” for a debug build. To solve this search for those files and add their location to PATH. They don’t appear to be automatically added by the installers.

Excluding the test folder from Windows Defender

Windows Defender likes to scan new files, including those that tests create. This changes their last accessed times and there’s one test known to fail because of this, “LLVM :: ThinLTO/X86/cache.ll”. To exclude the folder follow https://support.microsoft.com/en-us/windows/add-an-exclusion-to-windows-security-811816c0-4dfd-af4a-47e4-c301afe13b26 and add “<llvm-build-dir>/test”. (you shouldn’t need to restart defender or the machine, it takes effect automatically)

Install LLDB API and testsuite dependencies

  • Install GNUWin32 for make and coreutils. LLDB tests depend on make and some of the coreutils for running API tests. Install it using standard install as 32bit x86 emulated executable and add install location to PATH.

  • Install native Python. LLDB API requires native python library. Here is a link to a unofficial build of native python installer for WoA.

  • Install SWIG 3 or later. LLDB requires swig to generate Python bindings. Install SWIG via x86 emulated installer and add to PATH.