How to setup LLVM buildbots for Windows on Arm

We have moved regular LLVM buildbots to Windows 11 while some information on this page is still valid overall, please follow How to setup LLVM buildbots for Windows 11 on Arm - TCWG-public - Confluence for setting up Windows on Arm buildbots.

First off, get Windows development tools and a base system compiler with a decent performance that can build LLVM and its sub-projects.  Refer to XXX for bootstrap instructions of an LLVM-based toolchain for Windows on Arm.  The rest of this page assumes Windows development tools are installed as in How to setup Windows on Arm for llvm development.

Install buildbot version compatible with LLVM's upstream CI

Fetch pip and install buildbot-worker: refer to https://git.linaro.org/ci/dockerfiles.git/tree/tcwg-base/tcwg-llvmbot/Dockerfile.in#n39 – and adapt its procedure to Windows.  We also need pywin32, without it builds fail with an exception that they can't spawn commands.

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py C:\Users\%USERNAME%\AppData\Local\Programs\Python\Python38-32\python.exe get-pip.py C:\Users\%USERNAME%\AppData\Local\Programs\Python\Python38-32\Scripts\pip.exe install buildbot-worker pywin32 packaging

Note: Windows on Arm currently does not have an official native Arm64/Windows python release. However native python arm64 package can be installed via nuget[1]. After installing native arm64 python package via nuget buildbot-worker package is available via pip. However pywin32 package has to be installed using whl package[2].

Note: buildbot worker version: 3.4.1 and Twisted version: 22.4.0 are known to work with Arm64 native python release. In case of buildbot-worker installation failure try: pip.exe install buildbot-worker==3.4.1 Twisted==22.4.0

[1] Instructions for installing python Arm64 Windows native package: Pre-release Python Installation

[2] Instructions for installing pywin32 Arm64 Windows native package: Pre-release Python binary wheels and dependencies

MSYS2

LLVM's upstream CI needs a few unix-style utilities like rm, grep, diff etc.  Install msys2 package for x86_64 (if emulation allows) or i686.  MSYS2 recently deprecated i686 version, so it's a bit tricky to install.  Disabling signature checking in msys32/etc/pacman.conf should fix your problems:

# By default, pacman accepts packages signed by keys that its local keyring # trusts (see pacman-key and its man page), as well as unsigned packages. SigLevel = Never #SigLevel = Required DatabaseOptional LocalFileSigLevel = Optional #RemoteFileSigLevel = Required

then install diffutils and, potentially, other packages that LLVM build may require

pacman -S diffutils

Host compiler

You need a decent host compiler.  Refer to XXX for instructions on bootstrapping LLVM on Windows on Arm.  Let the force be with you.

Install your host compiler in, say, C:\Users\%USERNAME%\source\host-llvm\ .

Remove path length limit

In most cases Windows has set a maximum length for a file/directory path to a maximum value of 260 characters (Reference). This limitation can be removed by setting following registry key to 1:

Create buildbot

Open the command prompt and run 

to create buildbot.  Then edit C:\Users\%USERNAME%\source\buildbot\admin and C:\Users\%USERNAME%\source\buildbot\host similar to what https://git.linaro.org/ci/dockerfiles.git/tree/tcwg-base/tcwg-llvmbot/run.sh does:

C:\Users\%USERNAME%\source\buildbot\admin
C:\Users\%USERNAME%\source\buildbot\host

Buildbot startup script

Create C:\Users\%USERNAME%\source\buildbot.bat

C:\Users\%USERNAME%\source\buildbot.bat

MSVC_VER should be set to whatever the minimum llvm needs to build. Sometimes the version of clang we’re using defaults to that, sometimes it won’t.

If your buildbot is expected to build flang, the file above should also set the BUILTINS environment variable to the path to clang_rt.builtins-aarch64.lib, as in the following example (replace %VERSION% with the proper llvm version, e.g. 12.0.0):

If your buildbot is expected to build lldb you will also need to add the DIA SDK to the path and opt into using lldb server:

Create a Windows service for buildbot

Apparently, normal Windows applications and scripts are stopped when a user logs out or the laptop gets to lock screen.  To avoid buildbot stopping we need to create a Windows service for it, which will run in the background.  One way of doing this is to use RunAsRob utility: https://runasrob.com/ .  Follow instructions on https://runasrob.com/RunAsRobPermanentAsService.html and configure C:\Users\%USERNAME%\source\buildbot.bat to run as a service.

For the next service setup instead of RunAsRob try using open-source tool Windows Service management .

RunAsRob License

RunAsRob is free only for personal use.  We (Linaro) have purchased 10 licenses (minimal amount, costs $72 for all 10) to run on our WoA laptops.  These are 5-year non-transferrable licenses valid until 26 March 2026 within Linaro Limited.

Talk to @Maxim Kuvyrkov if you need to get a licensed copy of RunAsRob.

Fire up the buildbot!

Using Services app.

Stop/restart buildbot

To stop buildbot open "Services" app, scroll down to "RunasRob" and stop it.  Then open "Task Manager" and in "Details" tab find and kill "python.exe" process.  You can also kill "ninja.exe" and other related processes to avoid waiting for them to finish/fail.

To restart the buildbot restart "RunasRob" service.