We have been experimenting to run Windows on Arm binaries from a Linux-x64 machine using qemu-user and wine-arm64.

How it works

1. QEMU can run aarch64 binaries on x64 host: qemu-aarch64 ./program

2. Wine-arm64 is set of native aarch64 binaries that implements Windows interface, and can run windows-arm64 programs as simply as: wine ./program.exe

By combining both, we can run windows-arm64 binaries on a linux x64 machine:

qemu-aarch64 /path/to/wine-arm64/wine ./program.exe

QEMU

https://www.qemu.org/ is an open source machine emulator and virtualizer.

QEMU provides one binary per architecture, and comes in two variants under Linux: system and user modes.

System mode (qemu-system-aarch64) emulates a whole “virtual machine”, which can be accelerated with virtualization by using https://www.linux-kvm.org/page/Main_Page.

User mode provides emulation for a single program (and not a whole machine). In this case, QEMU intercepts system calls, which is the interface between user space and kernel space. It translates those calls from host architecture to guest architecture, talking directly to the Linux kernel. This is why user mode works only on Linux, and not on other OS.

Debian provides qemu-user-static which contains:

• qemu-user binaries statically compiled

• binfmt support binfmt_misc: In short, when you try to execute an elf-aarch64 binary, it calls automatically qemu-user-aarch64 to run it.

Wine

WineHQ - Run Windows applications on Linux, BSD, Solaris and macOS stands for “Wine is not an emulator”. Wine (software)

It is an implementation of Windows API interface, and a https://en.wikipedia.org/wiki/Portable_Executable loader to run windows programs. It can run unmodified windows binaries.

Arm64 has been supported since Windows on Arm was released: https://wiki.winehq.org/ARM64

Documentation for this platform can be out of date, or lacking, so be careful.

Setup

Instructions are given for Debian:

• Install docker: https://docs.docker.com/engine/install/debian/

• Add your user to docker group: sudo usermod -aG docker $USER

• Logout your session and reconnect (so your user groups are changed)

Check docker can execute a container by running:

wine-arm64 Docker image

We created a docker x64 container image with wine-arm64 prebuilt and embedded qemu-user-static (so binfmt support is not required). It can easily be used with any CI system interfacing with containers: GitLab, GitHub, etc.

Image is published as linaro/wine-arm64

Details can be found here: Unified docker image

GitLab and GitHub pipelines examples can be found here: https://gitlab.com/Linaro/windowsonarm/woa-linux-examples

Limitations

For now, it’s not yet possible to run arm64 version of cl.exe using wine-arm64, so it’s better to cross compile from windows-x64/Linux (using cl, clang, or llvm-mingw).

We target to run unit tests and programs, instead of full native compilation process for now.

Warning: Wine can have bugs. Either on the Windows interface, or specific arm64 bugs. In case you encounter a crash, don’t assume your program is necessarily faulty, and if you can, investigate on a Windows on Arm machine.

Experiments

We created a repository to collect binaries than can be use to evaluate wine correctness and qemu/wine performance: https://gitlab.com/Linaro/windowsonarm/woa-linux-test-binaries

OSQuery

Unittests from OSQuery. Some failures (9 tests on 70) found:

• filesystem issues:

  • https://gitlab.com/Linaro/windowsonarm/woa-linux-test-binaries/-/blob/master/fails/osquery/osquery/config/tests/osquery_config_tests-test.exe

  • https://gitlab.com/Linaro/windowsonarm/woa-linux-test-binaries/-/blob/master/fails/osquery/osquery/tables/events/tests/osquery_tables_events_tests_windowseventstests-test.exe

• extension socket not implemented: https://gitlab.com/Linaro/windowsonarm/woa-linux-test-binaries/-/blob/master/fails/osquery/osquery/extensions/tests/osquery_extensions_extensiontests-test.exe

• “various” windows issues: https://gitlab.com/Linaro/windowsonarm/woa-linux-test-binaries/-/blob/master/fails/osquery/osquery/tables/system/tests/osquery_tables_system_tests_systemtablestests-test.exe

• issue with registry: https://gitlab.com/Linaro/windowsonarm/woa-linux-test-binaries/-/blob/master/fails/osquery/osquery/tables/system/tests/osquery_tables_system_windows_tests-test.exe

• issue with CoCreateInstance
  https://gitlab.com/Linaro/windowsonarm/woa-linux-test-binaries/-/blob/master/fails/osquery/osquery/utils/conversions/osquery_utils_conversions_conversionstests-test.exe

• issue with pid:
  https://gitlab.com/Linaro/windowsonarm/woa-linux-test-binaries/-/blob/master/fails/osquery/osquery/utils/pidfile/osquery_utils_pidfile_tests-test.exe

OSQuery is using advanced for API on Windows, so it’s not a total surprise to find those issues.

Numpy-benchmarks

Running benchmarks from numpy https://github.com/numpy/numpy/tree/main/benchmarks/benchmarks .

Installing numpy with pip does not work under wine-arm64, as cl.exe is not available to compile native code.

All benchmarks work under wine-arm64

Performance:

• volterra: windows-arm64: 0.175s

• volterra: linux-arm64-vm + wine: 0.850s

• intel i7-10700k: linux-x64 + qemu-user + wine: 1.4s

• intel i7-10700k: linux-x64 + qemu-system + windows-arm64: 2.9-3.5s (variations)

Most of the tests are short, and bound by python startup.

Running full windows-arm64 vm is unresponsive (background services can randomly make the cpu busy) and slower than running qemu-user + wine. In more, booting the VM itself takes 4 minutes.