/
How to setup WoA VMs on a KVM host

How to setup WoA VMs on a KVM host

KVM host setup: Windows 11 ARM64 buildbot VMs

Target: Ubuntu 26.04 LTS Server, aarch64, bare metal. Result: 4 Windows 11 ARM64 VMs on a bridged LAN, fixed VNC ports, reachable from a macOS/Linux workstation.

0. Prerequisites — verify before starting

sudo apt update sudo apt install -y libvirt-clients virt-host-validate qemu

virt-host-validate must pass hardware virtualization. There is no kvm-ok on ARM — the aarch64 check is EL2 support, not VT-x/AMD-V flags. If this fails, stop: nested virt on a cloud aarch64 instance often lacks EL2 passthrough.

Have ready:

  • Windows 11 ARM64 ISO from VLSC/MSDN. Verify sha256sum against Microsoft's published hash — not a hash provided by a third party, and never one derived from the same media you are copying.

  • Host sized for the pool. This build: 64 vCPU / 192 GB RAM / 1 TB disk across 4 VMs.

ISO integrity check (do this every time)

sudo mkdir -p /mnt/winiso sudo mount -o loop,ro /path/to/windows.ISO /mnt/winiso ls -la /mnt/winiso/sources/install.wim ls -la /path/to/windows.ISO sudo umount /mnt/winiso

install.wim must be smaller than the ISO file containing it. A truncated image keeps intact UDF metadata describing the full-size install.wim, so it mounts fine, passes file, and boots far enough to show a boot prompt — then fails with timeouts or BCD errors deep into the process. This exact failure cost a full day on the first build. Never dd install media off a USB stick that has thrown Medium Error in dmesg; get a fresh copy from the original source.

1. Packages

sudo apt install -y \ qemu-system-arm qemu-efi-aarch64 \ libvirt-daemon-system libvirt-clients virtinst bridge-utils \ swtpm swtpm-tools

  • qemu-efi-aarch64 — mandatory. aarch64 has no legacy BIOS; guests boot via UEFI (AAVMF).

  • swtpm — mandatory for Windows 11 (TPM 2.0 presence check).

sudo usermod -aG libvirt,kvm $USER newgrp libvirt sudo systemctl enable --now libvirtd

2. Networking — bond + bridge

Skip if you have only one network card.

Two NICs on the same subnet with separate IPs is not redundancy — no failover, and it invites ARP flux. Bond them into one interface with one IP, then bridge on top.

Do this from console/IPMI, never over SSH. A bonding misconfiguration drops the link entirely.

/etc/netplan/01-bond-bridge.yaml (substitute your interface names, address, gateway):

network: version: 2 ethernets: eno1: dhcp4: no eno2: dhcp4: no bonds: bond0: interfaces: [eno1, eno2] parameters: mode: active-backup primary: eno1 mii-monitor-interval: 100 bridges: br0: interfaces: [bond0] addresses: [192.168.78.134/24] routes: - to: default via: 192.168.78.1 nameservers: addresses: [192.168.78.1] parameters: stp: true forward-delay: 4
sudo netplan apply cat /proc/net/bonding/bond0 # confirm one active slave, one backup, MII status up ip a show br0

active-backup needs no switch-side configuration. Use LACP only if the switch ports are already configured for it.

3. Storage layout

Keep shared install media outside any libvirt storage pool. Anything inside the pool directory is reachable by virsh storage operations, and virsh undefine --remove-all-storage deletes every disk source attached to a domain — including read-only CD-ROMs shared across VMs.

sudo mkdir -p /var/lib/libvirt/isos # NOT a pool — plain directory sudo mkdir -p /var/lib/libvirt/images sudo virsh pool-define-as vmpool dir --target /var/lib/libvirt/images sudo virsh pool-build vmpool sudo virsh pool-start vmpool sudo virsh pool-autostart vmpool virsh pool-info vmpool

Cross-check pool-info capacity against df -h on the target path. If they don’t match, the pool is not pointed where you think.

Place both ISOs in /var/lib/libvirt/isos/:

sudo wget -O /var/lib/libvirt/isos/virtio-win.iso \ https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso # plus your verified Windows ARM64 ISO

Note: virtio-fs (viofs) has no ARM64 driver as of 0.1.266 — do not plan on virtio-fs shared folders for these guests. Use network shares.

4. Create the VMs

Four settings below are non-obvious and each one caused a distinct failure on the first build. Do not drop them:

Setting

Why

Setting

Why

--video model=ramfb

virtio-gpu-pci gives Windows Setup no usable framebuffer. Symptom: TianoCore splash, CPU spinning, no output, looks frozen.

model=tpm-tis

tpm-crb is x86 (pc/q35) only. QEMU rejects it on the aarch64 virt machine.

Disk and Windows ISO on bus=scsi

One controller means one driver (vioscsi) at install time. Mixing virtio-blk disk with virtio-scsi CDs needs two drivers and yields INACCESSIBLE_BOOT_DEVICE if the boot-critical one doesn't register.

virtio-win ISO on bus=usb

WinPE has no virtio driver. If the driver ISO sits on virtio-scsi, WinPE cannot see the disk or the ISO holding the driver for it. USB is natively supported.

Template — substitute name, vCPUs, memory:

WIN_ISO=/var/lib/libvirt/isos/<windows-arm64>.ISO VIRTIO_ISO=/var/lib/libvirt/isos/virtio-win.iso virt-install \ --name <VM_NAME> \ --vcpus <N> --memory <MB> \ --controller type=scsi,model=virtio-scsi \ --disk pool=vmpool,size=256,format=qcow2,bus=scsi \ --disk "$WIN_ISO",device=cdrom,bus=scsi,readonly=on,boot.order=1 \ --disk "$VIRTIO_ISO",device=cdrom,bus=usb,readonly=on \ --os-variant win11 \ --arch aarch64 \ --boot uefi,firmware.feature0.name=secure-boot,firmware.feature0.enabled=yes \ --tpm backend.type=emulator,backend.version=2.0,model=tpm-tis \ --network bridge=br0,model=virtio \ --graphics vnc,listen=127.0.0.1 \ --video model=ramfb \ --noautoconsole

This build's pool:

Name

vCPU

RAM (MB)

Disk

Name

vCPU

RAM (MB)

Disk

flang-arm64-windows-msvc

8

65536

256

lldb-aarch64-windows

8

32768

256

clang-arm64-windows-msvc

16

32768

256

clang-arm64-windows-msvc-2stage

32

65536

256

If --video model=ramfb is rejected

Create without it, then virsh edit <vm> and replace the entire <video> block:

<video> <model type='ramfb' primary='yes'/> </video>

Delete the <address> and <alias> lines — ramfb is not a PCI device and cannot take a PCI address. virt-xml --edit --video model=ramfb fails because it changes the model but leaves the stale address behind.

5. Fixed VNC ports

Assign explicitly so the VM-to-port mapping survives restarts:

virt-xml flang-arm64-windows-msvc --edit --graphics vnc,port=5900,listen=127.0.0.1 virt-xml lldb-aarch64-windows --edit --graphics vnc,port=5901,listen=127.0.0.1 virt-xml clang-arm64-windows-msvc --edit --graphics vnc,port=5902,listen=127.0.0.1 virt-xml clang-arm64-windows-msvc-2stage --edit --graphics vnc,port=5903,listen=127.0.0.1

autoport is not a --graphics suboption. Verify it flipped to no:

for vm in flang-arm64-windows-msvc lldb-aarch64-windows \ clang-arm64-windows-msvc clang-arm64-windows-msvc-2stage; do echo "$vm: $(virsh dumpxml $vm | grep -o "port='[0-9]*' autoport='[a-z]*'")" done

If any still shows autoport='yes', set it by hand in virsh edit — that attribute overrides your fixed port. A VM left on autoport can also squat on a port you intended for another VM, producing Failed to find an available port: Address already in use. Diagnose with sudo ss -tlnp | grep -E '590[0-9]'.

Adding a fifth VM later: assign 5904+ explicitly. libvirt will not route around your fixed ports.

6. macOS/Linux workstation access

~/.ssh/config:

Host <KVM hostname> HostName <KVM host IP> User <user> IdentityFile <your SSH private key> LocalForward 15900 localhost:5900 LocalForward 15901 localhost:5901 LocalForward 15902 localhost:5902 LocalForward 15903 localhost:5903 ExitOnForwardFailure yes

Local ports are deliberately not 5900–5903: macOS Screen Sharing refuses loopback on 5900 with "Cannot control your own screen."

On the Mac, use TigerVNC, not Apple's Screen Sharing — the latter negotiates security types poorly against QEMU's no-auth VNC server and prompts for a password that does not exist.

brew install --cask tigervnc

Then ssh <KVM hostname> and connect via VNC to localhost:15900–15903.

If SSH reports Address already in use with nothing in lsof, check for a stale ControlMaster socket in /private/tmp/ and delete it.

7. Windows installation (per VM)

Expect half an hour per VM. Connect VNC before or immediately as the VM starts — the "Press any key to boot from CD or DVD" prompt times out in a few seconds and falls through to PXE.

  1. Press a key at the CD prompt.

  2. Setup jumps to Install driver to show hardware — expected; the virtio disk is invisible.

  3. Browse to D:\vioscsi\w11\ARM64 (the USB-attached virtio-win CD). Install.

    • vioscsi = virtio-scsi controller. viostor = virtio-blk. With the single-controller layout above you need only vioscsi.

  4. The 256 GB disk appears. Proceed with installation.

  5. At the mid-install reboot, do not press a key at the CD prompt — let it time out so it boots from disk.

  6. Do not configure internet in the first boot. Select “I don’t have internet” and create a regular user. Failing to do so will make the use of a Microsoft Account mandatory.

  7. After you reach the desktop for the first time, open the File Manager, browse to D:\NetKVM\w11\ARM64 and install the driver (right-click the .inf and select Install. You will notice that the network icon in the system tray will go live. Then, go to the Control Panel, Network Settings and configure the adapter to the network IP/Gateway/DNS you want.

  8. Next, go to Settings and Remote Desktop Configuration and activate Remote Desktop. From then on, you may use RDC/Windows App to connect to the VM directly.

  9. Open Settings one last time, click Activate Windows. Then Change Product Key and type the Linaro VLK.

Take the first VM all the way to a working desktop and a clean reboot before starting the others. Once one is proven, the rest can run in parallel.

If Setup complains the PC cannot run Windows 11: Shift+F10 → regedit → create HKLM\SYSTEM\Setup\LabConfig with DWORD values BypassTPMCheck, BypassSecureBootCheck, BypassRAMCheck, BypassStorageCheck, BypassCPUCheck set to 1. With TPM 2.0 and Secure Boot configured as above this should not trigger.

8. Teardown — safe commands

Never use --remove-all-storage. Delete the VM's own disk by path:

virsh destroy <vm> 2>/dev/null virsh undefine <vm> --nvram rm -f /var/lib/libvirt/images/<vm>.qcow2

--nvram clears the per-VM UEFI variables. Without it, stale boot entries carry into the rebuild and produce 0xc000000d / missing-BCD errors on the fresh install.

Never virsh destroy a VM mid-installation. A hard power-off during Windows Setup's file-copy/boot-configuration stage leaves an ESP and a boot entry with an incomplete BCD. All four VMs were lost this way on the first build. Check installer progress before any destructive operation.

9. Known-good minimal reproducer

When a VM fails to boot and the cause is unclear, this strips out libvirt, Secure Boot, and TPM in one step. If it reaches Setup, the problem is in the domain config; if it fails identically, the problem is firmware/QEMU/media.

qemu-system-aarch64 \ -M virt -accel kvm -cpu host -m 4G -smp 4 \ -bios /usr/share/AAVMF/AAVMF_CODE.no-secboot.fd \ -device ramfb \ -device qemu-xhci -device usb-kbd -device usb-tablet \ -device virtio-scsi-pci,id=scsi \ -drive file=/var/lib/libvirt/isos/<windows>.ISO,media=cdrom,if=none,id=install \ -device scsi-cd,drive=install,bus=scsi.0 \ -vnc 127.0.0.1:10 -S

-S starts paused so you can attach VNC before the boot prompt races past. Unpause from the QEMU monitor.

10. Diagnostics reference

# Is the guest doing I/O or genuinely hung? virsh domstats <vm> --block | grep rd.bytes # sample twice, 20s apart # What did QEMU actually get told to do? sudo cat /proc/$(pgrep -f "qemu-system-aarch64.*<vm>")/cmdline | tr '\0' '\n' # Startup errors sudo tail -30 /var/log/libvirt/qemu/<vm>.log # Honest disk read (drop cache first — otherwise you measure page cache) sudo sysctl -w vm.drop_caches=3 dd if=<file> of=/dev/null bs=4M status=progress