Setting up Debian on a virtual QEMU Aarch64 machine

Test images are great and all but sometime you just want to have a normal distro running in your guest. Here are some semi up to date runes that are correct for Debian Bookworm (although Buster did work much the same way).

In the old days installing distros on Arm machines involved all sorts of special installers and kernels. However these days thanks to fimeware standardisation you can simply plug in a virtual cdrom/usbkey and boot up the normal installer.

Block devices

The canonical block device for QEMU is QCOW2 files which are great for all sorts of things including snapshots and the ability to save and restore. However you can expose a raw block device as well (either spare disk, empty partition, or a virtual block device) and use that directly. In this case I’m using LVM on my host system so I can create a block device just for my guest:

lvcreate -L 30G -n bookworm-arm64 zen-ssd2

Here I create a 30Gb disk call bookworm-arm64 which I’ve create on my spare ssd VG

Firmware

Firmware comes in two parts, one on each flash device. The first contains the actual firmware and the second is an area where the firmware can store important details about the system such as boot order and potentially other secure variables.

QEMU from the source tree comes with some pre-built blobs you can use. There are also packaged blobs from your distro. Pick one or the other as variables are likely to move around so you’ll want to make sure you stick to one.

You will need to ensure the blobs have been rounded to 64 Mb otherwise the pflash device will complain. The firmware packages on Debian already come so tweaked so:

➜ ls -lh /usr/share/AAVMF/AAVMF_VARS.fd 
-rw-r--r-- 1 root root 64M Mar  5  2023 /usr/share/AAVMF/AAVMF_VARS.fd
🕙15:04:26 alex@draig:qemu.git  on  plugins/next [$!?] 
➜  cp /usr/share/AAVMF/AAVMF_VARS.fd ~/images/qemu-arm64-efivars.test

Run the installer

Most of the command line is the same as actually running the device except for the last two lines with attach the cdrom with the netinstall ISO. When you boot you may need to enter the UEFI config to ensure it boots off the CDROM.

 ./qemu-system-aarch64 \
           -machine type=virt,virtualization=on,pflash0=rom,pflash1=efivars \
           -cpu cortex-a53 \
           -smp 8 \
           -accel tcg \
           -device virtio-net-pci,netdev=unet \
           -device virtio-scsi-pci \
           -device scsi-hd,drive=hd \
           -netdev user,id=unet,hostfwd=tcp::2222-:22 \
           -blockdev driver=raw,node-name=hd,file.driver=host_device,file.filename=/dev/zen-ssd2/bookworm-arm64,discard=unmap \
           -serial mon:stdio \
           -blockdev node-name=rom,driver=file,filename=(pwd)/pc-bios/edk2-aarch64-code.fd,read-only=true \
           -blockdev node-name=efivars,driver=file,filename=$HOME/images/qemu-arm64-efivars \
           -m 8192 \
           -object memory-backend-memfd,id=mem,size=8G,share=on \
           -display none \
           -blockdev driver=raw,node-name=cdrom,file.driver=file,file.filename=/home/alex/Downloads/ISOs/debian-12.2.0-arm64-netinst.iso \
           -device scsi-cd,drive=cdrom

A few quick notes:

Don’t format your guest with LVM, this will just make direct kernel booting a pain.
You’ll want to install the openssh server.
Install the GUI if you want but your not likely to use it much

Booting the system with graphics

All the differences are at the end, this time with enabling a GTK display with GL support and a usb keyboard and tablet for keyboard and mouse input.

Booting a custom kernel

One of the main reasons to have a test system is to test kernels. This is done by applying the -kernel and -initrd options. If no UEFI rom is supplied then you will boot direct, otherwise the UEFI firmware will pick up the kernel via the fw_cfg interface and pass it on instead of loading grub

Installing Xen

The best way is to build the current Xen you are interested on your host with a crossbuild and then:

and you can copy the resulting image to your emulated system and install it. Make sure you haven’t installed any of the disto bits. You may need to run update-grub manually to add the Xen entries.

While you probably don’t want to touch most of the tooling stuff you may well want to build your own QEMU so you can have the latest Xen enabling bits. We shall strip down the config as building inside QEMU will be slower than native:

And finally tweak /etc/default/xencommons to point at it

And you you can systemctl restart xencommons.service or reboot and you should be able to list Xen domains:

Booting Xen Directly

Once you have the user space tooling installed you can now boot the hypervisor directly and manually load the dom0 kernel. Please note you’ll want to skip the UEFI bios for this, also we downgrade the CPU as Xen doesn’t support SVE+ out of the box.