Skip to end of banner
Go to start of banner

Xen: Booting Xen with Ubuntu on AVA

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

Version 1 Next »

When tried to install offical Xen package apt-get install xen-hypervisor-4.16-arm64, it failed to boot Xen hypervisor on AVA board. This article records the details for how to build GRUB and replace Xen hypervisor with TRS binary.

Build GRUB binary

I cross built GRUB binary for Arm64 target on my x86_64 machine.

Step 1: we need to download git source code:

$ git clone https://git.savannah.gnu.org/git/grub.git
$ cd grub

Step 2: execute bootstrap for downloading dependent package:

$ ./bootstrap

Step 3: configure the package:

$ ./configure --build=x86_64 --host=x86_64 --target=aarch64 --with-platform=efi \
  --disable-werror TARGET_CC=aarch64-linux-gnu-gcc \
  TARGET_OBJCOPY=aarch64-linux-gnu-objcopy TARGET_NM=aarch64-linux-gnu-nm \
  TARGET_RANLIB=aarch64-linux-gnu-ranlib TARGET_STRIP=aarch64-linux-gnu-strip \
  --prefix=$PWD --exec-prefix=$PWD

Step 4: prepare the grub-initial.cfg file.
Firstly, you need to know the UUID for the Ubuntu root file system:

root@ava:~# blkid 
/dev/nvme0n1p3: UUID="3e651572-426e-4280-86b6-752ce01bbe2f" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="f55eceef-7657-4af7-8e99-6301238e7ae0"
/dev/nvme0n1p1: UUID="E5E9-73BC" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="808434d6-d314-4ffe-92bd-f30037f1f832"
/dev/nvme0n1p4: UUID="5836e2e5-bdde-4355-bcab-d08d74a8f6ee" TYPE="swap" PARTUUID="a2d5fa9f-49a8-40af-9558-415fbefaff7d"
/dev/nvme0n1p2: UUID="7b2d7994-68ad-4a0b-bdca-db30c6f377ce" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="0baf2b82-0eb3-4782-b8a2-e8ae487040da"
/dev/nvme1n1p2: LABEL="rootfs" UUID="6091b3a4-ce08-3020-93a6-f755a22ef03b" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="rootfs" PARTUUID="f3374295-b635-44af-90b6-3f65ded2e2e4"
/dev/nvme1n1p1: SEC_TYPE="msdos" LABEL_FATBOOT="bootfs" LABEL="bootfs" UUID="7819-74F8" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="ESP" PARTUUID="00112233-1234-1111-2222-000123456789"

From above log we can know the Ubuntu’s root file system UUID is 7b2d7994-68ad-4a0b-bdca-db30c6f377ce. Then we can prepare grub-initial.cfg file as below. The main purpose for this file is to initialize root for GRUB and then it loads Ubunut’s grug config file $prefix/grub.cfg.

# First partition on first disk, most likely EFI system partition. Set it here
# as fallback in case the search doesn't find the given UUID.
search.fs_uuid 7b2d7994-68ad-4a0b-bdca-db30c6f377ce root
set prefix=($root)'/boot/grub'
configfile $prefix/grub.cfg

Step 5: build GRUB efi.
I used below command to build GRUB efi file, which can be loaded by UEFI.

#!/usr/bin/bash

set -eu

# GRUB doesn't allow loading new modules from disk when secure boot is in
# effect, therefore pre-load the required modules.
MODULES=
MODULES="$MODULES part_gpt fat ext2"           # partition and file systems for EFI
MODULES="$MODULES configfile"                  # source command
MODULES="$MODULES password_pbkdf2"             # hashed password
MODULES="$MODULES echo normal linux"           # boot linux
MODULES="$MODULES all_video"                   # video output
MODULES="$MODULES search search_fs_uuid"       # search --fs-uuid
MODULES="$MODULES reboot sleep"                # sleep, reboot

TMP_GRUB_CFG='grub-initial.cfg'
TMP_GRUB_EFI='grubaa64.efi'

rm -f $TMP_GRUB_EFI

make clean
make && make install

./bin/grub-mkstandalone \
        --disable-shim-lock \
        --format=arm64-efi \
        --modules="$MODULES" \
        --locale-directory=//usr/share/locale/ \
        --directory=/home/leoy/Dev1/arm_blueprint/grub/lib/grub/arm64-efi \
        --output="$TMP_GRUB_EFI" \
        "boot/grub/grub.cfg=$TMP_GRUB_CFG"

Step 6: Install GRUB and the associated modules.

Copy GRUB files from x86_64 host to AVA board: 

$ cd /path/to/grub/
$ scp grubaa64.efi root@ubuntu_ip:/tmp/
$ scp -r lib/grub/arm64-efi/ root@ubuntu_ip:/tmp/

The grub EFI is stored in the boot partition, the folder is efi/EFI/ubuntu. Use the built Grub EFI to replace the old one (and backup the old file):

$ mount /dev/nvme0n1p1 /mnt/
$ cp /mnt/EFI/ubuntu/grubaa64.efi /mnt/EFI/ubuntu/grubaa64.efi.bak # Backup original file
$ cp /tmp/grubaa64.efi /mnt/EFI/ubuntu/

Use below commands to copy Grub modules into :

$ mv /boot/grub/arm64-efi /boot/grub/arm64-efi.bak
$ mv /tmp/arm64-efi /boot/grub

Update Xen hypvervisor binary

The build Xen hypervisor has been uploaded to xen.efi, you can download and copy it into the Ubuntu’s folder /boot.

Add GRUB menu entry

Add a new GRUB entry for booting Xen hypervisor:

menuentry 'Ubuntu GNU/Linux, with Xen hypervisor' --class ubuntu --class gnu-linux --class gnu --class os --class xen $menuentry_id_option 'xen-gnulinux-simple-7b2d7994-68ad-4a0b-bdca-db30c6f377ce' {
        insmod part_gpt
        insmod ext2
        search --no-floppy --fs-uuid --set=root 7b2d7994-68ad-4a0b-bdca-db30c6f377ce
        echo    'Loading Xen xen.efi ...'
        if [ "$grub_platform" = "pc" -o "$grub_platform" = "" ]; then
            xen_rm_opts=
        else
            xen_rm_opts="noreboot dom0_mem=8192M bootscrub=0 iommu=on loglvl=all guest_loglvl=all"
        fi
        xen_hypervisor  /boot/xen.efi placeholder   ${xen_rm_opts}
        echo    'Loading Linux 6.7.0-rc8-00119-g1f874787ed9a ...'
        xen_module      /boot/vmlinuz-6.7.0-rc8-00119-g1f874787ed9a placeholder root=UUID=7b2d7994-68ad-4a0b-bdca-db30c6f377ce ro  ignore_loglevel sysrq_always_enabled nr_cpus=32 mitigations=off nmi_watchdog=0 nowatchdog nosoftlockup
        echo    'Loading initial ramdisk ...'
        xen_module      --nounzip   /boot/initrd.img-6.7.0-rc8-00119-g1f874787ed9a
}

Here three thing you need to change on your working AVA board:

Firstly, you need to replace the root file system UUID 7b2d7994-68ad-4a0b-bdca-db30c6f377c.
Then, you need to update the Linux kernel image file name and initrd file name respectively.

  • No labels