Introduction

We need to maintain the upstreaming patches for Xen and QEMU repositories for support virtio device modules (e.g. virtio GPU device mode), this page is to describe how to download the related source code and build TRS. It also gives commands for launching Android Cuttlefish images on Xen virtual machine.

Download TRS offical repository

We can refer the TRS documenation Developer setup to download TRS repository and Getting the host packages to install dependencies on the PC.

Checkout the maintenance branches

$ cd /path/to/trs-workspace/meta-ledge-secure
$ git remote add trs-maintenance https://gitlab.com/Linaro/blueprints/automotive/xen-aosp/trs-fragments/meta-ledge-secure.git
$ git fetch trs-maintenance
$ git checkout trs-maintenance/xen_aosp_maintenance_20231211

$ cd /path/to/trs-workspace/meta-virtualization/
$ git remote add trs-maintenance https://gitlab.com/Linaro/blueprints/automotive/xen-aosp/trs-fragments/meta-virtualization.git
$ git fetch trs-maintenance
$ git checkout trs-maintenance/xen_aosp_maintenance_20231211

$ cd /path/to/trs-workspace/trs/
$ git remote add trs-maintenance https://gitlab.com/Linaro/blueprints/automotive/xen-aosp/trs-fragments/trs.git
$ git fetch trs-maintenance
$ git checkout trs-maintenance/xen_aosp_maintenance_20231211

Build TRS image for qemuarm64 target

TS_SUPPORTED_TARGETS=tsqemuarm64-secureboot make

Flash TRS images on AVA platform

We need to copy TRS wic image trs-image-trs-qemuarm64.wic on AVA with Ubuntu system, then we can execuate below shell script to flash TRS images onto a NVME block device with command ./flash_ewaol_image.sh /dev/nvmeXnY (you might need to change the nvme block device name nvmeXnY based on your local environment).

Below is the source code for flash_ewaol_image.sh. Note, the script executes commands e2fsck and resize2fs to enlarge the partition size, when you use older version’s binary for e2fsck and resize2fs, it will fail and report errors. In this case, you need to download e2fsprogs package (version 1.47.0 is verified) and build these two binaries.

#!/bin/bash

DEV=$1

sudo parted ${DEV} print

echo "Please confirm if the partition is right?"
while : ; do
	read -n 1 k <&1
	if [[ $k = y ]] ; then
		printf "\n"
		break
	else
		printf "\nExit\n"
		exit 0
	fi
done

echo "Copy image to ${DEV}"
sudo dd bs=4M if=./trs-image-trs-qemuarm64.wic of=$DEV status=progress && sync

sudo growpart $DEV 2
sudo /home/leoy/Downloads/e2fsprogs-1.47.0/e2fsck/e2fsck -f "${DEV}p2"
sudo /home/leoy/Downloads/e2fsprogs-1.47.0/resize/resize2fs "${DEV}p2"
sudo parted ${DEV} print

Run Android image on Xen virtual machine

By following up the steps described in Xen: porting Android Cuttlefish on Xen Virtual Machine, we can get the Android Cuttlefish images, then we need to copy the U-boot binary and Cuttlefish images into Xen Dom0 (e.g. copy into the folder /home/cuttlefish/ (You need to replace the IP address in your local environment with TRS_IP_ADDRESS in below commands).

$ cd android-uboot
$ scp out/u-boot/dist/u-boot.bin root@TRS_IP_ADDRESS:~/

# On Arm64 Ubuntu/Debian system
$ scp -r andriod-cf/cuttlefish root@TRS_IP_ADDRESS:/home/

Hint: If Android Cuttlefish images are used from mounted partition or located under some other path,

then cuttlefish expects some image files under path /home/USERNAME/<path_to_cuttlefish>/cuttlefish/instances/cvd-1/ to be valid. The easiest way to do that is to create 3 symlinks to the right location:

os_composite.img -> /<correct_path_to_cuttlefish>/cuttlefish/instances/cvd-1/os_composite.img
persistent_composite.img -> /<correct_path_to_cuttlefish>/cuttlefish/instances/cvd-1/persistent_composite.img
sdcard.img -> /<correct_path_to_cuttlefish>/cuttlefish/instances/cvd-1/sdcard.img

Then we can use below Xen configuration file to launch Xen virtual machine with command xl -vvv create aaos_vm.cfg -c:

# File name: aaos_vm.cfg
# Copyright (c) 2023, Linaro Limited.
#
# SPDX-License-Identifier: MIT

name = "aaos-vm"
memory = 6144
vcpus = 4
extra = " earlyprintk=xenboot console=hvc0 rw androidboot.console=hvc0 androidboot.serialconsole=1"
root = "/dev/xvda2"
kernel = "/home/root/u-boot.bin"
disk = ['format=qcow2, vdev=xvda, access=rw, target=/home/cuttlefish/instances/cvd-1/overlay.img', 'format=raw, vdev=xvdb, access=rw, target=/home/cuttlefish/instances/cvd-1/persistent_composite.img', 'format=raw, vdev=xvdc, access=rw, target=/home/cuttlefish/instances/cvd-1/sdcard.img']
vif = ['mac=00:16:3E:74:34:32', 'mac=00:16:3e:5f:48:e4', 'mac=00:16:3E:74:34:32,script=vif-bridge,bridge=xenbr0']
virtio_qemu_domid = 0

Once the system boot up, since we have added the option -c in the xl create command, we can directly access the virtual machine's console. Android prints out verbose logs and this is hard for us to interact with the virtual machine, we can mute the logs with the command:

console:/ $ su
console:/ # echo 0 > /proc/sys/kernel/printk

Then we can check the virtual networking interface’s IP address:

console:/ # ifconfig
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0 
          inet6 addr: ::1/128 Scope: Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:22 errors:0 dropped:0 overruns:0 frame:0 
          TX packets:22 errors:0 dropped:0 overruns:0 carrier:0 
          collisions:0 txqueuelen:1000 
          RX bytes:3123 TX bytes:3123 

dummy0    Link encap:Ethernet  HWaddr 52:f2:68:a5:92:cf
          inet6 addr: fe80::50f2:68ff:fea5:92cf/64 Scope: Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0 
          TX packets:40 errors:0 dropped:0 overruns:0 carrier:0 
          collisions:0 txqueuelen:1000 
          RX bytes:0 TX bytes:8518 

eth1      Link encap:Ethernet  HWaddr 00:16:3e:5f:48:e4  Driver vif
          inet6 addr: fe80::216:3eff:fe5f:48e4/64 Scope: Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:951 errors:0 dropped:23 overruns:0 frame:0 
          TX packets:43 errors:0 dropped:0 overruns:0 carrier:0 
          collisions:0 txqueuelen:1000 
          RX bytes:66995 TX bytes:6893 

eth2      Link encap:Ethernet  HWaddr 00:16:3e:74:34:32  Driver vif
          inet addr:192.168.3.242  Bcast:192.168.3.255  Mask:255.255.255.0 
          inet6 addr: fe80::d6ee:1af5:b9cf:87d7/64 Scope: Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1128 errors:0 dropped:2 overruns:0 frame:0 
          TX packets:333 errors:0 dropped:0 overruns:0 carrier:0 
          collisions:0 txqueuelen:1000 
          RX bytes:119939 TX bytes:41431

Android Cuttlefish uses eth0 and eth1 for specific purpose for emulation WiFi and other networking devices, this is why we can see eth2 is used as normal networking interface and it acquires IP address via DHCP. We can connect eth2 with ADB on the host:

leoy@leoy-yangtze:~/$ adb connect 192.168.3.242:5555
connected to 192.168.3.242:5555
leoy@leoy-yangtze:~/$ adb devices
List of devices attached
192.168.3.242:5555	device
leoy@leoy-yangtze:~/$ adb shell
vsoc_arm64_only:/ $ su
vsoc_arm64_only:/ #