/
Build and set up virtio-gpu with venus protocol on Xen

Build and set up virtio-gpu with venus protocol on Xen

Owned by Takahiro Akashi (Deactivated)
Last updated: Feb 01, 2024

This document is a tentative summary of what I’m currently working on to evaluate virtio-gpu on Xen. So even if you follow the following instructions, you may not be able to enable virtio-gpu with venus protocol (gpu-accelarated Vulkan support) on Xen’s guest VM.

Please note that the WIP code referenced below may be subject to change without notice.

Target/Environment

Hardware Platform

AVA platform

Xen

modified v4.18

Host OS(Dom0)

Ubuntu 23.10 + modified kernel (6.7)

Guest OS (DomU)

Debian 12 + modified kernel (6.7)

Build Xen

For some reason, I temporily use Xen binary (from xen-aosp banch?) that Leo built along with xen tools that I compiled from xen-aosp’s v4.18-rc2-xen-aosp branch[1] with the following patch.

From 0e107b8cec9ee26b8a4044561f6141cd666ceeec Mon Sep 17 00:00:00 2001 From: AKASHI Takahiro <takahiro.akashi@linaro.org> Date: Wed, 13 Dec 2023 15:41:47 +0900 Subject: [PATCH] libxl: make use of device_model_args for xenpvh Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org> --- tools/libs/light/libxl_dm.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/libs/light/libxl_dm.c b/tools/libs/light/libxl_dm.c index f0bceee6d804..b4772c811b2d 100644 --- a/tools/libs/light/libxl_dm.c +++ b/tools/libs/light/libxl_dm.c @@ -1823,6 +1823,10 @@ static int libxl__build_device_model_args_new(libxl__gc *gc, flexarray_append(dm_args, "-machine"); switch (b_info->type) { case LIBXL_DOMAIN_TYPE_PVH: + flexarray_append(dm_args, "xenpvh"); + for (i = 0; b_info->extra_pv && b_info->extra_pv[i] != NULL; i++) + flexarray_append(dm_args, b_info->extra_pv[i]); + break; case LIBXL_DOMAIN_TYPE_PV: flexarray_append(dm_args, "xenpv"); for (i = 0; b_info->extra_pv && b_info->extra_pv[i] != NULL; i++) -- 2.40.1

Build and install xen (xen-tools) as follows:

$ ./configure --disable-docs --disable-golang --disable-ocamltools \ --enable-ioreq-server \ // --with-system-qemu=/home/akashi/.local/bin/qemu-system-aarch64 $ make debball $ sudo dpkg -i dist/xen-uppstream-4.18-rc.deb

NOTE: Under the environment with the setup described below, dom0 still boots with qemu-system-i386 while a guest will boot with qemu-system-aarch64.

Check /etc/default/xencommons and confirm that QEMU_XEN points to your local qemu ("/home/akashi/.local/bin/qemu-system-i386") that was built above.

Modify /etc/init.d/xencommons, adding "LD_LIBRARY_PATH=/home/akashi/.local/lib/aarrch64-lnux-gnu" and replacing qemu-system-aarch64 with qemu-system-i386.

Modify /boot/grub/grub.cfg, adding "dom0_mem=xxxM" at xen_hypervisor command.

Enable daemons:

$ sudo systemctrl enable xencommons $ sudo systemctrl enable xendomains $ sudo systemctrl enable xendriverdomain ($ sudo systemctrl enable xen-watchdog)

[1] linaro / Blueprints / Automotive / xen-aosp / Xen · GitLab

Build Lnux kernel (host)

Use my current repository[2].

T.B.D.

[2] https://github.com/t-akashi/linux.git branch: virtio-gpu/v67_rui2_digetx4

(As of today, I fail to push the branch above to github.) Instead use:

https://git.linaro.org/people/takahiro.akashi/linux-aarch64.git branch: virtio-gpu/v67_rui2_digetx4

Build virglrenderer library (host)

Use the latest upstrem master branch and apply the tweak below.

From 83acd6b3d5e8be87dd0075785e14d0f2977b2c2d Mon Sep 17 00:00:00 2001 From: AKASHI Takahiro <takahiro.akashi@linaro.org> Date: Mon, 18 Dec 2023 15:42:37 +0900 Subject: [PATCH] (HACK) to build Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org> --- src/vrend_renderer.c | 3 +++ src/vrend_winsys_gbm.c | 2 ++ 2 files changed, 5 insertions(+) diff --git a/src/vrend_renderer.c b/src/vrend_renderer.c index 04f87e79eb8d..68e958103fb6 100644 --- a/src/vrend_renderer.c +++ b/src/vrend_renderer.c @@ -11864,6 +11864,9 @@ static int get_glsl_version(void) const GLubyte *version_str; int count; +#if 1 + return 460; +#endif version_str = glGetString(GL_SHADING_LANGUAGE_VERSION); if (!version_str) { virgl_error("GL_SHADING_LANGUAGE_VERSION query failed with empty output."); diff --git a/src/vrend_winsys_gbm.c b/src/vrend_winsys_gbm.c index 19e6bb9ff001..781d319fcbc5 100644 --- a/src/vrend_winsys_gbm.c +++ b/src/vrend_winsys_gbm.c @@ -104,6 +104,7 @@ static const struct format_conversion conversions[] = { { GBM_FORMAT_XRGB8888, VIRGL_FORMAT_B8G8R8X8_UNORM }, { GBM_FORMAT_ABGR2101010, VIRGL_FORMAT_R10G10B10A2_UNORM }, { GBM_FORMAT_ABGR16161616F, VIRGL_FORMAT_R16G16B16A16_FLOAT }, + { GBM_FORMAT_XBGR16161616F, VIRGL_FORMAT_R16G16B16X16_FLOAT }, { GBM_FORMAT_NV12, VIRGL_FORMAT_NV12 }, { GBM_FORMAT_ABGR8888, VIRGL_FORMAT_R8G8B8A8_UNORM}, { GBM_FORMAT_XBGR8888, VIRGL_FORMAT_R8G8B8X8_UNORM}, @@ -189,6 +190,7 @@ static const struct planar_layout *layout_from_format(uint32_t format) case GBM_FORMAT_ABGR2101010: return &packed_4bpp_layout; case GBM_FORMAT_ABGR16161616F: + case GBM_FORMAT_XBGR16161616F: return &packed_8bpp_layout; default: return NULL; -- 2.40.1

Then configure and build the library as follows:

$ meson setup build \ -Dvenus=true \ -Drender-server=true \ -Drender-server-worker=thread \ -Dbuildtype=release \ -Dprefix=/home/akashi/.local $ ninja -C build install

Build mesa library (host, guest)

Use the latest upstream master branch.

On the host,

$ meson setup build \ -Dgallium-nine=false -Dgallium-xa=disabled -Dglx=dri \ -Dshared-glapi=enabled -Ddri3=enabled -Degl=enabled -Dgbm=enabled \ -Dglvnd=true -Dgallium-drivers=radeonsi,virgl -Dplatforms=x11,wayland \ -Dgles1=disabled -Dgles2=enabled -Dvulkan-drivers=virtio,amd \ -Dlibunwind=disabled -Dbuildtype=release \ -Dprefix=/home/akashi/.local

On the guest,

meson setup build \ -Dgallium-nine=false -Dgallium-xa=disabled -Dglx=dri \ -Dshared-glapi=enabled -Ddri3=enabled -Degl=enabled -Dgbm=enabled \ -Dglvnd=true -Dgallium-drivers=swrast,virgl -Dplatforms=x11,wayland \ -Dgles1=disabled -Dgles2=enabled -Dvulkan-drivers=swrast,virtio \ -Dlibunwind=disabled -Dbuildtype=release \ -Dprefix=/home/akashi/.local

Build the library as follows:

$ ninja -C build install

Build qemu (host)

Use my current repository[3].

Configure and build qemu as follows (assuming my own mesa/virglrenderer installed in my local dir):

$ export LD_LIBRARY_PATH=/home/akashi/.local/lib/aarch64-linux-gnu $ export PKG_CONFIG_PATH=/home/akashi/.local/lib/aarch64-linux-gnu/pkgconfig $ mkdir build; cd build $ ../configure \ --prefix=/home/akashi/.local \ --extra-cflags="-I/home/akashi/.local/include " \ --extra-ldflags="-L/home/akashi/.local/lib/aarch64-linux-gnu" \ --target-list=aarch64-softmmu,i386-softmmu \ --enable-kvm \ --enable-xen \ --disable-werror \ --enable-slirp \ --enable-opengl \ --enable-virglrenderer \ --enable-gtk \ --enable-sdl $ ninja install

[3] https://github.com/t-akashi/qemu.git branch: virtio-gpu/aosp_vv82_digetx_rui.2

Create and start Xen guest VM

Use the following xen configuration file.

name = "test_domU" type = "pvh" vcpus = 1 memory = 1024 # # PV(H) Boot parameters # kernel = "/home/akashi/disk/vmlinuz-6.7.0-rc5-00017-ge159be1d194d-dirty" ramdisk = "/home/akashi/disk/initrd.img-6.7.0-rc5-00017-ge159be1d194d-dirty" extra = "earlyprintk=serial serial=hvc0 no_console_suspend root=/dev/xvda2 rootfstype=ext4" virtio = [ "backend=0,type=virtio,device,transport=pci,bdf=00:03.0,backend_type=qemu,grant_usage=true" ] # # Devices # disk = [ "/home/akashi/disk/debian12_guest_xen.img,,xvda,rw,specification=xen" ] vif = [ 'mac=00:16:3e:12:34:56,script=vif-bridge,bridge=xenbr0' ] vfb = [ "vga=stdvga,sdl=enable,opengl=enable" ] # Graphics device_model_version = "qemu-xen" device_model_override = "/home/akashi/.local/bin/qemu-system-aarch64" #'-object', 'memory-backend-memfd,id=mem0,size=1G,share=on', device_model_args = [ '-device', 'usb-ehci', '-device', 'usb-kbd', '-device', 'usb-tablet', '-display', 'gtk,gl=on', '-device', 'virtio-gpu-gl-pci,iommu_platform=on,hostmem=1G,blob=true,context_init=true' ]

 

“bdf” option in “virtio” parameter must be modified for your platform/GPU card.

Then,

$ sudo xl create <guest.cfg> -c