Confidential Computing on Realms (CCR)
- Leonardo Garcia
Project name | Confidential Computing on Realms |
|---|---|
Target release | Jun 30, 2025 |
Epic | |
Document status | DRAFT |
Document owner | @Leonardo Garcia, @Mathieu Poirier and @Kevin Zhao |
LEO | @Mike Holmes and @Tom Gall |
BD | N/A |
Architect | @Leonardo Garcia |
QA | N/A |
Project Manager | N/A |
Last updated | Jun 13, 2023 |
Objective
Provide a comprehensive set of software and emulated hardware that enables a person to run Confidential Computing on Arm. This should be done through the usage of Realms without the need to have access to real hardware.
This will involve the provision of full system emulation capable of running the Realms Management Extension (RME) as defined by the Confidential Computing Architecture (CCA) specification, alongside all the software to run a full Confidential Computing solution on Arm (firmware, operating system, system libraries, virtualization management, and confidential virtual machine images).
Confidential Computing is inherently complex and covers a wide spectrum of technologies. As such we are proposing to divide the project in 6 sets for objectives. That way the project remains manageable and it provides a roadmap that is easier to understand. The first objective is to run confidential virtual machines on an existing hardware model. From there we intend to validate CCA support provided by an open source emulator and make sure that both (hardware model and emulator) provide the same behaviour. The third objective is to run Confidential Containers in confidential virtual machines while the fourth objective is to integrate CCA in Linaro’s Trusted Reference Stack (TRS), providing a delivery medium for the technology. The fifth objective is to integrate CCA in Edge related use cases such as live updates without service disruption. Lastly we plan to explore advanced scenarios like PCIe devices pass through and confidential VM live migration. Those are not fully covered by the current CCA specification but will surely be addressed in future versions.
Background
Even though Linaro is not planning to offer a product per se on Confidential Computing, we think it is valuable to various parties inside and outside the company to have one document that describes, in high level, our objectives with respect to Confidential Computing.
In line with traditions, Arm drove the base CCA enablement in the firmware and Linux kernel communities. In that respect they introduced the Runtime Security Subsystem (RSS) in TF-M, added support for the RME extension in the TF-A Monitor, released the Realm Management Monitor (TF-RMM) and provided support for Realms in the Linux kernel and KVM.
Our goal is to integrate the above components to provide a set of software and emulated hardware capable of running a full Confidential Computing solution based on Realms on Arm. Even though Linaro is not alone on this task, with many companies, notably Arm, working on various parts of the solution, there is currently no one approaching the problem with a holistic view. The objective is to address challenges identified while integrating all the components of the stack in order to provide a complete reference model that can be used by others to build their own Confidential Computing solutions on top of Arm platforms.
Arms traditionally avoids making contributions to the QEMU project and as such, it was already in Linaro’s roadmap to enable RME support in QEMU emulation. This work is now completed and upstream, targeted to be available in August 2023 as part of QEMU release 8.1. So, in theory, someone would have all the basic pieces needed to run an emulated system with CCA support: Arm simulator or QEMU emulator, RSS, TF-A Monitor, TF-RMM and Linux/KVM. How well these things fit together needs to be evaluated. The accuracy of the Arm simulator and QEMU emulator with respect to the CCA specification also need to be assessed.
On the upper side of the software stack, many applications and frameworks appeared in the past years to support Confidential Computing with confidential virtual machines on other platforms, most notably AMD with their SVE-SNP support. Even though some of these applications are catching up and adding support for the Arm CCA, there is still work to be done on that side as well to make sure we have a coherent reference implementation for the virtualization management.
Success metrics
Goal | Metric |
|---|---|
To be able to run a fully functional confidential virtual machine by integrating the layers of the CCA stack on an FVP model. | By 4Q 2023, provide an image that can be booted by the FVP model within which it is possible to create a Realm and run a confidential virtual machine. All elements of the stack have been proven to run properly on the FVP model. |
To be able to run a fully functional confidential virtual machine by integrating the layers of the CCA stack on the QEMU 8.1 emulator. | By 2Q 2024, provide an image that can be booted by a QEMU emulated system within which it is possible to create a Realm and run a confidential virtual machine. |
Create a simple confidential virtual machine using the CCA hardware extension (RME) without the need to have knowledge about how the underlying system works. | By 3Q 2024, provide a reference implementation of open source tools used to manage virtual environments (containers or virtual machines) capable of controlling the Realms according to the current CCA specification alongside any additional requirements demanded by the CCC definition of a confidential virtual machine. |
Create a simple confidential virtual machine in TRS using the CCA hardware extension on QEMU 8.1 using tools available on that distribution. | By 2Q 2025, integrate the layers of the CCA stack in TRS so that it is possible to instantiate confidential virtual machines in that distribution. |
Assumptions
Requirements
The requirements will be divided in 6 major objectives that are more or less self contained. This will make planning and execution easier.
Objective#1: Verify enablement in FVP models
Requirement | User Story | Importance | Jira Issue | Notes |
|---|---|---|---|---|
TechnicalUser#01 - Realms on FVP
| As a user, I want to be able to download an FVP model with RME support, boot it with an image with the proper support to create Realms and, within that image, execute one or more confidential virtual machines. | HIGH |
| Process:
|
Objective#2: Validate CCA support in QEMU 8.1
Requirement | User Story | Importance | Jira Issue | Notes |
|---|---|---|---|---|
TechnicalUser#02 - Realms on QEMU
| As a user, I want to be able to run a QEMU emulated system within which I can boot an image with the proper support to create Realms and, within that image, execute one or more confidential virtual machines.
Ideally, the confidential VM will be able to use all the Realm accessible CPU features available in the host processor. | HIGH |
| Process: Go through the same process as in TechnicalUser#01 but this time using QEMU rather than the FVP model. |
TechnicalUser#03 - Reference emulators
| As a user, I want to be able to run the same image with Realms support on both the FVP model and the QEMU emulated machine, provided we have the needed drivers and modules installed in the image to access the virtual hardware provided in each environment. As both emulated environments follow the same specification, they should behave exactly the same way when it comes to creating confidential virtual machines. |
|
| Process:
|
Objective#3: Validate the virtualization and container management stack
Requirement | User Story | Importance | Jira Issue | Notes |
|---|---|---|---|---|
User#01 - Running confidential virtual machines on Arm
| As a user, I want to be able to easily create confidential virtual machines on Arm platforms using commonly used open source management tools available for this. | HIGH |
|
|
User#02 - Running confidential containers on Arm
| As a user, I want to be able to easily run workloads on confidential containers on Arm platforms using commonly used open source container orchestration tools available for this.
The integration with the orchestration tools needs to be one that allows the creation of both confidential containers and non-confidential containers running side by side in the same platform. |
|
|
|
User#03 - Running confidential workloads as defined by the CCC
| As a user, I want to be able to easily run confidential workloads on Arm platforms using commonly used open source management tools available for this that also support the additional requirements imposed by the CCC definition of confidential computing, such as encryption of images and attestation of the running executables. |
|
|
|
Objective#4: Integrate the whole stack on a consumable image
Requirement | User Story | Importance | Jira Issue | Notes |
|---|---|---|---|---|
User#04 - Single image
| As a user, I want to have a one stop shop where I can download a single image with everything I would need to run confidential containers on Arm. This image could be run by the FVP model or by QEMU. It will contain all the firmware and operating system features needed, along with a VMM capable of launching confidential VMs using RME inside the emulated system and a simple confidential container workload that could be easily launched from within the image host. | HIGH |
|
|
Objective#5: Edge use cases
This objective is still in design phase. There aren’t enough details yet to properly describe it.
Requirement | User Story | Importance | Jira Issue | Notes |
|---|---|---|---|---|
User#05 - OTA live update
| As a user, I want to have a one stop shop where I can download a single image with everything I would need to run confidential containers on Arm. This image could be run by the FVP model or by QEMU. It will contain all the firmware and operating system features needed, along with a VMM capable of launching confidential VMs using RME inside the emulated system and a simple confidential container workload that could be easily launched from within the image host. | HIGH |
|
|
Objective#6: Advanced use cases
This objective is still in design phase. There aren’t enough details yet to properly describe it.
Requirement | User Story | Importance | Jira Issue | Notes |
|---|---|---|---|---|
Confidential VM attestation an verification |
| HIGH |
|
|
PCIe pass through |
|
|
|
|
Live migration |
|
|
|
|
Page fault |
|
|
|
|
Monitoring tools / inspection tools | The user shall be able to run monitoring tools (e.g. eBPF) on confidential VMs from the host… |
|
|
|
Milestones
This work will be tracked in Linaro’s Jira instance.
User interaction and design
For Objective#1 and Objective#2, the interaction will be through command line.
Objective#3 will use command line and graphical interfaces already available for the management tools identified as relevant during the investigation phase. Ideally, no changes will be introduced on the tools in order to add support for Realms on Arm platforms.
Open Questions
Question | Answer | Date Answered |
|---|---|---|
Do we need to start the stack with the Runtime Security Subsystem (RSS) or can the FVP start the TF-A Monitor right away? |
Out of Scope
This project will focus only on Trusted Execution Environments (TEEs) running on Realms. It will not do any work on the TrustZone technology available in Arm platforms.
References
Arm provided lots of good information on their efforts during Linaro Connect and in presentation to the Confidential Computing Consortium. Pointers below.
LHR23-304 - Runtime Security Subsystem [Arm CCA HES] - An overview
LHR23-311 - Arm Confidential Compute Architecture open-source enablement
Arm also made a presentation on Arm CCA open-source enablement (presented to the Confidential Computing Consortium, with information from LHR23-311 and LHR23-315 above). The recording is available here, and the presentation is at https://drive.google.com/file/d/1OKLnERWvu09k5LNRhz70-nkAfKkZEraE/view.