ZVM : New-Generation Type 1.5 RTOS Virtualization Solution (Hypervisor) #83991
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
What is changed? - Added a new qemu_max board for QEMU emulation. This includes the addition of board-specific files such as Kconfig, defconfig, board.yaml, and DTS files under `boards/qemu/max`. - Added support for multi-core SMP with QEMU through the `qemu_max_qemu_max_smp` configuration and associated DTS and YAML files. - Updated SoC-specific configurations under `soc/arm/qemu_max`, including Kconfig, SoC-specific settings, and memory regions in `mmu_regions.c`. - Added device tree include file `qemu-virt-max.dtsi` for enhanced compatibility with the qemu_max board. Why do we need this change? - The qemu_max board supports the hardware features of ARMv8.1+, while the existing qemu_cortex_a53 does not. Signed-off-by: Guoqi Xie <[email protected]>
1. ARM Virtualization Host Extension (VHE) is a feature introduced in the ARMv8.1-A architecture to optimize the performance of hypervisors by allowing them to run at EL2 (the hypervisor exception level) while behaving like they are running at EL0/EL1 (user/kernel mode) in the guest. VHE simplifies the implementation of Type-2 (hosted) hypervisors by reducing context-switching overhead between the host and guest environments. We add it for support Zephyr to run on EL2. 2. The VPIDR_EL2 is an ARM system register at EL2 used to provide a virtualized Processor ID to guest VMs, enabling consistent and isolated CPU identification in virtualized environments. Signed-off-by: Guoqi Xie <[email protected]>
Add ignored cmake and yaml file for qemu_max boards. Signed-off-by: Guoqi Xie <[email protected]>
ZVM is a hypervisor based on Zephyr, we add arm64 virtulization support with Zephyr. - Added CMakeLists.txt and Kconfig changes to enable conditional inclusion of ZVM-related files when CONFIG_ZVM is enabled. - Introduced offsets.c to define offset symbols for ZVM structures, ensuring proper alignment and memory layout for ARM64 registers and contexts. Implemented ARM64 virtualization exception handling and context switching in hyp_entry.S, hyp_vector.S, and switch.c: - hyp_entry.S: save and restore vCPU register contexts and handle guest entry/exit logic. - hyp_vector.S: Defined exception vector tables for various exception types at different privilege levels, including IRQs and synchronous exceptions. - switch.c: Implemented ARM64-specific exception handlers, virtual interrupt handling, and memory access emulation for virtual devices. - Updated kernel_arch_data.h to include definitions for ZVM-specific structures like zvm_vcpu_context_t and vcpu_t. - Added zvm_offsets_short_arch.h to define macro offsets for ZVM-specific structures, facilitating memory layout calculations in assembly code. Signed-off-by: Guoqi Xie <[email protected]>
1. cpu.c: - Implemented vCPU initialization, including system register initialization, context saving/restoring, and interrupt handling for ARM64 virtualization. - Added support for vGIC (virtual GIC) and virtual timers (vtimers) for ARM64. - Included context-switching logic for guest and host system registers in hypervisor mode (EL2). - Integrated hardware feature detection to ensure virtualization support (e.g., checking for Hyp mode, VHE, and GICv3). - Supported emulation of ARM64-specific system registers (e.g., CNTV_CTL_EL0, CNTV_CVAL_EL0) and handling of virtual interrupts. 2. timer.c: - Implemented support for virtual timer initialization, including virtual timer context setup and timeout mechanisms. - Added interrupt service routines (ISRs) for virtual and physical timers to handle vCPU timer events. - Integrated logic for emulating guest timer register accesses and managing virtual interrupt propagation. - Included initialization routines for connecting vtimer and ptimer interrupts to the Zephyr IRQ subsystem. Signed-off-by: Guoqi Xie <[email protected]>
- CMakeLists.txt: Added cpu.c, mmu.c, timer.c, switch.c, hyp_entry.S, and hyp_vector.S to the build system. - Kconfig: Added ZVM_MAX_VM_XLAT_TABLES configuration option to set the maximum number of stage-2 translation tables. - mmu.c: Implemented stage-2 memory management for VMs, including mapping/unmapping memory, translation table allocation, and memory domain management. Signed-off-by: Guoqi Xie <[email protected]>
1. isr_wrapper.S: add gic eoi deacrive when process irq. 2. mmu.c: replace sctlr_el1 with sctlr_el2. 3. prep_c.c, smp.c: using arch_set_cpu_id_elx to set tpidr_el2. 4. reset.c: add support for hcr_el2 and enalbe VHE, support mpidr_el1, vmpidr_el2, hstr_el2, mdcr_el2, vttbr_el2. 5. intc_gicv3.c: add deactive support. 6. init.c: add vcpu_struct for thread. Signed-off-by: Guoqi Xie <[email protected]>
- CMakeLists.txt,Kconfig: include ZVM module in the build system. zvm.c: - Implemented core initialization logic for the ZVM module, including hyperviso hardware support checks, ZVM management structure setup, device initialization, and IPI handler. - Added functions for managing guest VMs, such as loading OS images and managing devices. zvm_shell.c: - Introduced shell commands for managing VMs, including creating, running, pausing, deleting, and listing VM information. Signed-off-by: Guoqi Xie <[email protected]>
This commit implements key components for managing virtual machines (VMs) in the ZVM, focusing on VM lifecycle, VCPU handling, and memory management. - vm_manager.c: Implements VM lifecycle management, including creation (zvm_new_guest), deletion (zvm_delete_guest), and state transitions (e.g., running, paused). Adds logic for VM initialization, IRQ management, and runtime operations. - vm.c: Provides core VM handling logic, such as inter-VM communication, vCPU initialization (vm_vcpus_init), and VM state transitions. Implements functions for listing VM information and handling VCPU IRQs and context. - vm_cpu.c: Manages VCPU lifecycle, including state transitions (e.g., running, paused) and context switching. Introduces support for VCPU scheduling, SMP configurations, and CPU affinity management. - vm_mm.c: Handles VM memory management, including memory partitioning, dynamic allocation, and mapping between guest physical addresses (GPAs) and host physical addresses (HPAs). Supports memory domain creation and DTB handling for Linux VMs. Signed-off-by: Guoqi Xie <[email protected]>
This commit introduces key features for managing virtual devices and interrupts within the ZVM, enabling efficient resource allocation and IRQ management for virtual machines (VMs). - vm_device.c: Implements functions to add, remove, and manage virtual devices. Introduces memory mapping for device regions (vm_vdev_mem_add) and device-specific interrupt handling (vm_device_irq_init). Adds support for handling device emulation (handle_vm_device_emulate) and passthrough device unmapping. Also includes callback mechanisms for device IRQs and the initialization of board-specific devices. - vm_irq.c: Implements the virtual interrupt controller for managing VM interrupts. Includes functions for creating and initializing the VM interrupt control block (vm_irq_ctrlblock_create) and IRQ descriptors (vm_virq_desc_init). Adds vCPU IRQ handling logic (vcpu_wait_for_irq) and integrates IRQ initialization for devices. Provides mechanisms for mapping shared and hardware-specific IRQs across virtual and physical devices. Signed-off-by: Guoqi Xie <[email protected]>
- Kconfig: Adds the main configuration menu for enabling the ZVM hypervisor (ZVM), allowing users to set the configuration. - os.c: Implements essential OS-related functions, including support for Linux and Zephyr VM. Features include OS metadata handling (get_os_info_by_type), virtual memory image loading (load_vm_image), and OS object creation (vm_os_create). These enhancements enable seamless guest OS integration and setup within the ZVM hypervisor. Signed-off-by: Guoqi Xie <[email protected]>
- vgic_common.c: Implements common VGIC functionalities, including virtual interrupt configuration, type management (virt_irq_set_type, virt_irq_get_type), priority handling (vgic_virq_set_priority), and synchronization mechanisms (virt_irq_sync_vgic, virt_irq_flush_vgic). These enhancements allow fine-grained control over virtual interrupts and efficient communication between VMs and the hypervisor. - vgic_v3.c: Adds VGICv3-specific implementations, such as initialization (vdev_gicv3_init, vgicv3_lrs_init), list register management (vgicv3_lrs_load, vgicv3_lrs_save), state save/load for interrupts, and support for SGI (Software Generated Interrupts) handling (vgicv3_raise_sgi). These features provide support for ARM GICv3-based interrupt controllers in virtualized environments. Signed-off-by: Guoqi Xie <[email protected]>
This commit introduces PSCI emulation for managing virtual CPU states and system power in ZVM. It supports PSCI 0.2, including functions like SYSTEM_OFF, SYSTEM_RESET, CPU_ON, and CPU_OFF. Signed-off-by: Guoqi Xie <[email protected]>
- thread.c: set spsr register to EL2 mode. Whitout this, thread can not switch when Zephyr run EL2. - ipi.c: add zvm ipi handler when ipi occur. - sched.c: add logic for vCPU thread switch. Only vCPU need to switch in or switch out that will call do_vcpu_swap function. Signed-off-by: Guoqi Xie <[email protected]>
- Defined pass-through device structures in `pt_device.h`, including `pass_through_device_data` and `pass_through_device_config`, which support IRQ configuration and device-specific operations. - Implemented pass-through device initialization and IRQ handling in `pt_device_qemu_max.c`, providing a template for integrating pass-through devices with qemu_max boards. Signed-off-by: Guoqi Xie <[email protected]>
- qemu_max_smp.overlay:
- Defines memory regions for Zephyr and Linux VMs, including:
- Zephyr copy region: `zephyrcpy` at `0xe0000000`
- Linux memory regions: `linuxcpy`, `linuxdtb`, and `linuxrfs`
- Adds VM-specific memory configurations (`vm_zephyr_space` and
`vm_linux_space`) with corresponding address mappings, memory sizes,
and virtual CPU counts.
- Includes placeholder for pass-through device configuration.
- Sets `vm,console` to `uart0` for VM logging.
- qemu_max_smp.conf:
- Allocates virtual space for the hypervisor
(`CONFIG_KERNEL_VM_SIZE`).
- Enables virtual GIC (`CONFIG_VM_VGICV3`) and virtual serial
(`CONFIG_VM_VSERIAL`) devices for VM operation.
Signed-off-by: Guoqi Xie <[email protected]>
You can follow the README file to boot ZVM and VM. Signed-off-by: Guoqi Xie <[email protected]>
1. vpl011: Add a new device driver for the PL011 UART controller. 2. vserial: Add a new device driver for the virtual serial port. Signed-off-by: Guoqi Xie <[email protected]>
Add descritption of running VM on ZVM and how to connect VM console. Signed-off-by: Guoqi Xie <[email protected]>
1. Relase physical cpu when delete vm. 2. Move vcpu_struct to correct place. Signed-off-by: Guoqi Xie <[email protected]>
When delete vm, call the deinit operator to release virtual device resource. Signed-off-by: Guoqi Xie <[email protected]>
add deinit function for release vcpu in vm. Signed-off-by: Guoqi Xie <[email protected]>
Implemented a dynamic semaphore mechanism to synchronize vCPU shutdown in virtual machines. The semaphores are allocated based on the actual number of vCPUs, ensuring proper synchronization during VM deletion. This improves reliability and handles varying vCPU counts efficiently. Signed-off-by: Guoqi Xie <[email protected]>
zephyrbot
added
area: PM CPU ops
area: ARM64
zephyrbot
requested review from
andyross,
carlocaione,
ceolin,
cfriedt,
gdengi,
kartben,
npitre,
peter-mitsis,
povergoing,
SgrrZhf and
TaiJuWu
Whitespace, EOF, copyright, indentation and other issues. Signed-off-by: Guoqi Xie <[email protected]>
|
You want your PR to be against |
|
/close |
you'll want to manually close the PR, if that was your intention :) |
Thank you for your feedback and clarification! I’ll update the PR to target the main branch instead of the 3.7 branch. Additionally, I’ll squash the commits to streamline the commit history and ensure compliance with the required standards. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces ZVM (Zephyr-based Virtual Machine), a Type 1.5 hypervisor based on Zephyr RTOS, which allows running multiple virtual machines (VMs) on ZVM.
The sample provides support for running both Zephyr and Linux VMs on ZVM using qemu_max board. To build and run the ZVM, refer to the documentation in this PR for detailed instructions.
The project is led by Professor Guoqi Xie at Hunan University, China, with contributions from the openEuler SIG-Zephyr team and other collaborators.
For more information, refer to the project documentation and the
ZVM Main Page <https://gitee.com/openeuler/zvm>__.