virtio-gpu device #598

dorindabassey · 2024-01-08T14:05:02Z

Hi,

I and @MatiasVara are interested in developing a virtio-gpu device for rust-vmm.
We have just started working on an initial skeleton. As soon as we have something ready, we'll open a Draft PR.
The idea is to have a rust based virtio-gpu device working. Marc-Andre Lureau will be assisting us in this effort, given that he was involved in the development of the vhost-user-gpu C implementation in QEMU.

stsquad · 2024-01-08T16:42:22Z

Hi @dorindabassey et all,

We are certainly interested in a virtio-gpu backend for vhost-device. We are currently working towards a demo at our next Connect to bring together Arm, Xen and virtio-gpu (venus/vulkan) on the AVA platform. We are working with QEMU in the first instance but longer term having a separate rendering process makes sense, especially on Xen. What are your thoughts about supporting virgl and venus vs other gfxstream transports like wayland?

dorindabassey · 2024-01-09T16:08:53Z

@stsquad currently we are starting looking at virgl first to try to match the existing vhost-user-gpu in qemu, we would like to support all those other features but we will see later how the project evolves.

Are there somethings you think we should consider while working on this?

MatiasVara · 2024-01-15T15:17:37Z

I add @elmarco to the conversation.

DemiMarie · 2024-03-06T19:51:07Z

@dorindabassey How will this compare to the existing virtio-GPU device implemented in crosvm? crosvm already has vhost-user support. Blob objects (needed for Vulkan and native contexts) require non-standard vhost-user messages, but I don’t think that an alternative implementation would be able to avoid this.

Also paging @alyssais who is using this in Spectrum OS.

dorindabassey · 2024-03-07T15:11:40Z

Hi @DemiMarie, the vhost-device-gpu implementation we are working on is largely based on crosvm virtio-gpu device, given that we are using rutabaga+virglrenderer. We are not doing anything special other than how the display is being handled, we are going to be use QEMU display support for this. I hope that answers your question.

dorindabassey · 2024-03-07T15:24:54Z

update: adding @mtjhrc who is also working on the vhost-device-gpu with us.

alyssais · 2024-03-07T16:18:28Z

We are not doing anything special other than how the display is being handled, we are going to be use QEMU display support for this.

This is surprising to me, compared to using crosvm's display code in the backend. If this is a vhost-user backend, how does the device interact with the QEMU display code? What happens if the vhost-user frontend is not QEMU?

DemiMarie · 2024-03-07T19:58:40Z

@alyssais, @QubesOS, and Chrome OS all need each window in the guest to be a separate window on the host. Therefore, they are or will be using Wayland passthrough, rather than QEMU’s display code.

mtjhrc · 2024-03-07T21:30:34Z

We are not doing anything special other than how the display is being handled, we are going to be use QEMU display support for this.

This is surprising to me, compared to using crosvm's display code in the backend. If this is a vhost-user backend, how does the device interact with the QEMU display code? What happens if the vhost-user frontend is not QEMU?

It interacts using commands on socket obtained from VHOST_USER_GPU_SET_SOCKET , it is documented here: https://www.qemu.org/docs/master/interop/vhost-user-gpu.html

If the fronted doesn't support VHOST_USER_GPU_SET_SOCKET, then it wouldn't work, but we could also support Wayland passthrough too, I guess.

alyssais · 2024-03-08T14:55:02Z

It interacts using commands on socket obtained from VHOST_USER_GPU_SET_SOCKET , it is documented here: https://www.qemu.org/docs/master/interop/vhost-user-gpu.html

Ah, I see, so this is an implementation of QEMU's vhost-user-gpu protocol (which has the frontend do the rendering), not crosvm's (where the backend does it). It's very unfortunate that there's no consistency/standardisation between the two…

DemiMarie · 2024-03-08T18:02:12Z

Having the backend do the rendering is highly desirable from a security perspective, as it allows the frontend to be sandboxed. The backend must be quite privileged as it can directly access the host’s GPU drivers.

This program is a vhost-user backend daemon that provides VIRTIO GPU device emulation as specified in the VIRTIO Spec v.1.2 https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html This crate utilizes the rutabaga crate Imported from crosvm This crate depends on this PR[rust-vmm/vhost#239] that implements support for QEMU's vhost-user-gpu protocol. This crate supports both gfxstream and virglrenderer library either of these libraries can be used by simply changing the `renderer` command line option. This crate also includes some modifications from libkrun virtio-gpu device https://github.com/containers/libkrun/tree/main/src/devices/src/virtio/gpu This device can be tested following the instructions explained in the README.md file under staging/vhost-device-gpu/. Fixes: rust-vmm#598 Co-authored-by: Dorinda Bassey <[email protected]> Co-authored-by: Matej Hrica <[email protected]> Signed-off-by: Dorinda Bassey <[email protected]> Signed-off-by: Matej Hrica <[email protected]>

This program is a vhost-user backend daemon that provides VIRTIO GPU device emulation as specified in the VIRTIO Spec v.1.2 https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html This crate utilizes the rutabaga crate from crosvm with some minor modification to rutabaga crate to fix compilation. This crate depends on this PR[rust-vmm/vhost#239] that implements support for QEMU's vhost-user-gpu protocol. This crate supports both gfxstream and virglrenderer library either of these libraries can be used by simply changing the `renderer` command line option. This crate also includes some modifications from libkrun virtio-gpu device https://github.com/containers/libkrun/tree/main/src/devices/src/virtio/gpu This device can be tested following the instructions explained in the README.md file under staging/vhost-device-gpu/. Fixes: rust-vmm#598 Co-authored-by: Dorinda Bassey <[email protected]> Co-authored-by: Matej Hrica <[email protected]> Signed-off-by: Dorinda Bassey <[email protected]> Signed-off-by: Matej Hrica <[email protected]>

This program is a vhost-user backend daemon that provides VIRTIO GPU device emulation as specified in the VIRTIO Spec v.1.2 https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html This crate utilizes the rutabaga crate from crosvm with some minor modification to rutabaga crate to fix compilation. This crate depends on this PR[rust-vmm/vhost#239] that implements support for QEMU's vhost-user-gpu protocol. This crate supports both gfxstream and virglrenderer library either of these libraries can be used by simply changing the `renderer` command line option. This crate also includes some modifications from libkrun virtio-gpu device https://github.com/containers/libkrun/tree/main/src/devices/src/virtio/gpu Fixes: rust-vmm#598 Co-authored-by: Dorinda Bassey <[email protected]> Co-authored-by: Matej Hrica <[email protected]> Signed-off-by: Dorinda Bassey <[email protected]> Signed-off-by: Matej Hrica <[email protected]> Signed-off-by: Dorinda Bassey <[email protected]>

This program is a vhost-user backend daemon that provides VIRTIO GPU device emulation as specified in the VIRTIO Spec v.1.2 https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html This crate utilizes the rutabaga crate from crosvm with some minor modification to rutabaga crate to fix compilation. This crate depends on this PR[rust-vmm/vhost#239] that implements support for QEMU's vhost-user-gpu protocol. This crate supports both gfxstream and virglrenderer library either of these libraries can be used by simply changing the `renderer` command line option. This crate also includes some modifications from libkrun virtio-gpu device https://github.com/containers/libkrun/tree/main/src/devices/src/virtio/gpu Fixes: rust-vmm#598 Co-authored-by: Dorinda Bassey <[email protected]> Co-authored-by: Matej Hrica <[email protected]> Signed-off-by: Dorinda Bassey <[email protected]> Signed-off-by: Matej Hrica <[email protected]>

This program is a vhost-user backend daemon that provides VIRTIO GPU device emulation as specified in the VIRTIO Spec v.1.2 https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html This crate utilizes the rutabaga crate from crosvm with some minor modification to rutabaga crate to fix compilation. This crate depends on this PR[rust-vmm/vhost#239] that implements support for QEMU's vhost-user-gpu protocol. This device uses the rutabaga_gfx crate to offer two rendering backends: 1. Virglrenderer: - Rutabaga translates OpenGL API and Vulkan calls to an intermediate representation and allows for OpenGL acceleration on the host. 2. Gfxstream: - GLES and Vulkan calls are forwarded to the host. These backends can be used by simply changing the `--gpu-mode` command line option. This crate also includes some modifications from libkrun virtio-gpu device https://github.com/containers/libkrun/tree/main/src/devices/src/virtio/gpu Fixes: rust-vmm#598 Co-authored-by: Dorinda Bassey <[email protected]> Co-authored-by: Matej Hrica <[email protected]> Signed-off-by: Dorinda Bassey <[email protected]> Signed-off-by: Matej Hrica <[email protected]>

mtjhrc mentioned this issue Mar 12, 2024

Support for VHOST_USER_GPU_SET_SOCKET, and the protocol on this socket rust-vmm/vhost#236

Open

mtjhrc mentioned this issue Apr 18, 2024

Add support for VHOST_USER_GPU_SET_SOCKET rust-vmm/vhost#239

Merged

4 tasks

dorindabassey linked a pull request Jun 6, 2024 that will close this issue

vhost-device-gpu: Add Initial Implementation #668

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

virtio-gpu device #598

virtio-gpu device #598

dorindabassey commented Jan 8, 2024

stsquad commented Jan 8, 2024

dorindabassey commented Jan 9, 2024 •

edited

Loading

MatiasVara commented Jan 15, 2024

DemiMarie commented Mar 6, 2024

dorindabassey commented Mar 7, 2024

dorindabassey commented Mar 7, 2024

alyssais commented Mar 7, 2024

DemiMarie commented Mar 7, 2024 •

edited

Loading

mtjhrc commented Mar 7, 2024

alyssais commented Mar 8, 2024 •

edited

Loading

DemiMarie commented Mar 8, 2024

virtio-gpu device #598

virtio-gpu device #598

Comments

dorindabassey commented Jan 8, 2024

stsquad commented Jan 8, 2024

dorindabassey commented Jan 9, 2024 • edited Loading

MatiasVara commented Jan 15, 2024

DemiMarie commented Mar 6, 2024

dorindabassey commented Mar 7, 2024

dorindabassey commented Mar 7, 2024

alyssais commented Mar 7, 2024

DemiMarie commented Mar 7, 2024 • edited Loading

mtjhrc commented Mar 7, 2024

alyssais commented Mar 8, 2024 • edited Loading

DemiMarie commented Mar 8, 2024

dorindabassey commented Jan 9, 2024 •

edited

Loading

DemiMarie commented Mar 7, 2024 •

edited

Loading

alyssais commented Mar 8, 2024 •

edited

Loading