gui rpc drop keystroke flags like repeat

[iacore]

How to file a helpful issue

Qubes OS release

r4.1

Brief summary

Key repeat is usable in dom0, but the Qubes gui protocol gui-common doesn't include flags for keys.

This makes games extremely confused, because they can't differentiate between "pressing a key repeatedly" and "holding down a key".

Steps to reproduce

Run xinput test-xi2 in dom0. Hold any key. You will see something like this in stdout:

EVENT type 2 (KeyPress)
    device: 3 (3)
    detail: 113
    flags: repeat

Run xinput test-xi2 in a VM other than dom0. Hold any key. The flags are empty.

EVENT type 2 (KeyPress)
    device: 3 (3)
    detail: 113
    flags:

Expected behavior

X input events should include flags like repeat too.

[iacore]

In qubes-gui-protocol.h, seems like there is no flags here. I could really use some comments here.

struct msg_keypress {
    uint32_t type;
    uint32_t x;
    uint32_t y;
    uint32_t state;
    uint32_t keycode;
};

[DemiMarie]

Ouch. Thanks for reporting this.

@marmarek is this something that can be fixed in an R4.1 update? On the one hand it does require a protocol change, but on the other hand it seems to be causing actual problems for at least one user.

[iacore]

Maybe...

• create a new event KEYPRESS2
• make gui rpc server emit both KEYPRESS and KEYPRESS2
• make new rpc client only handle KEYPRESS2

[iacore]

Mouse scroll wheel event also has flags. Has not met problem with this so far.

EVENT type 4 (ButtonPress)
    device: 25 (25)
    detail: 5
    flags: emulated

[iacore]

There are too many RawKeyPress events, one for each repeated key event. There should only be one when the key is initially pressed.

[iacore]

Is there a plan to use TLV encoding for qubes events? That way, we can extend existing events with new fields in the future by simply adding new fields.

[DemiMarie]

@locriacyber Not that I am aware of. The encoding is deliberately kept as simple as possible to minimize attack surface. The protocol does support version negotiation, though, and the agent ignores events it is not aware of.

[iacore]

Ok, I found a partial solution. I turned off keyboard repeat in "Keyboard" application in dom0, now every client VM seem to generate the correct key repeat events themselves.

Still can't turn off emulated events for mouse wheels. Also, I lose key repeat in dom0 since I turned it off.

[iacore]

Also, mouse wheel motion events are never emitted on client VMs. Need to fix this as well.

[iacore]

Also, mouse wheel motion events are never emitted on client VMs. Need to fix this as well.

I'm not using QubesOS anymore, so how do I fix this? I don't think I can.

@andrewdavidwong @DemiMarie

[DemiMarie]

I’m not sure this is reasonably fixable with the current GUI protocol. The problem is not the quality of the PR or the complexity of the fix, but rather the lack of anyone with sufficient knowledge of X11 to review the changes. Most who worked on X11 have since moved on to Wayland, so it is very difficult to get help for X11 problems.

@iacore: Which documentation did you use to figure outt how XInput worked?

[iacore]

I’m not sure this is reasonably fixable with the current GUI protocol.

I agree. Given how xserver works, state synchronization with base X11 + xinput events feels like a meme. Better to patch the xserver to sync the internal states directly.

@iacore: Which documentation did you use to figure outt how XInput worked?

None.

I ran the following two commands and tried to make the output match.

xinput test-xi2
xinput test-xi2 --root

[DemiMarie]

I’m not sure this is reasonably fixable with the current GUI protocol.

I agree. Given how xserver works, state synchronization with base X11 + xinput events feels like a meme. Better to patch the xserver to sync the internal states directly.

Which X server would need to be patched? Agent (guest) side or daemon (host) side?

@iacore: Which documentation did you use to figure outt how XInput worked?

None.

I ran the following two commands and tried to make the output match.

xinput test-xi2
xinput test-xi2 --root

I see!

[iacore]

Which X server would need to be patched? Agent (guest) side or daemon (host) side?

no idea. i didn't look into xserver source code at all at that time.

Better to patch the xserver to sync the internal states directly.

"Better" as in "better than running an X11 client in domU".

i forgot what the Qubes OS gui architecture is like. i remembering window focus event and keyboard input event arrive in different order or just don't arrive at all. i fixed the alt+tab to work fairly consistently, but overall using Qubes OS still feels weird.

I forgot a lot about how i did what, so take a look at QubesOS/qubes-gui-daemon#115 and use xinput test-xi2 to see if the output in a user VM is acceptable for you.