clevis-decrypt-sss fails to kill all child processes (and their respective children)

[m-ueberall]

For an example, see olastor/clevis-pin-fido2#3 (however, this is not a problem specific to this pin; clevis-decrypt-sss should always ensure that no (grand)child processes are left behind).
As demonstrated below (using two terminals), threshold t=1 is reached and the decryption succeeds, but one of the FIDO2 keys is still being queried until the key-specific timeout kicks in while clevis-decrypt-sss already terminated:

[2024-04-12T13:16:01+0200] root@ubuntu:/tmp# echo "Hello, world." | clevis encrypt sss '{"t": 1, "pins": {"fido2": [{"device": "/dev/input/by-id/yubikey_12345678"}, {"device": "/dev/input/by-id/yubikey_87654321"}]}}' >test01.jwe
clevis-encrypt-fido2: Please insert your specified FIDO2 token /dev/input/by-id/yubikey_12345678
clevis-encrypt-fido2: Please insert your specified FIDO2 token /dev/input/by-id/yubikey_87654321
[…]
[2024-04-12T13:17:31+0200] root@ubuntu:/tmp# clevis decrypt < test01.jwe
Hello, world.
[2024-04-12T13:17:55+0200] root@ubuntu:/tmp# fido2-assert: fido_dev_get_assert: FIDO_ERR_ACTION_TIMEOUT

[2024-04-12T13:17:33+0200] root@ubuntu:/tmp# ps axu | grep -E 'clevis|fido2'
root     1183736  0.0  0.0  18680  3580 pts/7    S+   13:17   0:00 /bin/bash -e /usr/bin/clevis-decrypt
root     1183741  0.0  0.0   7800  5084 pts/7    S+   13:17   0:00 /usr/bin/clevis-decrypt-sss
root     1183742  0.0  0.0  18680  3588 pts/7    S+   13:17   0:00 /bin/bash -e /usr/bin/clevis-decrypt
root     1183743  0.0  0.0  18680  3492 pts/7    S+   13:17   0:00 /bin/bash -e /usr/bin/clevis-decrypt
root     1183752  0.0  0.0  18680  3604 pts/7    S+   13:17   0:00 /bin/bash /usr/bin/clevis-decrypt-fido2
root     1183753  0.0  0.0  18680  3584 pts/7    S+   13:17   0:00 /bin/bash /usr/bin/clevis-decrypt-fido2
root     1183802  0.0  0.0  18680  1716 pts/7    S+   13:17   0:00 /bin/bash /usr/bin/clevis-decrypt-fido2
root     1183804  0.3  0.0   8072  5668 pts/7    S+   13:17   0:00 fido2-assert -G -h -t up=true -t pin=false /dev/input/by-id/yubikey_12345678
root     1183808  0.0  0.0  18680  1640 pts/7    S+   13:17   0:00 /bin/bash /usr/bin/clevis-decrypt-fido2
root     1183811  0.3  0.0   8072  5788 pts/7    S+   13:17   0:00 fido2-assert -G -h -t up=true -t pin=false /dev/input/by-id/yubikey_87654321
root     1183819  0.0  0.0  18040  2316 pts/5    S+   13:17   0:00 grep --color=auto -E clevis|fido2
[2024-04-12T13:17:46+0200] root@ubuntu:/tmp# pstree -p 1183741
clevis-decrypt-(1183741)─┬─clevis-decrypt(1183742)───clevis-decrypt-(1183752)───clevis-decrypt-(1183802)─┬─base64(1183807)
                         │                                                                               ├─fido2-assert(1183804)
                         │                                                                               ├─head(1183805)
                         │                                                                               ├─jose(1183809)
                         │                                                                               └─tail(1183806)
                         └─clevis-decrypt(1183743)───clevis-decrypt-(1183753)───clevis-decrypt-(1183808)─┬─base64(1183814)
                                                                                                         ├─fido2-assert(1183811)
                                                                                                         ├─head(1183812)
                                                                                                         ├─jose(1183815)
                                                                                                         └─tail(1183813)
[2024-04-12T13:17:52+0200] root@ubuntu:/tmp# ps axu | grep -E 'clevis|fido2'
root     1183753  0.0  0.0  18680  3584 pts/7    S    13:17   0:00 /bin/bash /usr/bin/clevis-decrypt-fido2
root     1183808  0.0  0.0  18680  1640 pts/7    S    13:17   0:00 /bin/bash /usr/bin/clevis-decrypt-fido2
root     1183811  0.0  0.0   8072  5788 pts/7    S    13:17   0:00 fido2-assert -G -h -t up=true -t pin=false /dev/input/by-id/yubikey_87654321
root     1183865  0.0  0.0  18040  2304 pts/5    S+   13:18   0:00 grep --color=auto -E clevis|fido2
[2024-04-12T13:18:03+0200] root@ubuntu:/tmp# pstree -c 1183753
clevis-decrypt-───clevis-decrypt-─┬─base64
                                  ├─fido2-assert
                                  ├─head
                                  ├─jose
                                  └─tail
[2024-04-12T13:18:09+0200] root@ubuntu:/tmp# 

[sarroutbi]

Hello @m-ueberall . Could you reproduce this issue with "no FIDO2" pin? I was not able to do so ... but I might be missing something in my scenario

[m-ueberall]

@sarroutbi : You should be able to reproduce the above with the tang pin contained in this repository as follows:

[2024-04-12T22:31:53+0200] root@ubuntu:/tmp# echo "Hello, world." | clevis encrypt sss '{"t": 1, "pins": {"tang": [{"url": "http://example.org:18888"}, {"url": "http://example.org:18889"}]}}' >test03.jwe
The advertisement contains the following signing keys:

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX1

Do you wish to trust these keys? [ynYN] Y
The advertisement contains the following signing keys:

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX2

Do you wish to trust these keys? [ynYN] Y
[2024-04-12T22:32:42+0200] root@ubuntu:/tmp#
[2024-04-12T22:34:58+0200] root@ubuntu:/tmp# clevis decrypt < test03.jwe
Hello, world.

The first run above worked as expected. Let's create a situation where one of the Tang servers is unable to respond in a timely manner, i.e., before the decryption has taken place using the other pin/server. To keep things as simple as possible, we just silently prevent it from receiving the request at all:

[2024-04-12T22:35:49+0200] [email protected]:/tmp# iptables -I INPUT -p tcp -m tcp --dport 18888 -j DROP
[2024-04-12T22:36:32+0200] [email protected]:/tmp# ip6tables -I INPUT -p tcp -m tcp --dport 18888 -j DROP
[2024-04-12T22:36:36+0200] [email protected]:/tmp#

Now we rerun the test (no pstree output this time, but it should be obvious that all the processes shown below are related to our second run on the client); the only challenge here is to select the correct value for the first call to sleep if you want to do this in a single terminal, because everything usually happens very fast if both client and servers are in the same local network:

[2024-04-12T22:41:13+0200] root@ubuntu:/tmp# clevis decrypt < test03.jwe & sleep 0.25; ps axu | grep -E 'clevis|curl'; sleep 5; echo "----"; ps axu | grep -E 'clevis|curl'
[1] 1244745
root     1244745  0.0  0.0   4416  3168 pts/5    S    22:41   0:00 /bin/bash -e /usr/bin/clevis-decrypt
root     1244751  0.0  0.0   7800  5008 pts/5    S    22:41   0:00 /usr/bin/clevis-decrypt-sss
root     1244752  0.0  0.0   4416  3052 pts/5    S    22:41   0:00 /bin/bash -e /usr/bin/clevis-decrypt
root     1244753  0.0  0.0   4416  3200 pts/5    S    22:41   0:00 /bin/bash -e /usr/bin/clevis-decrypt
root     1244762  0.0  0.0   4416  3132 pts/5    S    22:41   0:00 /bin/bash -e /usr/bin/clevis-decrypt-tang
root     1244763  0.0  0.0   4416  3140 pts/5    S    22:41   0:00 /bin/bash -e /usr/bin/clevis-decrypt-tang
root     1244802  0.0  0.0   4416  1876 pts/5    S    22:41   0:00 /bin/bash -e /usr/bin/clevis-decrypt-tang
root     1244803  0.0  0.0   4416  1820 pts/5    S    22:41   0:00 /bin/bash -e /usr/bin/clevis-decrypt-tang
root     1244804  0.0  0.0  96720  9164 pts/5    S    22:41   0:00 curl -sfg -X POST -H Content-Type: application/jwk+json --data-binary @- http://example.org:18889/rec/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX2
root     1244805  0.0  0.0  96720  9316 pts/5    S    22:41   0:00 curl -sfg -X POST -H Content-Type: application/jwk+json --data-binary @- http://example.org:18888/rec/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX1
root     1244809  0.0  0.0   3624  1620 pts/5    S+   22:41   0:00 grep --color=auto -E clevis|curl
Hello, world.
[1]+  Done                    clevis decrypt < test03.jwe
----
root     1244763  0.0  0.0   4416  3140 pts/5    S    22:41   0:00 /bin/bash -e /usr/bin/clevis-decrypt-tang
root     1244803  0.0  0.0   4416  1820 pts/5    S    22:41   0:00 /bin/bash -e /usr/bin/clevis-decrypt-tang
root     1244805  0.0  0.0  96720  9316 pts/5    S    22:41   0:00 curl -sfg -X POST -H Content-Type: application/jwk+json --data-binary @- http://example.org:18888/rec/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX1
root     1244824  0.0  0.0   3624  1688 pts/5    S+   22:41   0:00 grep --color=auto -E clevis|curl
[2024-04-12T22:41:20+0200] root@ubuntu:/tmp# Error communicating with server http://example.org:18888

[2024-04-12T22:43:29+0200] root@ubuntu:/tmp#

In the next to last line above, I simply hit 'Enter' to get a current timestamp; you can see that it took more than 120 seconds for the remaining clevis-decrypt-tang instance and its children to terminate on its own instead of having been killed by clevis-decrypt[-sss].

Don't forget to undo the artificially introduced problem:

[2024-04-12T22:49:02+0200] [email protected]:/tmp# iptables -D INPUT 1
[2024-04-12T22:49:12+0200] [email protected]:/tmp# ip6tables -D INPUT 1
[2024-04-12T22:49:16+0200] [email protected]:/tmp#