restraint doesn't start running tasks

[wizardofoz123]

The console on a ppc64le system with RHEL-7 says:

Kernel <redacted!> on an ppc64le (hvc0) 
 
<redacted!> login: [   18.219440] restraintd[767]: * Fetching recipe: <redacted!>
[   18.957574] restraintd[767]: * Parsing recipe 
[   18.960519] restraintd[767]: * Running recipe 
[   19.060974] restraintd[767]: * Finished recipe 
[-- MARK -- Tue May  4 15:35:00 2021] 
[-- MARK -- Tue May  4 15:40:00 2021] 
[-- MARK -- Tue May  4 15:45:00 2021] 
[-- MARK -- Tue May  4 15:50:00 2021] 
[-- MARK -- Tue May  4 15:55:00 2021] 
[-- MARK -- Tue May  4 16:00:00 2021] 

It looks like restraint hasn't started running the reservesys task for whatever reason.

This is what the console should say instead:

<redacted!> login: [   13.615253] restraintd[3974]: * Fetching recipe: <redacted!>
[   13.738289] restraintd[3974]: * Parsing recipe 
[   13.886583] restraintd[3974]: * Running recipe 
[   13.889227] restraintd[3974]: ** Fetching task: 125724253 [/mnt/tests/github.com/beaker-project/beaker-core-tasks/archive/master.tar.gz/reservesys] 
[   15.669452] restraintd[3974]: ** Preparing metadata 
[   15.795899] restraintd[3974]: ** Refreshing peer role hostnames: Retries 0 
[   15.915267] restraintd[3974]: ** Updating env vars 
[   15.922941] restraintd[3974]: ** Updating external watchdog: 88200 seconds 
[   16.129993] restraintd[3974]: ** Installing dependencies 
[   28.357444] random: crng init done 
[-- MARK -- Mon May 10 09:45:00 2021] 
[  155.069269] restraintd[3974]: *** Current Time: Mon May 10 05:45:47 2021  Localwatchdog at:  * Disabled! * 
[  155.139951] restraintd[3974]: ** Running task: 125724253 [/distribution/reservesys] 

Server RPM: restraint-0.3.3-1.el7bkr.ppc64le
Client RPM: (not used, issue happens with reservesys , before actual tasks are run).

[wizardofoz123]

Adding internal beaker job numbers: 5339539 5339473 5339366

[StykMartin]

@cbouchar I guess you will need to check. I don't have time for upstream at this moment + we have internal numbers here now.

[cbouchar]

@wizardofoz123 I looked at your job 5339539. I removed the group=, cloned and reran it (ref: job 5372645) and it's working fine now. One thing I'd like to mention though, I recommend you have your fetches for downstream beaker as RH users can't count on upstream beaker code in github. Please refer to my job 5373724 for sample of the fetch operation.

[wizardofoz123]

@cbouchar I consider this to be a bug that doesn't reproduce well. It's not the highest priority of all issues here, but I don't consider 1x clone / rerun enough to say something is working fine. Let's keep this open so we have reference once we hit this again.