Darwin: inconsistencies between CI and Runtime

[shivaraj-bh]

CI uses nix flake check, which calls the process-compose app wrapped inside pkgs.runCommand. The apps that run inside runCommand gets a few packages by default, for example readlink from coreutils. This behaviour could make the tests pass in CI but fail on runtime (when ran using nix run), if the service doesn’t pass those packages in its environment.

For more detailed analysis, see: #164 (comment)

[shivaraj-bh]

Is it just darwin? wouldn’t it also affect linux?

Edit: So far, there hasn’t been any issue reported about the inconsistencies happening on Linux but stdenv in Linux could also have packages which the host machine will not during runtime

[srid]

Is this something to do with sandbox being disabled by default on darwin?

[shivaraj-bh]

Is this something to do with sandbox being disabled by default on darwin?

No, it is to do with some packages (belonging to stdenv-darwin) being present in the runCommand’s environment (making the CI pass), while not being present in the environment while nix runing the service. Leading to regressions like:

#163
#164