Skip to content

Commit

Permalink
doc/hooks/versionCheckHook.section.md: add comparison with testers.te…
Browse files Browse the repository at this point in the history
…stVersion
  • Loading branch information
AndersonTorres committed Nov 3, 2024
1 parent c8f89b3 commit be53783
Showing 1 changed file with 32 additions and 0 deletions.
32 changes: 32 additions & 0 deletions doc/hooks/versionCheckHook.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,35 @@ The following attributes control `versionCheckHook`:
- `postVersionCheck`: Hook that runs after the check is done. Defaults to an empty string.

Note: see also [`testers.testVersion`](#tester-testVersion).

## Comparison between `versionCheckHook` and `testers.testVersion` {#versioncheckhook-comparison-testversion}

The most notable difference between `versionCheckHook` and `testers.testVersion` is that `versionCheckHook` runs as a phase during the build time of the package, while `passthru.tests.versions` executes a whole derivation that depends on the package.

Below we list some advantages of `testers.testVersion` over `versionCheckHook`:

- `testers.testVersion` is a derivation, having access to the whole features provided by Nix and Nixpkgs
- E.g. extra tools, customized programs etc.
- `testers.testVersion` is executed in an identical environment of a consumer would, independent of the environment it was built.
- `testers.testVersion` can be executed without rebuilding the package, saving time especially when the rebuild takes too long.
- `testers.testVersion` do not add overhead to each build, since it runs after the build, not during it.
- Arguably, the overhead of `versionCheckHook` is negligible, so this point is not much a difference.g
- For content-addressed derivations, `testers.testVersion` runs __after__ rewriting instead of _before_, catching cases when the rewriting would break the executable.

Below we will list some advantages of `versionCheckHook` over `testers.testVersion`:

- Sometimes a package triggers no errors while being built - especially when the upstream provides no unit tests-, however it fails at runtime. When such a package is not used in a regular basis, it leaves room for a silent breakage lurking in the system / profile configuration, not being noticed for long periods of time.

Although `passthru.tests` has the same purpose, it is more prone to be forgotten and/or ignored by human beings; on the other hand, since `versionCheckHook` will be executed as a regular phase during the build time of the derivation, it will not be ignored, working around this limitation.

- When a pull request is opened against Nixpkgs repository, [ofborg](https://github.com/NixOS/ofborg)'s CI will automatically run `passthru.tests` attributes for the packages that are [directly changed by your PR (according to your commits' messages)](https://github.com/NixOS/ofborg?tab=readme-ov-file#automatic-building).

However, ofBorg does not run the `passthru.tests` attributes for _transitive dependencies_ of those packages. To execute them, commands like [`@ofborg build dependency1.tests dependency2.tests ...`](https://github.com/NixOS/ofborg?tab=readme-ov-file#build) are needed.

On the other hand, `versionCheckHook` will be executed as a regular phase during the build time of the derivation, therefore it will be indirectly executed by ofBorg, working around this limitation.

- Since [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) does not execute `passthru.tests` attributes, it will not execute `passthru.tests.version`.

On the other hand, `versionCheckHook` will be executed as a regular phase during the build time of the derivation, therefore it will be indirectly executed by `nixpkgs-review`, working around this limitation.

Given the current limitations of Nixpkgs's tooling and CI demonstrated above, `versionCheckHook` fits better in most of typical situations.

0 comments on commit be53783

Please sign in to comment.