doc/hooks/versionCheckHook.section.md: compare testers.testVersion

doc/hooks/versionCheckHook.section.md

@@ -2,7 +2,9 @@
 
 This hook adds a `versionCheckPhase` to the [`preInstallCheckHooks`](#ssec-installCheck-phase) that runs the main program of the derivation with a `--help` or `--version` argument, and verifies if the `${version}` string is found in the output.
 
-Here is a typical usage:
+It does so in a clean environment (using `env --ignore-environment --chdir=/`), and it checks for the `${version}` string in both the `stdout` and the `stderr` of the command. It will report to you in the build log the output it received and it will fail the build if it failed to find `${version}`.
+
+Here is a typical usage example:
 
 :::{.example #ex-versioncheckhook-phantom}
 
@@ -27,7 +29,7 @@ stdenv.mkDerivation (finalAttrs: {
 })
 ```
 
-It does so in a clean environment (using `env --ignore-environment`), and it checks for the `${version}` string in both the `stdout` and the `stderr` of the command. It will report to you in the build log the output it received and it will fail the build if it failed to find `${version}`.
+:::
 
 Note that `doInstallCheck` is enabled by default for [`buildPythonPackage`](#buildpythonpackage-function) and [`buildPythonApplication`](#buildpythonapplication-function).
 
@@ -58,3 +60,29 @@ The following attributes control `versionCheckHook`:
 - `postVersionCheck`: Hook that runs after the check is done. Defaults to an empty string.
 
 Note: see also [`testers.testVersion`](#tester-testVersion).
+
+## Comparison between `versionCheckHook` and `testers.testVersion` {#versioncheckhook-comparison-testversion}
+
+The most notable difference between `versionCheckHook` and `testers.testVersion` is that `versionCheckHook` runs as a phase during the build time of the package, while `passthru.tests.versions` executes a whole derivation that depends on the package.
+
+Given the current limitations of Nixpkgs's tooling and CI, `versionCheckHook` fits better in most of typical situations.
+
+Below we tabulate the differences between `versionCheckHook` and `testers.testVersion`.
+
+
+| Item                                   | `versionCheckHook`                                                                                                                                            | `testers.testVersion`                                                                                                                                                                                       |
+|:---------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
+| Customization                          | Besides the attributes described above, the hook provides no other methods for controlling it.                                                                | The tester has access to all feature set of Nix and Nixpkgs, like generating multiple version test derivations from a list of strings, and accessing programs / tools outside the inputs of the derivation. |
+| Execution environment                  | The hook uses `env --ignore-environment --chdir=/` to run the executables in an environment as clean as possible, but there is no way to change its behavior. | The tester executes in an identical environment of a consumer, independent from the building environment.                                                                                                   |
+| Rebuild after modification             | The hook rebuilds the package when the hook is modified, since it is a phase running during the build time of the package.                                    | The tester does not require rebuilding the package, since it is a derivation.                                                                                                                               |
+| Overhead during package building       | Negligible. Although running during the build time of the package, the hook is lean and executes few commands.                                                | Zero, since the tester is a derivation that runs after package building.                                                                                                                                    |
+| Content-addressed-derivation awareness | Since it runs during the building, the hook does not deal with failures happening after building, like rewritings that happen post installation.              | Since it runs after the building, the tester detects failures at this time.                                                                                                                                 |
+| Execution by OfBorg CI tool            | Yes, since the hook is a phase running during the build time of the package.                                                                                  | OfBorg does run the tester for _directly affected packages only_; transitive dependencies are ignored by default, requiring extra commands like `@ofborg build dependency1.tests dependency2.tests ...`.    |
+| Execution by `nixpkgs-review` tool     | Yes, since the hook is a phase running during the build time of the package.                                                                                  | No, since the tool has no support for executing `passthru.tests`. [^1]                                                                                                                                      |
+| Package breakage awareness             | Loud and clear as soon as the hook is reached, since it is a phase running during the build time of the package.                                              | Requires specific command to be noticed, e.g. `nix-build -A pkg.tests.version`, since it is a derivation dependent on the package                                                                           |
+| Transitive package breakage            | Never ever goes unnoticed by `nixpkgs-review`, since the tool executes the transitive dependencies.                                                           | Since human beings are prone to forget the duty of running passthru tests and `nixpkgs-review` has no support for running it, it certainly goes unnoticed.                                                  |
+
+
+[^1]: There is a [pull request proposal](https://github.com/Mic92/nixpkgs-review/pull/397) against [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review)  for an extra flag to run `passthru.tests`.
+
+As said before, given the current limitations of Nixpkgs's tooling and CI demonstrated above, `versionCheckHook` fits better in most of typical situations.