produce a reproducible index.tar.gz

[hannesm]

Dear Madam or Sir,

in the light of #5553, and from a security perspective, wouldn't it be great if we could rely on "taking a specific git commit from the opam-repository, we'll be able to reproduce the index.tar.gz served to clients".

This would improve the security since at the moment, it is tough for a client to check that the index.tar.gz contains all the opam files as advertised.

Now, the index.tar.gz is generated as far as I can tell by repository/opamHTTP.ml:

let make_index_tar_gz repo_root =
  OpamFilename.in_dir repo_root (fun () ->
    let to_include = [ "version"; "packages"; "repo" ] in
    match List.filter Sys.file_exists to_include with
    | [] -> ()
    | d  -> OpamSystem.command ("tar" :: "czhf" :: "index.tar.gz" :: "--exclude=.git*" :: d)
  )

When doing releases to the opam-repository, we've faced this issue several times, and gladly b0 (previous topkg) provide a minimal tar implementation to do that: https://github.com/b0-system/b0/blob/master/src/std/b0_tar.ml

Using that approach would immediately lead to a reproducible tar archive. The timestamp of all files can easily be set to the last git commit.

Is that something that would be accepted as a PR (in case you don't have the resources to work on it)?

EDIT: of course the existing "tar" can be used to produce a reproducible tarball as well (unfortunately this relies on the tar implementation -- e.g. GNU tar has a lengthy explanation https://www.gnu.org/software/tar/manual/html_node/Reproducibility.html).

[kit-ty-kate]

Is that something that would be accepted as a PR (in case you don't have the resources to work on it)?

sure, that would be welcome. I don't think that's something high priority at the moment so i don't think i would personally work on it right away.

Although as for the implementation of it, it would probably be better to use ocaml-tar instead if that fits the same goal, as we already plan to use it to fix #5741