linux static build juvix version shows UNKNOWN for hash and date #2294

paulcadman · 2023-08-11T15:50:29Z

You can clearly see the issue by looking at the nightly-build release:

https://github.com/anoma/juvix-nightly-builds/releases/tag/nightly-2023-08-11-0.4.2-UNKNOWN

The version string for the published Juvix binary is 0.4.2-UNKNOWN, instead of 0.4.2-<hash>.

Cause?

After merging #2275 we run the linux static build inside glcr.b-data.ch/ghc/ghc-musl:9.4.5. For some reason the gitrev library cannot detect the git info in the Juvix checkout as part of the build, and is substituting UNKNOWN instead.

I cannot reproduce this by running the build locally in the ghc-musl:9.4.5 docker container (i.e when I do a manual build, the version git information is populated correctly). So the bug may come from a bad interaction between the docker container an the checkout action in the GitHub action build.

The text was updated successfully, but these errors were encountered:

jonaprieto · 2023-08-15T22:57:16Z

It's probably because we're not fetching all the git info. To fetch all history and the tags, I'd add the following to the nightly-builds ci to corresponding places, e.g:

...
    steps:
      - name: checkout code
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
          ...
...

paulcadman · 2023-08-21T11:25:07Z

It's probably because we're not fetching all the git info.

Unfortunately the fetch-depth trick doesn't work https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5924999532/job/16063576847

benz0li · 2023-08-21T14:16:32Z

Could it be that https://github.com/anoma/juvix-nightly-builds/blob/976dae867348b7bf388b501222b66616b8feda58/.github/workflows/release-nightly.yaml#L6-L9 + https://github.com/anoma/juvix-nightly-builds/blob/976dae867348b7bf388b501222b66616b8feda58/.github/workflows/release-nightly.yaml#L29 does not exactly the same as https://github.com/anoma/juvix-nightly-builds/blob/859869e0e67ead7bf88cb9e5eb82e102025a581d/.github/workflows/release-nightly.yaml#L21?

I know that would be weird. But you never know...

(Not only the image was changed in In anoma/juvix-nightly-builds@d0dbb42, but also these lines.)

paulcadman · 2023-08-21T15:57:27Z

Could it be that https://github.com/anoma/juvix-nightly-builds/blob/976dae867348b7bf388b501222b66616b8feda58/.github/workflows/release-nightly.yaml#L6-L9 + https://github.com/anoma/juvix-nightly-builds/blob/976dae867348b7bf388b501222b66616b8feda58/.github/workflows/release-nightly.yaml#L29 does not exactly the same as https://github.com/anoma/juvix-nightly-builds/blob/859869e0e67ead7bf88cb9e5eb82e102025a581d/.github/workflows/release-nightly.yaml#L21?

I know that would be weird. But you never know...

(Not only the image was changed in In anoma/juvix-nightly-builds@d0dbb42, but also these lines.)

Thanks for looking at this. My understanding is that ${{ github.event.inputs.ref }} would become main in the default case (i.e no input ref is specified). Do you have a different understanding, as this could indeed be the issue if so.

benz0li · 2023-08-21T16:18:30Z

Do you have a different understanding, as this could indeed be the issue if so.

No.

${{ inputs.ref }} should be sufficient, though. Single or double quotes does not matter. But you should define the type.

on:
  workflow_dispatch:
    inputs:
      ref:
        description: "the repository ref to build"
        required: true
        default: "main"
        type: string

I do not use GitHub Actions myself. I am more of a GitLab guy...

benz0li · 2023-08-21T17:24:04Z

See the difference:

https://github.com/anoma/juvix-nightly-builds/actions/runs/5828443441/job/15806173529#step:3:27 vs https://github.com/anoma/juvix-nightly-builds/actions/runs/5915039870/job/16040920936#step:3:28

paulcadman · 2023-08-21T17:30:34Z

See the difference:

https://github.com/anoma/juvix-nightly-builds/actions/runs/5828443441/job/15806173529#step:3:27 vs https://github.com/anoma/juvix-nightly-builds/actions/runs/5915039870/job/16040920936#step:3:28

You mean in the second one there's a warning about default branch during git init? I think this can be explained because the second build uses a newer version of the git client. As far as I can see they both checkout the correct ref in the "checking out the ref" step. However the difference in git version may account for the UNKNOWN error in itself. I can try locally with git version 2.38.4 to see if I can reproduce the issue with this git version and the haskell gitrev library, I am using git 2.39.1 on my machine.

benz0li · 2023-08-21T17:32:56Z

Simply revert the workflow_dispatch part to the old state and check, if it works again.

If not, there might be something missing in my image.

benz0li · 2023-08-21T17:48:50Z

Related to acfoltzer/gitrev#23?

benz0li · 2023-08-21T18:02:57Z

Solution is maybe haskell/cabal#7355 (comment).

In this case adding

extra-source-files: ../.git/HEAD
                  , ../.git/refs/heads/main

to the cabal file.

paulcadman · 2023-08-21T18:21:50Z

Simply revert the workflow_dispatch part to the old state and check, if it works again.

If not, there might be something missing in my image.

Good idea - I will try that.

We would like to test to see if this is causing an issue with the gitrev dependency. See anoma/juvix#2294

paulcadman · 2023-08-21T18:33:15Z

Solution is maybe haskell/cabal#7355 (comment).

In this case adding
extra-source-files: ../.git/HEAD
                  , ../.git/refs/heads/main
to the cabal file.

This an issue with cabal install sandboxed builds, the .git folder is not available. In some cases (e.g for local development builds) this particular set of extra source files is not sufficient. This does not affect stack install. This is the reason why we must use stack for releases.

benz0li · 2023-08-22T06:50:51Z

Still the same...

Two things:

jobs.<job_id>.steps[*].shell: bash vs unspecified
jobs.<job_id>.steps[*].env: Setting LANG required?

There are no locale-related ENV set in the container (see Dockerfile).

benz0li · 2023-08-22T07:07:42Z

P.S.: The following images are more recent and already include an unsupported (= self-built), statically linked binary of Stack (v2.11.1):

glcr.b-data.ch/ghc/ghc-musl:9.6.2
glcr.b-data.ch/ghc/ghc-musl:9.4.6
glcr.b-data.ch/ghc/ghc-musl:9.2.8

P.P.S.: The Haskell Tool Stack switched to glcr.b-data.ch/ghc/ghc-musl:9.4.6 to build Stack.

paulcadman · 2023-08-22T07:13:08Z

P.S.: The following images are more recent and already include an unsupported (= self-built), statically linked binary of Stack (v2.11.1):

glcr.b-data.ch/ghc/ghc-musl:9.6.2

glcr.b-data.ch/ghc/ghc-musl:9.4.6

glcr.b-data.ch/ghc/ghc-musl:9.2.8

Great - this means we won't need to install stack when we next upgrade the LTS stackage resolver.

P.P.S.: The Haskell Tool Stack switched to glcr.b-data.ch/ghc/ghc-musl:9.4.6 to build Stack.

Well done!

benz0li · 2023-08-22T09:28:51Z

Switch to githash?: https://hackage.haskell.org/package/githash

benz0li · 2023-08-23T08:35:03Z

Maybe @snoyberg has an idea why this happens.

The Haskell Tool Stack switched to githash some time ago: commercialhaskell/stack@a0276ae

paulcadman · 2023-08-23T08:54:44Z

Maybe @snoyberg has an idea why this happens.

The Haskell Tool Stack switched to githash some time ago: commercialhaskell/stack@a0276ae

Thanks for looking into this, we're planning to remove this information from the version string to avoid these issues #2130

paulcadman · 2023-08-23T12:41:53Z

Switch to githash?: https://hackage.haskell.org/package/githash

I used githash - I see this error in the container when evaluating template haskell:

/__w/juvix-nightly-builds/juvix-nightly-builds/src/Juvix/Extra/Version.hs:16:13: error:
[8529](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8530)
    • Exception when trying to run compile-time code:
[8530](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8531)
        GHEGitRunFailed "." ["rev-parse","--show-toplevel"] (ExitFailure 128) "" "fatal: detected dubious ownership in repository at '/__w/juvix-nightly-builds/juvix-nightly-builds'\nTo add an exception for this directory, call:\n\n\tgit config --global --add safe.directory /__w/juvix-nightly-builds/juvix-nightly-builds\n"
[8531](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8532)
      Code: tGitInfoCwd
[8532](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8533)
    • In the Template Haskell splice $$tGitInfoCwd
[8533](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8534)
      In the expression: $$tGitInfoCwd
[8534](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8535)
      In an equation for ‘gitInfo’: gitInfo = $$tGitInfoCwd
[8535](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8536)
   |
[8536](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8537)
16 | gitInfo = $$tGitInfoCwd
[8537](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8538)
   |             ^^^^^^^^^^^
[8538](https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228#step:10:8539)
Completed 179 action(s).

https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951105777/job/16140262228

So there's a permissions issue in the container which makes the git command fail.

I will try running chown -R $(id -un):$(id -gn) . in the container before running the Juvix build as a workaround.

paulcadman · 2023-08-23T13:28:15Z

So there's a permissions issue in the container which makes the git command fail.

I will try running chown -R $(id -un):$(id -gn) . in the container before running the Juvix build as a workaround.

This workaround fixes the issue. @benz0li I'm not sure if this is an issue with the container, or it's how permissions in containers are supposed to work? Should I raise an issue in https://github.com/benz0li/ghc-musl for this?

https://github.com/paulcadman/juvix-nightly-builds/actions/runs/5951493332/job/16141415424

benz0li · 2023-08-23T13:39:21Z

@benz0li I'm not sure if this is an issue with the container, or it's how permissions in containers are supposed to work?

If a local build succeeds with glcr.b-data.ch/ghc/ghc-musl:9.4.5, I do not consider this an issue with the image.

Should I raise an issue in https://github.com/benz0li/ghc-musl for this?

Rather not.

paulcadman · 2023-08-23T13:56:15Z

If a local build succeeds with glcr.b-data.ch/ghc/ghc-musl:9.4.5, I do not consider this an issue with the image.

I have not tried this locally, it's not easy for me to do this as I'm using a mac. Agreed we don't need to raise this unless it can be reproduced locally. Thanks

benz0li · 2023-08-23T14:01:50Z

I have not tried this locally, it's not easy for me to do this as I'm using a mac.

My machine is a MacBook Pro. 'Docker Desktop' or Podman should do the job.

And because the image is multi-arch (linux/amd64, linux/arm64/v8), everything runs fine/natively, even on Apple silicon.

paulcadman · 2023-08-23T14:03:33Z

My machine is a MacBook Pro. 'Docker Desktop' or Podman should do the job.

And because the image is multi-arch (linux/amd64, linux/arm64/v8), everything runs fine/natively, even on Apple silicon.

You're right! I will try it then.

benz0li · 2023-08-23T14:23:32Z

Dockerfile

ARG GHC_VERSION=9.4.5
ARG STACK_VERSION=2.11.1

FROM glcr.b-data.ch/commercialhaskell/ssi:${STACK_VERSION} as ssi

FROM glcr.b-data.ch/ghc/ghc-musl:${GHC_VERSION}

ARG STACK_VERSION

ENV STACK_VERSION=${STACK_VERSION}

COPY --from=ssi /usr/local/bin/stack /usr/local/bin/stack

Host

docker build -t ghc-musl-stack .

docker run --rm -ti ghc-musl-stack bash

Container

apk add --no-cache clang14

git clone https://github.com/anoma/juvix.git && cd juvix

make runtime LIBTOOL=llvm14-ar

stack install --no-install-ghc --system-ghc --ghc-options='-split-sections' --flag juvix:static

ℹ️ And yes: Juvix also builds fine on/for Linux AArch64 aka linux/arm64/v8.

juvix_build-log_ghc-9.4.5-arm64v8.txt

paulcadman · 2023-08-23T14:51:28Z

My machine is a MacBook Pro. 'Docker Desktop' or Podman should do the job.
And because the image is multi-arch (linux/amd64, linux/arm64/v8), everything runs fine/natively, even on Apple silicon.

You're right! I will try it then.

I built locally and juvix --version works as expected in the container. So the issue is with how GitHub actions is configuring permissions in the container. Thanks for the steps @benz0li

This PR replaces [gitrev](https://hackage.haskell.org/package/gitrev) with [githash](https://hackage.haskell.org/package/githash) which provides more reliable git information about the current git info, it seems to work as expected. githash originated as a fork of gitrev containing fixes snoyberg/githash#11 This PR also fixes the issue with git info detection in the linux static build. There was a permission issue in the build container that caused git cli calls to fail. Closes: * #2294 * #2130

paulcadman added bug pending-review labels Aug 11, 2023

paulcadman mentioned this issue Aug 21, 2023

Remove inaccurate hash information displayed by the --version command #2130

Closed

paulcadman added a commit to anoma/juvix-nightly-builds that referenced this issue Aug 21, 2023

Revert workflow-dispatch input parameter

b31aae3

We would like to test to see if this is causing an issue with the gitrev dependency. See anoma/juvix#2294

paulcadman mentioned this issue Aug 21, 2023

Revert workflow-dispatch input parameter anoma/juvix-nightly-builds#4

Merged

paulcadman mentioned this issue Aug 23, 2023

Replace gitrev with githash for obtaining build-time git info #2308

Merged

paulcadman linked a pull request Aug 23, 2023 that will close this issue

Replace gitrev with githash for obtaining build-time git info #2308

Merged

paulcadman closed this as completed in #2308 Aug 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

linux static build juvix version shows UNKNOWN for hash and date #2294

linux static build juvix version shows UNKNOWN for hash and date #2294

paulcadman commented Aug 11, 2023

jonaprieto commented Aug 15, 2023

paulcadman commented Aug 21, 2023

benz0li commented Aug 21, 2023 •

edited

Loading

paulcadman commented Aug 21, 2023

benz0li commented Aug 21, 2023

benz0li commented Aug 21, 2023

paulcadman commented Aug 21, 2023

benz0li commented Aug 21, 2023

benz0li commented Aug 21, 2023

benz0li commented Aug 21, 2023

paulcadman commented Aug 21, 2023

paulcadman commented Aug 21, 2023 •

edited

Loading

benz0li commented Aug 22, 2023 •

edited

Loading

benz0li commented Aug 22, 2023

paulcadman commented Aug 22, 2023

benz0li commented Aug 22, 2023

benz0li commented Aug 23, 2023

paulcadman commented Aug 23, 2023

paulcadman commented Aug 23, 2023 •

edited

Loading

paulcadman commented Aug 23, 2023 •

edited

Loading

benz0li commented Aug 23, 2023 •

edited

Loading

paulcadman commented Aug 23, 2023

benz0li commented Aug 23, 2023 •

edited

Loading

paulcadman commented Aug 23, 2023

benz0li commented Aug 23, 2023 •

edited

Loading

paulcadman commented Aug 23, 2023

linux static build juvix version shows UNKNOWN for hash and date #2294

linux static build juvix version shows UNKNOWN for hash and date #2294

Comments

paulcadman commented Aug 11, 2023

Cause?

jonaprieto commented Aug 15, 2023

paulcadman commented Aug 21, 2023

benz0li commented Aug 21, 2023 • edited Loading

paulcadman commented Aug 21, 2023

benz0li commented Aug 21, 2023

benz0li commented Aug 21, 2023

paulcadman commented Aug 21, 2023

benz0li commented Aug 21, 2023

benz0li commented Aug 21, 2023

benz0li commented Aug 21, 2023

paulcadman commented Aug 21, 2023

paulcadman commented Aug 21, 2023 • edited Loading

benz0li commented Aug 22, 2023 • edited Loading

benz0li commented Aug 22, 2023

paulcadman commented Aug 22, 2023

benz0li commented Aug 22, 2023

benz0li commented Aug 23, 2023

paulcadman commented Aug 23, 2023

paulcadman commented Aug 23, 2023 • edited Loading

paulcadman commented Aug 23, 2023 • edited Loading

benz0li commented Aug 23, 2023 • edited Loading

paulcadman commented Aug 23, 2023

benz0li commented Aug 23, 2023 • edited Loading

paulcadman commented Aug 23, 2023

benz0li commented Aug 23, 2023 • edited Loading

Dockerfile

Host

Container

paulcadman commented Aug 23, 2023

benz0li commented Aug 21, 2023 •

edited

Loading

paulcadman commented Aug 21, 2023 •

edited

Loading

benz0li commented Aug 22, 2023 •

edited

Loading

paulcadman commented Aug 23, 2023 •

edited

Loading

paulcadman commented Aug 23, 2023 •

edited

Loading

benz0li commented Aug 23, 2023 •

edited

Loading

benz0li commented Aug 23, 2023 •

edited

Loading

benz0li commented Aug 23, 2023 •

edited

Loading