Can you split up the build and push steps?

[MauriceNino]

Related to #166

I want to first build the image, then do some other release steps and then release the image to docker hub.
For that, I thought I should use -o type=image,name=xxx.
But when I run docker manifest inspect xxx after that, it just returns the latest manifest from docker hub.

Where is the built manifest, and how can I push it after the build?

I am building for --platform linux/amd64,linux/arm/v7,linux/arm64/v8 if that is important.

[tonistiigi]

To push from buildx you would run the build again with push option. If you have image in docker then you do docker push. If you have exported image in some other way(local registry, oci tarball) then there are external programs that can push from one registry to another etc.

[MauriceNino]

Thank you for your answer!
Why is there no option to create the manifest and save it locally, so that it can later be picked up again? Or is that something that is technically not possible?

[jedwards1211]

@tonistiigi What is the best output format and external program for outputting a multi-platform image and later pushing it to docker hub? As far as I read there are still problems with pushing OCI images to docker hub: docker/hub-feedback#1871

Building again with --cache-from and --push isn't very great because I'm using a multi-stage build, and only the first stage hits the cache.

[ssosso]

Thank you for your answer! Why is there no option to create the manifest and save it locally, so that it can later be picked up again? Or is that something that is technically not possible?

@MauriceNino
Hello, Have you received any answers to this question?
I want the build and the push to be separated, too.
Is there another way without using the --push option?

[clemlesne]

Also interested. GitHub repository often returns errors, and I'll be interested in separating the build and push to be able to backoff retry the push step.

[rsavage-nozominetworks]

Yeah, this becoming a necessity. There should be a method to build a multi arch image, and then push at a later time. I agree with @MauriceNino this is needed for additional CI processes.

[Oliniusz]

Another reason for the separation is that I would like to run those steps in parallel:

• terraform apply that also creates the ECR repository,
• in the meantime I want to start the docker build.

When the terraform apply has finished I can push the image to the newly created repository.

[leemeador]

@tonistiigi

To push from buildx you would run the build again with push option....

I would be concerned if I build again that it would be possible for something pulled in to do the build again would have changed in the interim. It's unlikely if the interim is short, but not impossible.

[jandry]

Hello, I'm also intéressée on this topic. I want to build, then test and then push.
Any workaround on that ?

[mixja]

@tonistiigi

To push from buildx you would run the build again with push option....

I would be concerned if I build again that it would be possible for something pulled in to do the build again would have changed in the interim. It's unlikely if the interim is short, but not impossible.

If you have already ran the build previously and nothing has changed, then building with the --push flag will use the cached build (unless you add the --pull flag, which will attempt to pull any referenced external images)

[sprat]

This issue is related to my discussion about running targets sequentially: #1976

[sgasquet]

Also interested, in any pipeline we need security tests after build and before any push to external registry.
Any help so far ?

[lodotek]

bump

[th3wingman]

bump

[flowconic]

Bump

[hectorsanchezb]

Bump

[LozanoMatheus]

Bump +it would be nice to have support within buildx for the docker load to load an OCI tarball file (maybe directly in the push)

[ryancurrah]

Is this a potential work around?

# Build and push for amd64
docker build -t myusername/myimage:amd64 --build-arg ARCH=amd64 .
docker push myusername/myimage:amd64

# Build and push for arm64
docker build -t myusername/myimage:arm64 --build-arg ARCH=arm64 .
docker push myusername/myimage:arm64

# Create the multi-arch manifest
docker manifest create myusername/myimage:multi-arch \
    myusername/myimage:amd64 \
    myusername/myimage:arm64

# Annotate the manifest
docker manifest annotate myusername/myimage:multi-arch myusername/myimage:amd64 --arch amd64
docker manifest annotate myusername/myimage:multi-arch myusername/myimage:arm64 --arch arm64

# Push the multi-arch manifest
docker manifest push myusername/myimage:multi-arch

[JohnYoungers]

If your CI system allows for sidecars, an alternative might be to run your own registry:

Github example:

    services:
      registry:
        image: registry:2
        ports: ["5000:5000"]
    steps:
      ...
      - uses: docker/setup-buildx-action@v3
        with:
            driver-opts: network=host
      - run: |
          docker buildx build --platform $targetPlatforms --push -t localhost:5000/$imageName .
          ...
          docker buildx imagetools create --tag "$targetRegistry/$image" "localhost:5000/$image"

I believe that should work, but I'm having issues with the registry migration piece: I believe it's due to the target being an AWS ECR with immutability enabled (I'll update this post if I end up resolving)

edit: This works instead of imagetools create for immutable, but it looks like you end up with several artifacts: regctl image copy "localhost:5000/$image" "$targetRegistry/$image"

[sprat]

If your CI system allows for sidecars, an alternative might be to run your own registry:

I find it easier to call the same build command twice, the first one without --push and the second one with --push if the first one succeeded.