Automated Docker image builds

[magnusbaeck]

Description

As discussed on Slack there are several eiffel-community repositories that contain one or more dockerfiles but only a small number of them have publicly available images (https://hub.docker.com/u/eiffelericsson). We should set up automated builds of all our images and have them push to e.g. Docker Hub after every merged PR. There are at least a few ways to accomplish this:

• Use Docker Hub's automated builds feature.
• Use the Jenkins instance owned by the Nordix foundation for building and pushing images to Docker Hub (or some other public registry).
• Use a docker/build-action GitHub action to build and push images to Docker Hub (or some other public registry).

I have set up an eiffelcommunity Docker Hub organization. The free tier only allows three users, but with sufficient automation that might go a long way. Otherwise prices start at $25/mo for five users.

Motivation

Public and up to date Docker images would be useful to many people. It would also reduce the risk of mistakes and reduce the burden on maintainers (who'd otherwise have to build and push images) and community maintainers (who'd have to administer push permissions).

Exemplification

After merging a PR I'd love to just be able to flip the image tag in our local deployment YAML files and deploy instead of having to sync the git in question, build, and push to a local registry, or spend time on automating this task locally on our site.

Benefits

See Motivation, above.

Possible Drawbacks

It would be another cloud-based entity owned and administered by the Eiffel community. Right now we only have the GitHub repositories.

[fdegir]

Putting summary of the discussion we had on Slack beginning of the summer here as reference purposes.

• Docker Hub Automated Builds: Use of docker hub automated build functionality may not be sufficient since the functionality offered for "free tier" is pretty basic. If I am not mistaken, no of concurrent builds is limited and also it takes time for builds to be run - some queueing is happening. We attempted docker hub builds in OPNFV community and weren't impressed.
• GitHub Actions: I must confess I have very little knowledge about this so I can't say much about it with regards to what is offered to open source projects. However, from usability point of view, we may not be able to access to the resources where the builds take place for troubleshooting or fine tuning purposes which I'm not sure how much we would need.
• Hosting builds on Nordix: I had a chat within Nordix Community and we would be happy to collaborate with Eiffel Community on this with no limitations with regards to what we could provide since we manage the community infrastructure. But for the build topic, we created an example job for Eiffel Gerrit Herald on Nordix Jenkins.

Taking this a bit further even though I know this issue just talks about docker build automation. Some other possibilities are

• Extend the CI we do for Eiffel further - for example actually deploying whatever built by Eiffel on wherever - standalone nodes/Kubernetes/etc.
• Bring up a real Eiffel Domain on Nordix so when things happen there (promotion/CL, release, etc.), events could be published from it and interested organizations/people could consume those in Eiffel way as well.

[fdegir]

Sharing a related announcement here since we need to take this type of changes into account while deciding where to host the infrastructure or which tools to use for the community.

Docker announced changes to their container image retention policy.
According to the policy, they plan to limit retention of images for free accounts to 6 months.

This policy will come into effect on November 1st and images that are marked as inactive will be scheduled for deletion.
Inactive in this context means the images that have not been pushed or pulled in the last 6 months.

This has potential to expose the Eiffel Community to issues for example old(er) releases may become unavailable or requires the community to ensure they are pulled once every 6 months.

Similar thing is possible for other free but not-open source stuff as well since they are not community driven so we need to approach Docker Hub like services a bit carefully.

[fdegir]

Sharing the proposal which includes enabling builds as well as additional topics.

https://docs.google.com/presentation/d/1-Qcbh0y7rJahG8-FMK2xKQ8vuH_tX3VfgzRgla-WTWE
https://hackmd.io/X3Nrxe4MRdSJI_lsTkSwiQ?view

[fdegir]

@magnusbaeck we have the periodic builds for eiffel-gerrit-herald available. Created an issue in project backlog to describe the setup followed by few asks: eiffel-community/eiffel-gerrit-herald#12

I suppose we can close this issue and follow up on the one created in project backlog.

[magnusbaeck]

Yeah, I think we're more or less ready to close this one. From my PoV I'd prefer using GitHub Actions for building the image and registry.nordix.org as the Docker registry. Harbor, has a System Robot Account feature that seems to fit the use case well (you don't want to add your personal password or token to the GitHub configuration).

I'll try this out for eiffel-goer and perhaps send a PR to document what was needed to make it happen.

[magnusbaeck]

Filed eiffel-community/eiffel-goer#79 to implement this for Goer.