[JENKINS-29239] Containerized Jenkins via Data Volume Container (--vo…

[superflav]

…lumes-from)

This patch adds support for Docker Data Volume Containers (https://docs.docker.com/engine/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container) and a "Jenkins dirs in Data Volume Container" checkbox, which, when used together, allows docker-custom-build-environment to be used when Jenkins itself is running in a Docker container.

To fully accomplish this, create a Data Volume Container named 'jenkinsdata' which exposes volumes '/var/jenkins_home' and '/tmp'. Mount these volumes in the 'jenkinsmaster' container via --volumes-from jenkinsdata. Once 'jenkinsmaster' is up and running, create a job that uses the docker-custom-build-environment plugin and mount the same volumes by specifying 'jenkinsdata' in the Data Volume Container Name field. To finish, make sure the "Jenkins dirs in Data Volume Container" (which prevents the Jenkins workspace and /tmp dirs from being mounted from the Docker host) checkbox is checked, and you will now be able to successfully run your build from within a Docker container.

[jenkinsadmin]

Thank you for this pull request! Please check this document for how the Jenkins project handles pull requests.

[ndeloof]

I had something comparable in mind, but using a distinct approach:

• detect Jenkins agent in running inside a container (read /proc/1/cgroup and search for /docker/<containerID>)
• start container with additional `--volumes-from

[superflav]

OK, so, to clarify, you are suggesting that the jenkinsMountedInDataVolumeContainer boolean setting be replaced with an internal runningInDocker check, which will look for the mention of /docker/<containerID> when reading /proc/1/cgroup, correct? If so, that works for me, and I should have some time to work on this change later this week. Thanks.

[gf-atebbe]

@superflav Any chance you have had any more time to look at this? I'm blocked by this issue as well and would love to see this pull request get merged!

[superflav]

I was waiting to hear back from @ndeloof before putting any time into making the change I outlined. @ndeloof, thoughts?

[ndeloof]

@superflav I don't have short terms bandwidth to work on this, so feel free to start working on this

[BincyV]

@superflav Thanks for the above patch, it works very well with Jenkins setup, where Jenkins itself is running in a Docker container.
So before when using Custom Build Environment Plugin, the builds were failing as the build container can't bind-mount the workspace and tmp dir. As you suggested in this Pull Request (PR), the workaround did was

1. Created datavolume container mounting both /workspace and /tmp.
2. Configured job using [JENKINS-29239] Containerized Jenkins via Data Volume Container (--vo… #38. (Provided the Datavolume Conatiner name under Data Volume Containers and checked the Jenkins dirs in Data Volume Container field.)

So it would be great, if you can get this PR merged after resolving conflicts.

[ndeloof]

A better approach imho is to detect the build is ran within a container (can just check for /proc/self/cgroup) then inject a --volume-from option to share workspace with build agent. Docker-pipeline-plugin just does this.

[BincyV]

@superflav : Do you have any plans to work on the change which @ndeloof suggested.? I would like to see this PR merged.

[superflav]

@bincy1234, I don't have time right now, but you can take it over if you want. Might be best to just scrap what I have here and instead follow along with what the Docker-pipeline-plugin does, as @ndeloof suggested.

[ndeloof]

see https://github.com/jenkinsci/docker-workflow-plugin/blob/b237e3266eb01b26f4a7b1a69856b0198774a2bb/src/main/java/org/jenkinsci/plugins/docker/workflow/client/DockerClient.java#L311

[BincyV]

If I can find some time to work on this, I am happy to do it.