Continuous Delivery with Docker Containers & Java EE

This repository contains the source code and step-by-step instructions needed to reproduce the Continuous Delivery with Docker Containers & Java EE webinar.

1. Overview

1.1. Goals

This demo sets up a complete Continuous Delivery environment using Docker containers.

This demo uses Docker Compose as a simple orchestration tool. Docker is used for simplicity in this demo and is not essential to the delivery pipeline.

More details discused in Markus' Blog.

1.2. Infrastructure

This demo uses various components to create a delivery pipeline. Jenkins, Nexus and Sonar run in their own Docker containers while GitHub is used as an external Git-based source repository.

1. Jenkins

   Description: continuous delivery orchestration engine

   Address: http://DOCKER_HOST:8080/jenkins

2. Sonatype Nexus

   Description: artifact repository for archiving release binaries

   Address: http://DOCKER_HOST:8081/nexus

3. SonarQube

   Description: static code analysis engine extracting various quality metrics from the code

   Address: http://DOCKER_HOST:9000

4. GitHub

   Description: source repository hosting the ticket-monster Java application

   Address: https://github.com/jbossdemocentral/continuous-delivery-demo-app.git

Note	If running boot2docker on Mac OSX, DOCKER_HOST is the ip of boot2docker virtual machine

1.3. Delivery Pipeline

The delivery pipeline in this demo is divided into five phases each containing a number of activities (jobs) that need to succeed in order to promote the artifact to the next phase. Each change in the application is a potential production release according to Continuous Delivery principles and can go in production if it successfully passes through all the phases in the pipeline.

1. Build: compilation and unit test, integration tests and static code analysis

2. Dev: release to Nexus, create release tag in Git, deploy to DEV server and run functional tests

3. System Test: deploy to System Test server and run system tests

4. Perf Test: deploy to Performance Test server and run performance tests

5. Pre Production (Stage): deploy to Pre-Production server

1.4. Ticket Monster JEE project

Continuous Delivery from dev → prod in a Docker environment will be demonstrated in this exercise using the Ticket Monster JEE application.

The Ticket Monster JEE project is a moderately complex application that demonstrates how to build modern applications using JBoss web technologies.

2. Pre-requisites

1. Network Bandwidth

   During this lab exercise, Docker images will be pulled from the internet. You’ll want a high-speed broadband connection and open access to the internet.

2. Hardware

   To execute this lab exercise, three machines will be needed. Each machine should consist of a minimum of 2 cores and 4G RAM. The machines can run locally or as VMs in a cloud environment.

3. Red Hat Network Subscription

   You must have a valid _Red Hat Network (RHN) subscription with entitlements that allow for:

   1. management of rhel-server-7 operating systems.

   2. download of supported Red Hat JBoss middleware software.

4. Operating System

   Currently, the host operating system required to build the docker images provided in this project needs to be Red Hat Enterprise Linux (RHEL) 7. The RHEL7 hosts should be registered with the Red Hat Network (RHN) via subscription-manager.

   In the near future, Red Hat will be providing a RHEL CDK (Container Development Kit). The RHEL CDK will allow for building of images in this project on host operating systems such as Fedora, OSX and Windows.

5. github account

   A github account will be needed to fork the original ticket-monster source code

6. git

   Ensure that git is installed on the operating system of all machines.

7. Docker

   Ensure that docker is installed and running on the operating system of all machines

8. Docker Compose

   docker-compose is a handy utility for defining and running complex applications with Docker. It is not a feature complete paas. However, it does a nice job of facilitating the use of multiple Docker containers in this Continuous Delivery exercise.

3. Instructions

3.1. Start 3 host environments

This lab requires the following 3 machines as per the specifications listed in the Pre-requisites section:

1. dev

2. systest

3. prod

These hosts may be either virtual machines or bare-metal. They should be on a network that allows TCP assessibility between them.

3.2. Clone this project

This project needs to be cloned on all machines (dev, systest and prod) used in this project.

git clone https://github.com/jbossdemocentral/continuous-delivery-javaee-container-src.git

Doing so creates a directory on your local workstation called: continuous-delivery-javaee-container-src.

Note	For the remainder of these instructions, the directory created from having cloned this project will be referred to as: $PROJECT_HOME.

3.3. ssh key pair

3.3.1. Overview

In this demo, SSH keys are used for the following purposes:

1. Authentication to Github

   This lab makes use of the Maven Release Plugin. One of the operations attempted by the Maven Release Plugin will be to push changes to your project repository in Github. SSH kays will be shared between the Maven Release Plugin and your Github account to allow maven to push these commits.

2. ssh connections to systest and prod hosts

   This lab makes use of Jenkins jobs that push releases from your Jenkins container to your systest and prod host machines. SSH key pair authentication is used to allow for creation of ssh / scp connections between lab environments.

3.3.2. ssh key pair creation

In this step, an ssh key pair will be created in your dev machine.

1. cd $PROJECT_HOME/demo

2. execute: ssh-keygen -q -f images/jenkins-ci/config/.ssh/id_rsa -t rsa -N ''

   The key pair will be created (with no passphrase) in the directory: $PROJECT_HOME/demo/images/jenkins-ci/config/.ssh

3.3.3. ssh key pair distribution

Add public key to Github

To allow for authentication between the Maven Release Plugin and Github, execute the following:

1. Authenticate into Github

2. Navigate to Github account settings

3. Click SSH Keys → Add SSH Key

   

4. Copy and paste the contents of your previously created public key to the key text-box in Github.

   Your public key should be located at: PROJECT_HOME/demo/images/jenkins-ci/config/.ssh/id_rsa.pub.

5. Click the AddKey button

Add public key to systest and prod hosts

Add the previously created public key to the following file of the root operating system user of both the systest and prod machines:

$HOME/.ssh/authorized_keys

3.4. ticket-monster project

Continouos Delivery from dev → prod is demonstrated using the Ticket Monster JEE application.

During the release stage, the release version of the ticket-monster project is updated (ie: 2.6.49-SNAPSHOT → 2.6.49.Final). This change is version controlled. Subsequently, a git repository of the ticket-monster application is needed to push these release changes to.

The source to the ticket-monster application can be found in Red Hat’s jbossdemocentral. This git repository is public (any git client can clone the project) however direct commit access to this repository is controlled. You’ll want to utilize a version of the ticket-monster application that you can modify.

3.4.1. Fork ticket-monster project

You’ll want to fork the ticket-monster project using your own github account:

1. In your browser, navigate to: https://github.com/jbossdemocentral/continuous-delivery-demo-app

2. Authenticate into gitub.

3. On the far right, click Fork

   

4. After a few seconds, a fork of the ticket-monster app will appear in your own github account.

5. Make note of the clone URL to your forked repository. You’ll make use of this URL through this exercise.

3.4.2. Modify ticket-monster pom.xml

The pom.xml of the `ticket-monster project includes references to the original git repository in jbossdemocentral . These references should be changed.

1. Clone the forked ticket-monster project to your local workstation.

2. Using your favorite text editor, edit the following section of: continuous-delivery-demo-app/pom.xml:

   

   In particular, change all of the URLs in the scm section such that they reference your forked repository.

3. Commit and push the change to your forked git repository.

3.5. Build and start dev-env

3.5.1. Build dev-env images

Docker images from this project need to be built prior to the creation of docker containers.

Docker-compose is used to build the Docker images included in this project.

1. cd $PROJECT_HOME/demo

2. add jboss-eap-6.4

   1. Use your RHN userId to access the Red Hat Support Portal.

   2. Download jboss-eap-6.4.0.zip

   3. Copy jboss-eap-6.4.0.zip to the following directory on the host machine: $PROJECT_HOME/demo/images/appserverbase/resources

3. Build the project’s base image.

   One limitation of docker_compose is that it’s not smart enough to build images in the order specified in the project’s docker-compose yaml config. Instead, docker-compose attempts to build images in alphabetic order. This becomes a problem when building the images defined in $PROJECT_HOME/demo/baseimages.yml.

   This docker-compose limitation can be overcome by explicitly specifying the order in which to build the project’s images.

   execute:

   docker-compose -f baseimages.yml build base

   Note	Depending on network bandwidth, expect this step to take a few minutes while the RHEL7 base docker image is downloaded and updated.

4. Build all other images specified in the project’s baseimages.yml config file.

   execute:

   docker-compose -f baseimages.yml build

   Note	Expect this step to take longer than the previous. Several products (such as Jenkins) are downloaded.

5. Execute: docker images

   This command should provide a listing of all of the images that have been built thus far. The output should appear similar to the following:

   

3.5.2. Start dev-env containers

1. cd $PROJECT_HOME/demo

2. docker run -d --name="demo_storage" demo_storage

3. docker-compose -f dev-env.yml up -d

   Note	Depending on network bandwidth, expect this step to take a while as well as the Sonar and Nexus images are downloaded.

4. docker-compose -f dev.yml up -d

   This step starts a container that includes the Ticket Monster application.

5. docker ps -a

   Make note of the newly instantiated docker containers and their port mappings.

Your dev-env machine should now have the following running Docker containers:

1. demo_jenkins_1

   Accessible via your browser at the following URL:

   http://<address_of_dev_machine>:9080/jenkins

   

2. demo_nexus_1

   Accessible via your browser at the following URL:

   http://<address_of_dev_machine>:8081/nexus

   

3. demo_sonar_1

   Accessible via your browser at the following URL:

   http://<address_of_dev_machine>:9000

   

3.5.3. Modify Jenkins Jobs with new ticket-monster URL

Several of the Jenkins jobs that come pre-configured in the jenkins-demo-1 container reference URL to the ticket-monster application in github/jbossdemocentral. All of these jenkins jobs should instead reference the URL of your forked ticket-monster app previously created in the section: Fork ticket-monster project.

Of particular importance is the Jenkins job: ticket-monster-release. The ticket-monster-release job is responsible for creating a release of the latest ticket-monster application.

The ticket-monster-release job updates the version in the pom.xml of the application and pushes this change back to the ticket-monster git repository. Subsequently, your Jenkins jobs will need commit access to a ticket-monster git repository.

Using the clone URL of your forked ticket-monster project, modify the Jenkins jobs as follows:

1. In your browser, navigate to the jenkins dashboard.

2. Click the ticket-monster-analysis link

   

3. On the left panel, click Configure

4. Scroll down to the Source Code Management section

5. Git should already be selected as the repository type.

6. Replace the value of the Repository URL with the URL of your forked ticket-monster repository in github.

   

7. Under the Credentials text box, click the Add button

8. In the Add Credentials pop-up, populate with your preferred mechanism to authenticate into gihub (ie: username/password or ssh keys)

9. Scroll to the bottom and click Save

Repeat the above procedure for the following additional Jenkins jobs:

1. ticket-monster-build

2. ticket-monster-release

3. ticket-monster-test-int

3.6. Build and Start systest

Overview

The systest environment consists of a single Docker container running JBoss EAP 6.4. This infrastructure will be used as the test environment.

Start systest containers

Execute the following steps in your systest machine.

1. cd $PROJECT_HOME/demo

2. docker-compose -f systest.yml up -d

3. docker ps -a

   Your systest machine should now have one running Docker container: demo_systest_1

   This container is accessible via your browser at the following interfaces:

   1. JBoss EAP Management Console

      http://<address_of_systest_machine>:19990

      Authenticate using a userId/password of: admin / admin-123

      

   2. Ticket Monster Homepage

      http://<address_of_systest_machine>:18080/ticket-monster

      

3.7. Build and Start prod

The production environment used in this project is identical to systest.

On the production machine, start its containers in the same manner as was done previously in the systest machine with one change: execute docker-compose -f prod.yml up -d

3.8. Start ticket-monster-build job

After all images are built and containers started on all enviornments (dev, systest and prod), continuous delivery of a JEE application in a Docker environment can be demonstrated.

The Jenkins Delivery Pipeline can be manually started via the Jenkins Dashboard.

At the far right of the dashboard, click the green start button corresponding to the ticket-monster-release job.

4. Real-World Considerations

4.1. ticket-monster-build initiation

4.2. Email set-up

4.3. Human Task: systest → prod