Introduction to CI/CD pipelines using Docker and Containers
It is straightforward to run your own development infrastructure with Docker, giving you an alternative to hosted services. It’s also straightforward to use these services for your own deployment workflow, including full Docker CI/CD.
There are no hard dependencies on any of the technologies – or any stack (.NET, Java, LAMP…see this post for Docker on Windows / Docker on Linux Production Tips).
What does a CI CD pipeline look like in a containerized ( Docker ) world?
A CI CD pipeline with continuous integration looks like this:
- When developers push code to the shared source repository, it should trigger a build that produces a release candidate.
- The release candidates should be tagged Docker images that are stored in a local registry.
- The CI workflow deploys the solution from the built images as containers and runs an end-to-end
test pack. - If the tests pass, the image versions are made publicly available on Docker Hub, and the pipeline can start a rolling upgrade in a public environment running on a remote Docker Swarm. (In a full CI/CD environment, you can automate the deployment to production in your pipeline).
Every Stage of the Docker CI/CD pipeline is Containerized
The stages of the pipeline will all be powered by software running in Docker containers:
- Source control: Gogs, a simple open source Git server
- Build server: Jenkins, a Java-based automation tool that uses plugins to support many workflows
- Build agent: The .NET (or java) SDK packaged into a Docker image to compile code in a container
- Test agent: NUnit (or JUnit) packaged into a Docker image to run end-to-end tests against deployed code
- Gogs and Jenkins can run in long-running containers on a Docker Swarm or on an individual Docker Engine.
The build and test agents are task containers. These will be run by Jenkins to perform the pipeline steps, and will subsequently exit. The release candidate will be deployed as a set of containers.
The CI/CD workflow using Docker Images
The workflow for this pipeline starts when a developer pushes code to the Git server,
which is running Gogs in a Docker container.
Jenkins is configured to poll the Gogs repository, and it will start a build if there are any changes. All the custom components in the solution use multi-stage Dockerfiles, which are stored in the Git repository for the
project.
Jenkins runs docker image build commands for each Dockerfile, building the image on the same Docker host where Jenkins itself is running in a container.
When the builds complete, Jenkins deploys the solution locally as containers on the same Docker host. Then, it runs end-to-end tests, which are packaged in a Docker image and run as a container in the same Docker network as the application being tested.
If all the tests pass, then the final pipeline step pushes these images as release candidates to the local
registry, and the registry is also running in a Docker container.
Summary – Docker CI CD, Continuous Delivery with Containers
It is straightforward to run your own development infrastructure with Docker, giving you an alternative to hosted services. It’s also straightforward to use these services for your own deployment workflow, including full CI/CD.
There are no hard dependencies on any of the technologies. The process shown here using Gogs, Jenkins, and the open source registry can just as easily be implemented with hosted services like GitHub, AppVeyor, and Docker Hub.