Using Buildkite for Docker builds

Each project in the Moneybird stack has its own Dockerfile in the root of the project. This allows engineers to change the dependencies of the project when they work on the project. The Dockerfile also makes the changes to dependencies tracable, making upgrades way easier than before.

When an engineer pushes new code to our git repository, we want to test this code and make it ready for deployment. There are a lot of great tools for automating builds and testing. We are using Buildkite (opent in nieuw tabblad) because is delivers the best combination of easy configuration and flexibility.

Choosing a CI tool #

There are many CI tools available, many of them are capable of executing an arbitrary shell script to build things like Docker images. When we started using Docker, we were running Jenkins (opent in nieuw tabblad) and tried to incorporate all our requirements:

1. We want to start builds when commits are pushed to Github
2. We want the tool to report the status back to Github using the commit status API (opent in nieuw tabblad)
3. We want the tool to report to our internal chat application, at that time Hipchat, currently we are using Slack
4. We want the tool to report by email when a team member prefers to receive email

Installing plugins and maintaining Jenkins takes a lot of time. We were able to build Docker images from the Dockerfiles and test them on using Jenkins.

Due to the hassles with Jenkins, we decided to look for a hosted solution. We don’t need CI to run internally and there are great hosted solutions like Travis CI (opent in nieuw tabblad), Codeship (opent in nieuw tabblad) and Wercker.

Docker and hosted CI tools #

Many hosted CI tools run your test suite on a clean virtual machine. This machine has a predefined set of dependencies installed, allowing your test suite to start quickly. Docker itself has these dependencies defined inside the image. It depends on cached layers to speed up the process of building images.

At the time of our research (early 2014), none of the hosted CI tools had a solution for running Docker. Building a Docker image was possible but slow: in a clean virtual machine the Docker build would take ages to complete because nothing was cached.

Buildkite provides a combination of a SaaS frontend and agents running on any server you like. The frontend fulfills all our requirements about Github integration and notifications about builds (and no maintenance hassles).

We run 10 Buildkite agents on a bare metal server, located in our office. The agents take on jobs from the Buildkite servers and build our projects. Biggest advantage: all agents run on a single server and building images a blazing fast due to the caching!

Setting up Buildkite #

Buildkite is very easy to setup: in their online tool, you define how you want to build your project. Currently, we have just one build step, calling a bin/build_container script in the project. Buildkite already checked out the right commit that needs to be build.

# Build image
docker build -t registry.acme.org/image_name:$BUILDBOX_COMMIT .

# Push to registry
docker push registry.acme.org/image_name:$BUILDBOX_COMMIT

# Start dependencies: make sure all dependencies are running
docker run -d --name=redis_$BUILDKITE_COMMIT registry.acme.org/redis:2.8-1
REDIS_IP=$($DOCKER inspect --format='' redis_$BUILDKITE_COMMIT)

# Run container with specs
docker run --rm=true --name=image_name_test_$BUILD_ID -e REDIS_IP="$REDIS_IP" registry.acme.org/image_name:$BUILDBOX_COMMIT bin/cibuild

# Tag image with current branch name and push when specs are green
docker tag -f registry.acme.org/image_name:$BUILDBOX_COMMIT registry.acme.org/image_name:$BUILDBOX_BRANCH
docker push registry.acme.org/image_name:$BUILDBOX_BRANCH

The bin/ci_build script contains the test suite to be run. All output of the script is handled by the Buildbox agent and pushes back to their online tool:

More configuration in Buildkite #

At this moment, we only have one step in our builds. Buildkite provides options to create a more complex workflow. This allows you to run jobs in parallel for faster results. Another option is to incorporate a step that does deployment only when the job is run in the master branch. We are experimenting with these features and will report back on this blog with the results!