Build and release Docker Compose using Azure DevOps

Table of contents:

I’ve been fighting a bit with some of the Azure DevOps pipeline tasks trying to configure end-to-end solution for one of my side project. It is based on a good old Docker Compose and I am pretty happy with how it works in production. What I wanted to do is schematically described down below.

What is Azure DevOps #

Azure DevOps helps to plan smarter, collaborate better, and ship faster with a set of modern dev services. It’s a end-to-end solution for any software development cycle. Anyone can use it even for free with some limitations/conditions of usage (public projects, limited pipeline minutes per month etc). And it’s free unlimited git!

Multistage pipeline #

If you are not familiar with multistage pipeline concept, have a look. In short, it brings all CI/CD experience into yaml, where you define all your stages (no UI.. ). It’s been really big missing thing for a while since only build pipeline could be explain like this..

 1
 2stages:
 3  - stage: Build_docker_containers
 4    jobs:
 5    - job: Build
 6      pool:
 7        vmImage: 'Ubuntu-16.04'
 8      continueOnError: true
 9      steps:
10      - task: Docker@2
11        inputs:
12          containerRegistry: 'AZURE-CONTAINER-REGISTRY-NAME'
13          repository: 'AZURE-CONTAINER-REGISTRY-REPOSITORY-NAME'
14          command: 'buildAndPush'
15          Dockerfile: '**/Dockerfile'
16      - task: PublishPipelineArtifact@1
17        inputs:
18          targetPath: '$(Pipeline.Workspace)'
19          artifact: 'docker-compose'
20          publishLocation: 'pipeline'
21  
22  - stage: 'Deploy_to_production'
23    jobs:
24    - deployment: Production
25      pool:
26        vmImage: 'Ubuntu-16.04'
27      environment: 'Production'
28      strategy:
29        runOnce:
30          deploy:
31            steps:
32            - task: CopyFilesOverSSH@0
33              inputs:
34                sshEndpoint: 'SSH-END-POINT-NAME-FROM-SERVICE-CONNECTIONS'
35                sourceFolder: '$(Pipeline.Workspace)/docker-compose/s/'
36                contents: |
37                  docker-compose.yaml
38                  .env                  
39                targetFolder: 'TARGET-PATH'
40            - task: SSH@0
41              inputs:
42                sshEndpoint: 'SSH-END-POINT-NAME-FROM-SERVICE-CONNECTIONS'
43                runOptions: 'inline'
44                inline: |
45                  sed -i 's/##BUILD##/$(Build.BuildId)/g' docker-compose.yaml                  
46            - task: SSH@0
47              inputs:
48                sshEndpoint: 'SSH-END-POINT-NAME-FROM-SERVICE-CONNECTIONS'
49                runOptions: 'inline'
50                inline: |
51                  docker-compose up -d 2> docker-compose.log
52                  cat docker-compose.log                  

Let’s break down the above into small parts and explain what was going on there. In the first stage Build_docker_containers there are two tasks: build image (it’s actually three actions in one: build, tag and push) and publish pipeline artifact. Use this task in a pipeline to publish artifacts for the Azure Pipeline (note that publishing is NOT supported in release pipelines. It is supported in multi stage pipelines, build pipelines, and yaml pipelines). AZURE-CONTAINER-REGISTRY-NAME and AZURE-CONTAINER-REGISTRY-REPOSITORY-NAME both have to be changed accordingly. Note that a built image gets tag which is by default built-in Build.BuildId predefined variable which helps me properly roll out my app update on the second stage.

For the sake of simplicity this pipeline (stages) has been simplified.

The second stage is Deploy_to_production and it rolls out built image to my production server. All it’s tasks are based on SSH deployment tasks. The SSH endpoint has to be configured first in service endpoints (there used to be some issues in new service endpoint experience with >2048 public keys in 2019, but Microsoft team has fixed this).

sed -i 's/##BUILD##/$(Build.BuildId)/g' docker-compose.yaml replaces build number, so docker-compose up -d 2> docker-compose.log brings something to update in docker-machine.

Docker compose for Azure DevOps #

I use docker for packaging an app and docker registry (Azure Container Registry). There is no problem to use dockerhub, just a corresponding endpoint has to be present in service endpoints. Compose helps me to combine multiple containers and define the logic between them and some other objects, as well as their behavior.

The following docker-compose.yaml is in my project.

 1version: '3'
 2volumes:
 3  postgres_data: {}
 4services:
 5  app:
 6    image: AZURE-CONTAINER-REGISTRY-NAME.azurecr.io/AZURE-CONTAINER-REGISTRY-REPOSITORY-NAME:##BUILD##
 7    restart: always
 8    environment: 
 9      RAILS_SERVE_STATIC_FILES: 'true'
10      COMPOSE_PROJECT_NAME: 'PROJECT-PREFIX'
11    depends_on:
12      - db
13  db:
14    restart: always
15    image: postgres
16    volumes:
17      - postgres_data:/var/lib/postgresql/data

Let’s break down the above into small parts and explain what was going on there. There are two services (of course there are more, but for the sake of simplicity of this exercise there are only two). AZURE-CONTAINER-REGISTRY-NAME.azurecr.io/AZURE-CONTAINER-REGISTRY-REPOSITORY-NAME:##BUILD## has to be changes slightly except ##BUILD## which is being changed every pipeline execution.

Summary #

Any feature or bug fix can be delivered to production in less than 6 minutes.

With 1800 free minutes of pipeline per month and 6 minutes of all my stages total duration I can do 300 cycles building and releasing my software.

Love it!

All templates are generalized and uploaded to this repository.