Using Make For Your Docker Builds

I create, build, publish and maintain a lot of docker images these days and I’m always struggling with a good build system.
Some projects which stand on their own like the Payara project I let build by
the Docker hub itself, but what if you have lots of docker images and don’t want or have the auto build function available?

At first I used shell scripts but they became ugly and unmaintainable fast and finally to my huge surprise my choice of build
tools was make

Problem

You have lots of docker images to maintain for a single project and want a consistent build system but don’t know how.

Solution

Use make and a Makefile.

howto

On a Mac and Linux machine the command-line tool make is standard available which make using it incredibly easy.
What I’ve tried to do is create a way of defining my build targets in such a way that if you adhere to the convention you have a very easy
to use build tool.

In order to maintain images for a project I choose to create a directory structure like the one below.
Essentially a main project directory with the Makefile in it and top level sub-directories with a Dockerfile
and optional extra files needed for the image.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
.
├── Makefile
├── makefile.env
├── README.md
├── hello-world
│   └── Dockerfile
├── ngnix
│   └── Dockerfile
├── node
│   ├── Dockerfile
│   └── src
│   ├── index.js
│   ├── index.html
│   └── angular.js
└── payara
└── Dockerfile

The top level sub-directories represent the name of the image as it should be published as and that combined
with a VERSION and REGISTRY as defined in the makefile.env file it represents the fully qualified name
if the image.

makefile.env:

1
2
3
4
VERSION=0.1
REGISTRY=ivonet
#REGISTRY=localhost:5000
#REGISTRY=192.168.2.42:5555

Below you see the full Makefile and by typing make help you will get the help needed.
What this Makefile makes does is to make it possible to have a consistent build and versioning mechanism for
all the images in the project.
It is also possible to build or release just one of the project images, without having to change the Makefile at all.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
# import config.
# You can change the default config with `make cnf="config_special.env" build`
# Needed keys are:
# VERSION=
# REGISTRY=
cnf ?= makefile.env
include $(cnf)
export $(shell sed 's/=.*//' $(cnf))

DOCKERFILES=$(shell find * -type f -name Dockerfile)
IMAGES=$(subst /,\:,$(subst /Dockerfile,,$(DOCKERFILES)))
RELEASE_IMAGE_TARGETS=$(addprefix release-,$(IMAGES))

# HELP
# This will output the help for each task
# thanks to https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html
.PHONY: help $(IMAGES) $(RELEASE_IMAGE_TARGETS)

help: projects ## This help
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)

.DEFAULT_GOAL := help

projects: ## prints which projects have build targets
@for fo in $(IMAGES); \
do \
echo $$fo | awk '{printf "\033[36m%-30s\033[0m Builds %s\n", $$1, $$1}'; \
echo $$fo | awk '{printf "\033[36mrelease-%-22s\033[0m Releases %s\n", $$1, $$1}';\
done

$(IMAGES): %: ## builds a specific project by its folder name
docker build -t $(REGISTRY)/$@ $(subst :,/,$@)

$(RELEASE_IMAGE_TARGETS): %: ## release a single image from the project
@project=$(subst release-,,$@); \
docker build --no-cache -t $(REGISTRY)/$$project $(subst :,/,$$project); \
docker tag $(REGISTRY)/$$project:latest $(REGISTRY)/$$project:$(VERSION); \
docker push $(REGISTRY)/$$project:latest; \
docker push $(REGISTRY)/$$project:$(VERSION);

all: build ## Build all the images in the project as 'latest'

build: ## Build all the images in the project as 'latest'
@for img in $(IMAGES); \
do \
docker build -t $(REGISTRY)/$$img $(subst :,/,$$img) ; \
done

build-nc: ## Build all the images in the project as 'latest' (no-cache)
@for img in $(IMAGES); \
do \
docker build --no-cache -t $(REGISTRY)/$$img $(subst :,/,$$img) ; \
done \

tag: ## Tags all the images to the VERSION as found int makefile.env
@for img in $(IMAGES); \
do \
docker tag $(REGISTRY)/$$img:latest $(REGISTRY)/$$img:$(VERSION) ; \
done

version: build tag ## Builds and versions all the images in this project

release: build-nc publish ## Make a release by building and publishing the `{version}` and `latest` tagged containers to the registry

# Docker publish
publish: publish-latest publish-version ## Publish the `{version}` and `latest` tagged containers to the registry

publish-latest:
@for img in $(IMAGES); do \
docker push $(REGISTRY)/$$img:latest ; \
done

publish-version: tag
@for img in $(IMAGES); do \
docker push $(REGISTRY)/$$img:$(VERSION) ; \
done

clean: rm-containers rmi-version rmi-latest ## Cleans up the mess you made
@echo "Cleaning dangling images in general"; \
for img in $(docker images --filter dangling=true -q); do \
echo Deleting dangling image $$img; \
docker rmi $$img ; \
done; \
for img in $(docker images | grep "^<none>" | awk '{print $3}'); do \
echo Deleting <none> image $$img; \
docker rmi $$img; \
done

rmi-version: ## Removes all the local images from this project with the defined version
@echo Removing all images with version $(VERSION) from this project; \
for cont in $$(docker images -q); \
do \
cname=$$(docker inspect $$cont|jq '.[0].RepoTags[0]'|sed 's/\"//g'); \
for img in $(IMAGES); \
do \
if [ $(REGISTRY)/$$img:$(VERSION) == $$cname ]; \
then \
docker rmi $$cname 2>/dev/null; \
fi \
done \
done

rmi-latest: ## Removes all the local images from this project with the 'latest' tag
@echo Removing all images with version 'latest' from this project; \
for cont in $$(docker images -q); \
do \
cname=$$(docker inspect $$cont|jq '.[0].RepoTags[0]'|sed 's/\"//g') ; \
for img in $(IMAGES); \
do \
if [ $(REGISTRY)/$$img:latest == $$cname ]; \
then \
docker rmi $$cname 2>/dev/null; \
fi \
done \
done

rm-containers: stop ## Stops and removes running containers based on the images in this project
@echo Removing all created containers from this project; \
for cont in $$(docker ps -aq); \
do \
cname=$$(docker inspect $$cont | jq .[].Config.Image | sed 's/\"//g');\
for img in $(IMAGES); \
do \
if [ $(REGISTRY)/$$img == $$cname ]; \
then \
echo Removing container: $$cname -\> $$cont; \
docker rm -f $$cont >/dev/null; \
fi \
done \
done

stop: ## Stops all running containers based on the images in this project
@echo Stopping all running containers from this project; \
for cont in $$(docker ps -q); \
do \
cname=$$(docker inspect $$cont | jq .[].Config.Image | sed 's/\"//g');\
for img in $(IMAGES); \
do \
if [ $(REGISTRY)/$$img == $$cname ]; \
then \
echo Stopping container: $$cname -\> $$cont; \
docker stop $$cont >/dev/null; \
fi \
done \
done

Companion project

Discussion

If you know a better way of doing this please let me know as I am constantly searching for a more efficient way.