Using Make For Your Docker Builds

docker

I create, build, publish and maintain a lot of docker images these days and I’m always struggling with a good build system.
Some projects which stand on their own like the Payara project I let build by
the Docker hub itself, but what if you have lots of docker images and don’t want or have the auto build function available?

At first I used shell scripts but they became ugly and unmaintainable fast and finally to my huge surprise my choice of build
tools was make…

Problem

You have lots of docker images to maintain for a single project and want a consistent build system but don’t know how.

Solution

Use make and a Makefile.

howto

On a Mac and Linux machine the command-line tool make is standard available which make using it incredibly easy.
What I’ve tried to do is create a way of defining my build targets in such a way that if you adhere to the convention you have a very easy
to use build tool.

In order to maintain images for a project I choose to create a directory structure like the one below.
Essentially a main project directory with the Makefile in it and top level sub-directories with a Dockerfile
and optional extra files needed for the image.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
.
├── Makefile
├── makefile.env
├── README.md
├── hello-world
│   └── Dockerfile
├── ngnix
│   └── Dockerfile
├── node
│   ├── Dockerfile
│   └── src
│       ├── index.js
│       ├── index.html
│       └── angular.js
└── payara
    └── Dockerfile

The top level sub-directories represent the name of the image as it should be published as and that combined
with a VERSION and REGISTRY as defined in the makefile.env file it represents the fully qualified name
if the image.

makefile.env:

1
2
3
4
VERSION=0.1
REGISTRY=ivonet
#REGISTRY=localhost:5000
#REGISTRY=192.168.2.42:5555

Below you see the full Makefile and by typing make help you will get the help needed.
What this Makefile makes does is to make it possible to have a consistent build and versioning mechanism for
all the images in the project.
It is also possible to build or release just one of the project images, without having to change the Makefile at all.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
#
# Copyright 2019 Ivo Woltring <WebMaster@ivonet.nl>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#


# import config.
# You can change the default config with `make cnf="config_special.env" build`
cnf ?= makefile.env
include $(cnf)
export $(shell sed 's/=.*//' $(cnf))

DOCKERFILES=$(shell find * -type f -name Dockerfile)
IMAGES=$(subst /,\:,$(subst /Dockerfile,,$(DOCKERFILES)))
RELEASE_IMAGE_TARGETS=$(addprefix release-,$(IMAGES))

# HELP
# This will output the help for each task
# thanks to https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html
.PHONY: help $(IMAGES) $(RELEASE_IMAGE_TARGETS)

help: projects ## This help
	@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST);
	@echo ">>> make sure to have 'jq' installed."

.DEFAULT_GOAL := help

projects: ## prints which projects have build targets
	@for fo in $(IMAGES);                                                                 \
	do                                                                                    \
		echo $$fo | awk '{printf "\033[36m%-30s\033[0m Builds %s\n", $$1, $$1}';          \
		echo $$fo | awk '{printf "\033[36mrelease-%-22s\033[0m Releases %s\n", $$1, $$1}';\
	done

$(IMAGES): %: ## builds a specific project by its directory name
	docker build -t $(REGISTRY)/$@ $(subst :,/,$@)

$(RELEASE_IMAGE_TARGETS): %: ## release a single image from the project
	@project=$(subst release-,,$@);                                           \
	docker build --no-cache -t $(REGISTRY)/$$project $(subst :,/,$$project);  \
	docker tag $(REGISTRY)/$$project:latest $(REGISTRY)/$$project:$(VERSION); \
	docker push $(REGISTRY)/$$project:latest;                                 \
	docker push $(REGISTRY)/$$project:$(VERSION);

all: build  ## Build all the images in the project as 'latest'

build: ## Build all the images in the project as 'latest'
	@for img in $(IMAGES);                                                    \
	do                                                                        \
		docker build -t $(REGISTRY)/$$img $(subst :,/,$$img) ;                \
	done

build-nc: ## Build all the images in the project as 'latest' (no-cache)
	@for img in $(IMAGES);                                                    \
	do                                                                        \
		docker build --no-cache -t $(REGISTRY)/$$img $(subst :,/,$$img) ;     \
	done                                                                      \

tag: ## Tags all the images to the VERSION as found int makefile.env
	@for img in $(IMAGES);                                                    \
	do                                                                        \
		docker tag $(REGISTRY)/$$img:latest $(REGISTRY)/$$img:$(VERSION) ;    \
	done

version: build tag ## Builds and versions all the images in this project

release: build-nc publish ## Make a release by building and publishing the `{version}` and `latest` tagged containers to the registry

# Docker publish
publish: publish-latest publish-version ## Publish the `{version}` and `latest` tagged containers to the registry

publish-latest:
	@for img in $(IMAGES); do                                                 \
		docker push $(REGISTRY)/$$img:latest ;                                \
	done

publish-version: tag
	@for img in $(IMAGES); do                                                 \
		docker push $(REGISTRY)/$$img:$(VERSION) ;                            \
	done

clean: rm-containers rmi-version rmi-latest ## Cleans up the mess you made
	@echo "Cleaning dangling images in general";                              \
	for img in $(docker images --filter dangling=true -q); do                 \
		echo Deleting dangling image $$img;                                   \
		docker rmi $$img ;                                                    \
	done;                                                                     \
	for img in $(docker images | grep "^<none>" | awk '{print $3}'); do       \
	    echo Deleting <none> image $$img;                                     \
		docker rmi $$img;                                                     \
	done

deep-clean: clean rmi-base-images ## same as clean plus removal of base images
	@echo "Also removes base images";

rmi-version: ## Removes all the local images from this project with the defined version
	@echo Removing all images with version $(VERSION) from this project;      \
	for cont in $$(docker images -q);                                         \
	do                                                                        \
		cname=$$(docker inspect $$cont|jq '.[0].RepoTags[0]'|sed 's/\"//g');  \
		for img in $(IMAGES);                                                 \
		do                                                                    \
			if [ $(REGISTRY)/$$img:$(VERSION) == $$cname ];                   \
			then                                                              \
				docker rmi $$cname 2>/dev/null;                               \
			fi                                                                \
		done                                                                  \
	done

rmi-latest: ## Removes all the local images from this project with the 'latest' tag
	@echo Removing all images with version 'latest' from this project;        \
	for cont in $$(docker images -q);                                         \
	do                                                                        \
		cname=$$(docker inspect $$cont|jq '.[0].RepoTags[0]'|sed 's/\"//g') ; \
		for img in $(IMAGES);                                                 \
		do                                                                    \
			if [ $(REGISTRY)/$$img:latest == $$cname ];                       \
			then                                                              \
				docker rmi $$cname 2>/dev/null;                               \
			fi                                                                \
		done                                                                  \
	done

rmi-base-images: ## Removes all the base (FROM) images used in the projects
	@echo "Removing all the base (FROM) images used in the projects";         \
	baseimgs=$$(find . -type f -name Dockerfile -exec grep ^FROM {} \; | sed 's/FROM //g');\
	for base in $$baseimgs;                                                   \
	do                                                                        \
	    noversion=$$(echo $$base|sed 's/:.*//');                              \
	    if [ "$$noversion" == "$$base" ];                                     \
	    then                                                                  \
	       echo "No version fount, assumong latest";                          \
	       base="$$base:latest";                                              \
	    fi;                                                                   \
	    echo "$$base";                                                        \
		for cont in $$(docker images -q);                                     \
		do                                                                    \
		    cname=$$(docker inspect $$cont|jq '.[0].RepoTags[0]'|sed 's/\"//g') ; \
			if [ $$base == $$cname ];                                         \
			then                                                              \
        	    echo "Removing base-image: $$base";                           \
				docker rmi $$base 2>/dev/null;                                \
			fi                                                                \
		done                                                                  \
	done

rm-containers: stop ## Stops and removes running containers based on the images in this project
	@echo Removing all created containers from this project;                  \
	for cont in $$(docker ps -aq);                                            \
	do                                                                        \
		cname=$$(docker inspect $$cont | jq .[].Config.Image | sed 's/\"//g');\
		for img in $(IMAGES);                                                 \
		do                                                                    \
			if [ $(REGISTRY)/$$img == $$cname ];                              \
			then                                                              \
				echo Removing container: $$cname -\> $$cont;                  \
				docker rm -f $$cont >/dev/null;                               \
			fi                                                                \
		done                                                                  \
	done

stop: ## Stops all running containers based on the images in this project
	@echo Stopping all running containers from this project;                  \
	for cont in $$(docker ps -q);                                             \
	do                                                                        \
		cname=$$(docker inspect $$cont | jq .[].Config.Image | sed 's/\"//g');\
		for img in $(IMAGES);                                                 \
		do                                                                    \
			if [ $(REGISTRY)/$$img == $$cname ];                              \
			then                                                              \
				echo Stopping container: $$cname -\> $$cont;                  \
				docker stop $$cont >/dev/null;                                \
			fi                                                                \
		done                                                                  \
	done

Companion project

Discussion

If you know a better way of doing this please let me know as I am constantly searching for a more efficient way.