Docker Hub Refresh

Docker

Hopefully this blog will help to keep my (and your) images alive after this Terms of Service change.

Problem

Due to the new Terms of Service inactive images will be removed after 6 month.

On the docker site they explain what inactive means:

What is an “inactive” image?
An inactive image is a container image that has not been either pushed or pulled from the Docker Hub image repository in 6 or more months.

This is not an unreasonable thing. Never used images should probably be cleaned.

The problem I have is that I have made some images explicitly created to keep hold of some older stuff like older java versions.
Just to be able to run older software. So I want these images to stay even though I do not use them often.

Solution

According to the rule described above I only have to pull the images once in a couple of month
to keep them marked as active.

So I need something that can get all the images with the tags I have from the docker hub and
refresh them by either pulling them and removing them again or only refreshing them if I already have
it locally.

Time to start scripting:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#!/bin/bash

DEBUG=0
USR="YOUR_USERNAME_HERE"
PWD="YOUR_PASSWORD_HERE"
ORG="${USR}"

set -e
function list_include_item {
  local list="$1"
  local item="$2"
  if [[ $list =~ (^|[[:space:]])"$item"($|[[:space:]]) ]] ; then
    result=0
  else
    result=1
  fi
  return $result
}

echo "Retrieving local images for ${ORG}..."
LOCAL_EXISTING_IMAGES=$(docker images --format '{{.Repository}}:{{.Tag}}' | grep "^${ORG}")

echo "Retrieving token for ${USR}..."
TOKEN=$(curl -s -H "Content-Type: application/json" -X POST -d '{"username": "'${USR}'", "password": "'${PWD}'"}' https://hub.docker.com/v2/users/login/ | jq -r .token)

echo "Retrieving repository list for ${ORG}..."
REPO_LIST=$(curl -s -H "Authorization: JWT ${TOKEN}" https://hub.docker.com/v2/repositories/${ORG}/?page_size=200 | jq -r '.results|.[]|.name')

echo "Refreshing all repositories and tags for: ${ORG}"
for i in ${REPO_LIST}
do
  IMAGE_TAGS=$(curl -s -H "Authorization: JWT ${TOKEN}" https://hub.docker.com/v2/repositories/${ORG}/${i}/tags/?page_size=300 | jq -r '.results|.[]|.name')
  for j in ${IMAGE_TAGS}
  do
     TAG="${ORG}/${i}:${j}"
     if list_include_item "${LOCAL_EXISTING_IMAGES}" "${TAG}"; then
        echo "Refreshing          : ${TAG}"
        docker pull ${TAG}
     else
        echo "Pulling and removing: ${TAG}"
        docker pull ${TAG}
        docker rmi ${TAG}
     fi
  done
done

This small script seems to do the job.

Now I can run this on my NAS or something and even as a scheduled job.

e.g. every 3 month:

1
0 0 1 */3 *    /path/to/script/docker-hub-refresh.md

Hope you like it 😄.

Greetz,
Ivo.