How to cache docker images in Gitlab runner

GitLab CI/CD
, , ,
1

Hi
I want to add the configuration in Gitlab runner for solving the “too many requests” error in my pipelines.

What can I do?

Thanks in advance

2

Hi,
where do you get the “too many requests” error? Could you post output of a job?

3

this is the outPut of job:

Preparing the “docker” executor 00:31 Using Docker executor with image maven:3.6.3-jdk-11 … Pulling docker image maven:3.6.3-jdk-11 … ERROR: Preparation failed: Error response from daemon: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: Understanding Your Docker Hub Rate Limit | Docker (docker.go:131:3s) Will be retried in 3s …

and this is the gilab-ci file:

cache:
key: global
paths:
- .m2/repository

variables:
MAVEN_OPTS: “-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true”
MAVEN_CLI_OPTS: “-s .m2/settings.xml --batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=false -DdeployAtEnd=false”

stages:

  • compile
  • test
  • deploy

mvn_compile:
stage: compile
image: maven:3.6.3-jdk-11
script:
- mvn $MAVEN_OPTS $MAVEN_CLI_OPTS clean package
cache:
key: ${CI_COMMIT_SHA}
untracked: true
only:
- merge_requests

mvn_test:
stage: test
image: maven:3.6.3-jdk-11
script:
- mvn $MAVEN_OPTS $MAVEN_CLI_OPTS verify -Dmaven.test.skip=false --quiet
- mvn $MAVEN_OPTS $MAVEN_CLI_OPTS jacoco:report
cache:
key: ${CI_COMMIT_SHA}
untracked: true
artifacts:
reports:
junit:
- “/target/surefire-reports/TEST-*.xml"
paths:
- "/target/site/jacoco”
expire_in: 1 week
only:
- merge_requests

mvn_deploy:
stage: deploy
image: maven:3.6.3-jdk-11
script:
- echo ${CI_COMMIT_SHA}
- mvn $MAVEN_OPTS $MAVEN_CLI_OPTS clean javadoc:jar source:jar deploy
cache:
key: ${CI_COMMIT_SHA}
untracked: true
only:
- tags
- master

4

This error has nothing to do with GitLab Runner. This is the Docker Hub rate limit that is described in the link in the output after the error.
In order to get your own rate limit and not the anonymous you need to create Docker Hub account (it’s free) and then use that account to authenticate to Docker Hub.
You can read Define an image from a private Container Registry how to setup and use credentials for Docker Hub in GitLab Runner.

5

ok, I’ll definitely read the link that you shared.
But I asked this question in stackoverflow and they said this problem occurred when the Gitlab runner doesn’t cache the images.

So as I realized, I first create a docker hub account and then use the account credential according to the link that you shared.

Am I right?

6

There are some docs on caching Docker Hub images which might be useful to you.