Gitalb ci DID cache

Hy Guys,

I’ve an Omnibus installation of Gitlab in my local server, and on another server a gitlab-runner that performs the CI tasks.

In my pipeline I’ve a Job that check the code quality using Docker In Docker with codeclimate.

The problem is that it is very slow (20 minutes) because every time it download and extract the docker images of codeclimate.
I tried to configure the gitlab-ci cache but nothing changed.

docker pull codeclimate/codeclimate
Using default tag: latest
latest: Pulling from codeclimate/codeclimate 
2f3f3e5e133b: Pulling fs layer
2654c654a6e7: Pulling fs layer
412e64056adf: Pulling fs layer
a3ed95caeb02: Pulling fs layer
b79986a12326: Pulling fs layer
1bbb14422caf: Pulling fs layer
46bd7e5026ba: Pulling fs layer
7df61bb88827: Pulling fs layer
0381f4ee19b5: Pulling fs layer
7df87a0871f8: Pulling fs layer
4d99acb499d7: Pulling fs layer
ee6a20c6ea98: Pulling fs layer
cb945e8bccc7: Pulling fs layer
a789fd50eff6: Pulling fs layer
1bbb14422caf: Waiting
46bd7e5026ba: Waiting
7df61bb88827: Waiting
a3ed95caeb02: Waiting
b79986a12326: Waiting
7df87a0871f8: Waiting
4d99acb499d7: Waiting
cb945e8bccc7: Waiting
a789fd50eff6: Waiting
ee6a20c6ea98: Waiting
412e64056adf: Download complete

How I have to set the gitlab-ci cache to cache all the docker layers that the DID runner download every time?

thank you

That sounds a docker issue than a runner configuration. Can you check your script, make sure you’re not pruning after your check? “docker pull” should check locally first (unless the codeclimate image is getting updated that often).

I’d try this by hand to make sure docker works first as the runner user (as the runner user).

Yes you are right but the docker pull is performed inside a docker container. if the runner doesn’t cache the downloaded images then as soon as the job is finished the container will be deleted loosing all the downloaded docker images right?

If I execute the job directly on my operating system (and not using dockerInDocker) there is no problem:

These are the commands that I execute in the gitlab-ci.yml:

stage: reports
  image: docker:latest
  before_script: []
  variables:
    DOCKER_DRIVER: overlay2
  services:
    - docker:dind
  script: 
    - docker info
    - docker pull codeclimate/codeclimate
    - mkdir -pv build/reports/codequality
    - cp .codeclimate.yml src
    - docker run --env CODECLIMATE_DEBUG=1 --env CODECLIMATE_CODE="$PWD/src" --volume "$PWD/src":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate validate-config
    - docker run --env CODECLIMATE_DEBUG=1 --env CODECLIMATE_CODE="$PWD/src" --volume "$PWD/src":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate analyze -f html > build/reports/codequality/codequality.html
    - docker run --env CODECLIMATE_DEBUG=1 --env CODECLIMATE_CODE="$PWD/src" --volume "$PWD/src":/code --volume /var/run/docker.sock:/var/run/docker.sock --volume /tmp/cc:/tmp/cc codeclimate/codeclimate analyze -f json > build/reports/codequality/raw_codequality.json
    - cat build/reports/codequality/raw_codequality.json
    - cat build/reports/codequality/raw_codequality.json | docker run -i stedolan/jq -c 'map({check_name,fingerprint,location})' > build/reports/codequality/codequality.json
    - rm src/.codeclimate.yml

So every time the runner starts it performs the docker pull codeclimate/codeclimate and since the runner hasn’t cached the last job execution, it has to download all the layers again.

My current cache section is like this (I’m not sure if this is working or not):

cache:
    key: "$CI_COMMIT_REF_NAME"
    paths:
      - build
      - .gradle
    untracked: true

I tried to remove also the line key: "$CI_COMMIT_REF_NAME" and nothing changed

Ok I missed the fact that you’re running the runner itself in a docker instance. You would have to create a volume for the runner docker that can cache your images. /var/lib/docker/overlay2 probably needs to be local rather than in the container.

BTW, does this actually work??

variables:
DOCKER_DRIVER: overlay2

I’d bet that this is a docker daemon setting and could not be manipulated this way but I have not tested this.

The configuration comes from the official gitlab website:
https://docs.gitlab.com/ee/ci/examples/code_climate.html

My config.toml is the following:

[[runners]]
  name = "DockerBuilder"
  url = "http://gitlab.mycompany.com"
  token = "mytoken"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker:latest"
    privileged = true
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock", "/etc/default/docker:/etc/default/docker", "/etc/docker/daemon.json:/etc/docker/daemon.json"]
    extra_hosts = ["localhost:172.X.X.X"]
    shm_size = 0
  [runners.cache]

Do you need other information?

Ahoy fabry!
Did you found a solution for the issue? I have the exact same problem…
Thanks!
Felix

I have the same issue. CodeClimate docker operations take forever to load. I have caching enabled in my runners.