How can I speed up cache loading on a docker executor, which is slower than the shell executor?

GitLab CI/CD
,
1

Hi,

I’m attempting to switch from shell executors to docker executors in the custom gitlab setup at work. When switching to the docker executor setup, cache loading takes upwards of three minutes. When using a shell executor, cache loading takes seconds.

I’ve tried setting CACHE_COMPRESSION_LEVEL to fastest in the pipeline itself, but this has made no difference. There’s possibly something obvious I’m missing, but how can I speed this up?

pipeline log output…

Restoring cache   03:14
Checking cache for my_cache_key-protected...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. 
Successfully extracted cache

config.toml for the docker runner

concurrent = 8
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "*"
  url = "*"
  token = "*"
  executor = "docker"
  environment = ["*"]
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "<CUSTOM IMAGE>"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/certs/client", "/jdk-11/:/jdk-11/", "/apache-maven-3.3.9:/apache-maven-3.3.9", "/maven-mvnd-1.0-m6-m39-linux-amd64:/maven-mvnd-1.0-m6-m39-linux-amd64", "/etc/localtime:/etc/localtime", "/usr/local/bin/docker-compose:/usr/local/bin/docker-compose", "/root/.m2/:/root/.m2/", "/etc/profile.d/BuildTools.sh:/etc/profile.d/BuildTools.sh", "/root/mavend_conf:/root/mavend_conf", "/var/run/docker.sock:/var/run/docker.sock", "/usr/bin/docker:/usr/bin/docker", "/etc/yum.repos.d/:/etc/yum.repos.d/", "/etc/pki/rpm-gpg/:/etc/pki/rpm-gpg/", "/etc/pki/rhui/:/etc/pki/rhui/", "/usr/local/bin/lsof:/usr/local/bin/lsof", "/var/lib/docker:/var/lib/docker", "/workdir:/workdir", "/cache"]
    cache_dir = "/efs_docker_images/gitlab-runner-build-cache"
    shm_size = 0

config.toml for the shell runner

concurrent = 2
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "*"
  url = "*"
  token = "*"
  executor = "shell"
  environment = ["*"]  # Identical to docker executor
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]

Thanks in advance

2

Hi,

it seems you have set the cache dir to be "/efs_docker_images/gitlab-runner-build-cache", but you are not mounting it anywhere in volumes so it’s not really used.

cache_dir:
Absolute path to a directory where build caches are stored in context of selected executor. For example, locally, Docker, or SSH. If the docker executor is used, this directory needs to be included in its volumes parameter.

Unless you have symlink from /cache to /efs_docker_images/gitlab-runner-build-cache you have not mentioned.

1 Like
3

Hi, thanks for your answer. There wasn’t a symlink; I mounted it in volumes but it hasn’t affected the cache loading speeds