What is the cache_dir of the docker executor?

AndreKR · October 18, 2016, 12:40pm

At least with the default configuration, every build container gets a volume mounted to /cache.

What is this for? Can my build script use that freely as a cache (e.g. for a package manager) or is it a technical requirement for the normal runner cache to work?

srghma · March 11, 2019, 9:09am

/cache volume is added automatically on gitlab-runner installation and managed by cache_dir setting

more about cache_dir:
https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section

gitlab.com

gitlab-org/gitlab-runner/blob/master/docs/executors/docker.md#the-builds-and-cache-storage

# The Docker executor

GitLab Runner can use Docker to run jobs on user provided images. This is
possible with the use of **Docker** executor.

The **Docker** executor when used with GitLab CI, connects to [Docker Engine]
and runs each build in a separate and isolated container using the predefined
image that is [set up in `.gitlab-ci.yml`][yaml] and in accordance in
[`config.toml`][toml].

That way you can have a simple and reproducible build environment that can also
run on your workstation. The added benefit is that you can test all the
commands that we will explore later from your shell, rather than having to test
them on a dedicated CI server.

The following table lists what combinations of containers, executors,
and OS are supported.

| Container Type    | Executor         | OS Type | Supported |
|:-----------------:|:----------------:|:-------:|:---------:|

This file has been truncated. show original

If you modify the /cache storage path, you also need to make sure to mark this
directory as persistent by defining it in volumes = ["/my/cache/"] under the
[runners.docker] section in config.toml.

TLDR

/cache volume is different from cache config in gitlab-ci.yml

/cache dir in job container is where the cached files are stored
files matching to cache config in gitlab-ci.yml are copied to /cache/CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number> at the end of job
The “Clear Runner Caches” button schedules NOT TO extract /cache/CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number>/cache.zip to dir specified in cache config in gitlab-ci.yml

P.S.

There is container named gitlab-runner-cache (https://gitlab.com/gitlab-org/gitlab-runner/blob/af343971874198a1923352107409583b78e8aa80/executors/docker/executor_docker.go#L382)
Seems like this container is used to create anonymous volume where /cache data is stored and then stopped
Seems like the job containers mount this anonymous volume using --volumes-from

I may be wrong

Proofs

HAVING gitlab-ci.yml

image: srghma/docker-nixos-with-git-crypt

cache:
  key: "test00000" # to reset cache - change this key OR clear cache in project settings page
  paths:
    - .mycache # gitlab allows only cache dirs that are relative to project root OR /cache (created automatically)

testtest:
  script:
    - nix-env -i tree

    - tree --dirsfirst -L 4 /cache
    - ls -al ./.mycache || true

    - echo "test" > /cache/test
    - mkdir -p ./.mycache
    - echo "test" > ./.mycache/test

    - tree --dirsfirst -L 4 /cache
    - ls -al ./.mycache || true

Output:

on first run

Running with gitlab-runner 11.6.0 (f100a208)
  on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via stage.getgreenbadger.com...
Fetching changes...
Removing .mycache/
HEAD is now at 675caa7 feat: cache update
From https://gitlab.nordicresults.com/srghma/greenbadger
   675caa7..3d1e223  nix        -> origin/nix
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-11...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
  /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache

0 directories, 0 files
$ ls -al ./.mycache || true
$ echo "test" > /cache/test
ls: ./.mycache: No such file or directory
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
`-- test

0 directories, 1 file
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:44 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:44 ..
-rw-r--r--    1 root     root             5 Feb 24 11:44 test
Creating cache test00000-11...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded

on second run

Running with gitlab-runner 11.6.0 (f100a208)
  on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via stage.getgreenbadger.com...
Fetching changes...
Removing .mycache/
HEAD is now at 3d1e223 feat: cache update
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-11...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
  /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- greenbadger
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:44 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:44 ..
-rw-r--r--    1 root     root             5 Feb 24 11:44 test
$ echo "test" > /cache/test
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- greenbadger
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:44 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:44 ..
-rw-r--r--    1 root     root             5 Feb 24 11:44 test
Creating cache test00000-11...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded

after you clear cache by clicking on “Clear Runner Caches” in your project “Pipelines” page

Running with gitlab-runner 11.6.0 (f100a208)
  on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via stage.getgreenbadger.com...
Fetching changes...
Removing .mycache/
HEAD is now at 3d1e223 feat: cache update
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-12...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
  /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- greenbadger
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
ls: ./.mycache: No such file or directory
$ echo "test" > /cache/test
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
|   `-- greenbadger
|       `-- test00000-11
|           `-- cache.zip
`-- test

3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x    2 root     root          4096 Feb 24 11:45 .
drwxrwxrwx   20 root     root          4096 Feb 24 11:45 ..
-rw-r--r--    1 root     root             5 Feb 24 11:45 test
Creating cache test00000-12...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded

Topic		Replies	Views
Runner Cache with Docker Executer GitLab CI/CD ci , runner , docker	0	260	September 19, 2024
Cache in self-managed Runner GitLab CI/CD	2	5329	September 2, 2017
gitlab-runner kubernetes cache GitLab CI/CD	2	7110	April 9, 2018
Gitlab-runner exec docker: local cache? GitLab CI/CD ci , runner	1	2097	September 21, 2019
Caching example from docs not working GitLab CI/CD	2	1382	November 16, 2018

What is the cache_dir of the docker executor?

TLDR

Proofs

Related topics