Gitlab cache key file differences

GitLab CI/CD
1

We are using the cache option in the .gitlab-ci.yml file to cache the node_modules folder, based on the package.json and package-lock.json files:

build-angular:
  image: node:12.18.3
  stage: build
  cache:
    key:
      files:
        - src/MyApp.Web/ClientApp/package.json
        - src/MyApp.Web/ClientApp/package-lock.json
    paths:
      - src/MyApp.Web/ClientApp/node_modules
  script:
    - cd src/MyApp.Web/ClientApp
    - export PATH=$(pwd)/node_modules/.bin:$PATH
    - npm install

Our expectation is that after a successful build, any job that runs where the package.json and package-lock.json files are the same, it will use that cache (then with whatever changes the npm install does after that).

However, our observations are showing that we are getting different cache keys even for identical package.json and package-lock.json files. This usually has been resolved by modifying the package.json back, then reverting, to resolve the problem.

Our scenario is:

C1 => package.json (v1), package-lock.json (v1) => cache key == 12345
C2 => package.json (v2), package-lock.json (v1) => cache key == 45678
C3 => package.json (v1), package-lock.json (v1) => cache key == 67890

The documentation for the cache:key attribute states:

The cache key is a SHA checksum computed from the most recent commits (up to two, if two files are listed) that changed the given files.
Source: CI/CD YAML syntax reference | GitLab

It’s unclear though what that actually means. What is the purpose of looking back more than 2 commits if two files are listed? Shouldn’t it use the contents from the code when checking out the current commit and building?

2 Likes
2

I experience a similar issue where my job running npm ci creates a cache beginning with 1_package-lock but the downstream jobs try to pull a cache beginning with 0_package-lock …

The important bits are:

.node_modules_cache: &node_modules_cache
  key:
    files:
      - package-lock.json
  policy: pull
  paths:
    - node_modules

.install_cache: &install_cache
  cache:
    - <<: *node_modules_cache
      policy: pull-push

default:
  cache: &default_cache
    - <<: *node_modules_cache

install: # creates 1_package-lock-sha
  stage: prepare
  <<: *install_cache
  script:
    - npm ci --cache .npm --prefer-offline

lint: # tries to pull 0_package-lock-sha
  stage: test
  script:
    - npm run lint:ci

What the heck is going on?

/edit: zomfg - the answer’s here: CI jobs with multiple cache and cache🔑files experiencing cache miss due to new cache key prefix (#384390) · Issues · GitLab.org / GitLab · GitLab