Define/adjust cache policy at runtime

GitLab CI/CD
1

Define/adjust cache policy at runtime

When using gitlab cache for a job I would like to achieve the following:

  • If the cache doesn’t exists (empty) I want my job act as if the policy was push;
  • if the cache exists (not empty) I want the job to act as if the policy was pull;

Currently, with pull-push policy I spend about the same time doing push as I would spend on building cache content.
Inside the script I know what case it is (after checking if the the directory is empty). Is there a way to tell gitlab that I prefer not to do a push for a given cache?

2

Nope. There is nothing like that.

3

My current findings show that you can trick gitlab and make it act as if you switch to cache.policy pull from pull-push.

To do so you:

  1. set cache policy to pull-push.
  2. On the start (e.g. in before_script) you check the folder and decide if it is empty or not. if it is empty then you want to execute push part of pull-push policy and if cache is not empty you want to skip it. In my case I create an empty file (touch CI_CACHE_IS_NEW) to indicate that the cache was empty.
  3. After job is done I check if CI_CACHE_IS_NEW file exists and if so then that’s it (do nothing or some cleanup in cache directory) otherwise I want to cancel the push part. And here is a dirty trick: I rename (mv cache_dir cache_dir_skipme) the cache folder so that gitlab would not see one. Luckily, gitlab doesn’t consider this to be an error and job succeeds (takes about 2-5 seconds).

The downside is that I get a warning:

WARNING: cache_dir/: no matching files. Ensure that the artifact path is relative to the working directory (/builds/whatever/the/project/path) 
Archive is up to date!

And what bothers me with that solution is how reliable is that. I don’t want to find myself one day that gitlab starts treating “missing cache dir” as an error.