How to have a single cache by pipeline?

GitLab CI/CD
1

Hi all,

I’m trying to set up a pipeline that would have the following logic of jobs :

  • first tests in the runner
  • signing an ssh key for external server access (copy in a cache folder)
  • new tests on external server thanks to the key
  • deploy on server thanks to the key

I was thinking of using the cache mechanism … but it’s persistent between different pipelines:

“By default caching is enabled and shared between pipelines and jobs, starting from GitLab 9.0”

I do not want to trust of the users by asking to think well to delete the cache at the end of the pipeline (some actions are manual … and therefore entails a risk of access to the cache).

I tried to use the artifacts … but they become available via the interface … and I can only define their duration.

Is there a way to pass between certain jobs of a pipeline some files in a secure way and limited to this single pipeline?

Thanks in advance :slight_smile:

1 Like
2

Hello Lord,
do you have any progress with this? I have exactly same question.

3

Hi,

No progress on this : i have choose to not use any cache pour theses operation… i’m re-signing a key on each stage, only way to secure my access :confused:

4

You can do it by specifying a cache key, that is unique to your current pipeline.

A suitable key for a specific pipeline, that shares between jobs of a particular pipeline would be: $CI_PIPELINE_ID

In Context:

build:
  stage: build
  before_script:
  - echo Hello Before
  script:
  - echo Hello Script
  cache:
    key: "$CI_PIPELINE_ID"
    paths:
    - my/path

Subsequent jobs in the current pipeline than have access to the stuff in my/path, Other pipelines won’t.

Further Resources

5

Hello,

Thanks for your answer, but i don’t see how to remove cache when pipeline ended.
If i add in script :

- find /cache -ls

You will have :

$ find /cache -ls
 39425    0 drwxrwxrwx   3 root     root           60 Nov  8 09:08 /cache
 62503    0 drwx------   3 root     root           60 Nov  8 09:08 /cache/group
 62504    0 drwx------   4 root     root           80 Nov  8 09:21 /cache/group/test-pipelines
 64359    0 drwx------   2 root     root           60 Nov  8 09:21 /cache/group/test-pipelines/989
 64360    4 -rw-------   1 root     root          388 Nov  8 09:21 /cache/group/test-pipelines/989/cache.zip
 62505    0 drwx------   2 root     root           60 Nov  8 09:12 /cache/group/test-pipelines/988
 63237    4 -rw-------   1 root     root          600 Nov  8 09:12 /cache/group/test-pipelines/988/cache.zip

I am in $CI_PIPELINE_ID 989… but cache from $CI_PIPELINE_ID 988 is accessible.

How can we purge cache when pipeline ended ?

6

I guess if to remove files from a cached folder on a final “cleanup” step they will be removed from a cache…

I did not test it

7

Hi, thanks for the idea of a final “cleanup” step… but as it’s in the gitlab-ci file, user can remove it… change it… as i say in my first post :

IMHO (sysop side), this task must be done by gitlab/gitlab-runner for the sake of pure security :wink: