I have init job on the begging of my pipeline which product dotenv artifact. I want to use generated variables in all other jobs. This solution was fine unless I started to use “needs” for job dependencies. I realised, that when I have tree jobs:
init (produces dotenv artifact)
build (needs init)
deploy (needs build)
the deploy job don’t see the dotenv artifact until I explictily set needs to init.
As this is only example, in reality, the pipeline consist of much more jobs with various dependencies.
Should I put the dependency on init to all the jobs? Why gitlab-ci doesn’t count with “transitivity”?
I spent a whole lot of time banging my head against the wall, trying to understand this exact issue you’re describing.
The pipeline I’m building has a similar structure to yours, with a job in the init stage that authenticates against an AWS ECR to then store the password in a file managed as an artifact so subsequent stages can use it to push and pull images, without having each of the jobs authenticate each time – thus making the pipeline more efficient.
I can think of several ways to work around this, and the one I like best so far involves using an S3 bucket to store our artifacts.
Another solution would be implementing a block storage to avoid wasting time interacting with the bucket, but this approach will require further research…
