Passing artifacts from (build,test)-pipeline to (deploy)-pipeline

GitLab CI/CD
1

Hi!

I can’t figure out how Gitlab deals with artifacts between different pipelines. I am trying to set up a pipeline in which commits lead to builds and tests, after which a deployment can be triggered by a pipeline trigger.

I have a single project with a .gitlab-ci.yml file, in which I have defined 3 stages [in pseudo-yaml]:

stages:
- build
- test
- deploy

My build stage packages some python code, sql files, and config files in a dir, and stores it in an artifact.

build:
  stage: build
  script:
    - mkdir target
    - python setup.py [foo] --dist_dir=target
    - cp -r sqlscripts target
    - cp -r configfiles target
  artifacts:
    name: "build"
    paths:
    - target

The test stage tests the code but doesn’t do anything special with the artifact. The build and test stage always run for commits on a branch.

Next I have the deploy stage. This stage does not run on commits, but on triggers only.

deploy:
  stage: deploy
  dependencies:
    - build
  script:
    # Deploy to the unit.
    - deploy.sh
  only:
    - triggers

The problems

  1. I get two different pipelines. I wouldn’t expect this based on my ‘stages’ definition.
  2. I cannot find the build artifact. I think this is because I have two different pipelines. How do I get the artifact available in the deploy job?

My attempts

In order to get the build artifact, I have tried using the artifact API, adding the following lines to the deploy script entry:

- 'curl --location --output artifacts.zip "https://gitlab.com/api/v4/projects/my-id/jobs/artifacts/my-ref/download?job=build&job_token=$CI_JOB_TOKEN"'
- 'unzip artifacts.zip'

Unfortunately, this artifacts.zip file is not a zip file. Instead, curl gives a 404: project not found error. Is the CI_JOB_TOKEN only valid for Gitlab Premium?

My question

How do I use the artifact from the build stage in my triggered deploy stage?

2

Hi,

weird, I would have guessed that the pipeline consists of just 3 stages then. Can you please share the version of GitLab you are using? (e.g. via /help) I’d like to be sure that you’re not hitting a bug before looking into a reproducer.

Cheers,
Michael

3

Hi,

I am using gitlab.com, so AFAIK that’s the latest version.

Thanks!

4

It’s offered in premium: https://docs.gitlab.com/ee/ci/triggers/README.html#ci-job-token

forum
forum1177×475 28.7 KB

Mel

1 Like
5

Hi,

from a quick test last week (forgot to continue with the reply), I can only see that my deploy job is invisible. So the triggers has an influence here.

One other solution might be to push the artifact somewhere else in the last job, e.g. a nextcloud host and then fetch that in the triggers job. A binary asset repository/registry is not yet there within GitLab, we’ve been discussing that in the issue tracker lately.

Cheers,
Michael

6

@melissa, thanks! I also noticed that line in the trigger docs, but I didn’t take it to mean that the use of the CI_JOB_TOKEN was still exclusive to premium, 3 versions down the road.

@dnsmichi, thanks for the confirming. It indeed seems the triggered job is not part of the pipeline (ps, note that your ‘test’ job happens in the ‘build’ stage, although that shouldn’t affect the outcome of the test). For now I an rebuilding my application in the deploy stage (build time is not too long). I might check out nextcloud in the future!

Cheers,
Jasper