Can't download artifact using $CI_JOB_TOKEN

GitLab CI/CD
1

I have a very simple pipeline:

build1:
  stage: build
  script: 
    - echo "Before script section" >> docker-compose.yml
  artifacts:
    paths:
      - docker-compose.yml

test3:
  script:
    - >
      curl --location --output artifacts.zip
      "${CI_API_V4_URL}/projects/10/jobs/${CI_JOB_ID}/artifacts"
      --header "TOKEN: ${CI_JOB_TOKEN}"
    - ls -lah
    - cat artifacts.zip

I’ve tried different URLs to download the artifact:
“${CI_API_V4_URL}/projects/${CI_PARENT_PROJECT_ID}/jobs/artifacts/${CI_PARENT_PIPELINE_ID}/download?job=build1”
${CI_API_V4_URL}/projects/10/jobs/artifacts/main/download?job=build1
I’ve tried to append the job token to the url (?job_token=$CI_JOB_TOKEN)
And I’ve tried countless other variations.
What I get is either a 404 (Project Not Found) or 401 errors.
When I access ${CI_API_V4_URL}/projects using CI_JOB_TOKEN, I get an empty list.
In Settings>CI/CD>Token access the project has access to itself.
Before you suggest to use another way, the tasks are supposed to run in two different projects, but if it’s not working in a simple project, I can’t really move any further.
When I use the URL in the browser while being logged in as a regular user, I can download the artifacts just fine, so the URL is alright. It’s the token that is the problem.
I’m using an omnibus v16.4

2

When I create a personal or a project tokens with all the permissions, I get the same thing. But I can clone a project using a token just fine.

3

First thing, why do you download artifact manually? Artifacts from previous stages are automatically downloaded for you in the background.

You can’t use ${CI_JOB_ID}, because that is different for each job. You are trying to download artifact of the test3 job which is running the curl. Obviously, you get 404 since there is no such thing.

4

I’m downloading it manually because the idea is to trigger a pipeline in a different project, which needs artifacts from the main project. And there doesn’t seem to be a better way to do it.
You are right about the ${CI_JOB_ID}, but that’s just my mistake of simplifying the pipeline. In real life I save the ${CI_JOB_ID} in dotenv file and pass it to the next job.

5

There is a native way to achieve that, but it’s only for paid subscriptions, which can’t be used if you don’t have one.

Have you tried to use different method to authenticate, like Personal/Project/Group/Deploy token?

While using CI_JOB_TOKEN, if the parent project has the Token Access feature enabled, you need to add the child project to the allowed list.

6

Here is a fuller version of my pipeline:

unit_test:
  stage: test
  environment: dev
  tags:
    - webcc
    - test
  script:
    - echo "test" >> docker-compose.yml
    - echo "BUILD_TAG=$TAG" >> build.env
    - echo "CI_PARENT_JOB_ID=$CI_JOB_ID" >> build.env
  artifacts:
    paths:
      - docker-compose.yml
    reports:
      dotenv: build.env


test_downloading_artifact:
  stage: test
  environment: dev
  tags:
    - webcc
    - test
  variables:
    CI_DEBUG_TRACE: "true"
  script:
    - echo "${CI_PARENT_JOB_ID}"
    - >
      curl --location --output artifacts.zip
      "${CI_API_V4_URL}/projects/129/jobs/${CI_PARENT_JOB_ID}/artifacts"
      --header "TOKEN: ${CI_JOB_TOKEN}"
    - cat artifacts.zip
  needs:
    - job: unit_test
      artifacts: true

I’ve tried creating a project and personal tokens. Both give me the same result. I have Token Access feature enabled in the parent project, but for now I’m just trying to do it within the same project.
But again, when I acesss the ${CI_API_V4_URL}/projects/129/jobs/${CI_PARENT_JOB_ID}/artifacts URL from debug, while being logged in as myself, I can download the files just fine.
I’ve tried two different installation of gitlab (on is v16.4 and the other is v14.10.5). Neither version is a fresh install, both have been updated before, so may be I missed an important update, although I followed the guide and gitlab-ctl reconfigure returns no errors.

cat artifacts.zip
{"message":"404 Project Not Found"}
7

That pipeline works for me on an instance with Premium license. It’s not really clear from the docs and it took me a while to understand, but CI_JOB_TOKEN can be used only in paid tiers.

Personal Access Tokens should work, tho.

8

Thanks for going above and beyond for me, unfortunately, personal tokens do not work either. I’ve created one the root user with all the rights, but when I access ${CI_API_V4_URL}/projects I still get an empty list. Downloading an artifact once again gives me ```
{“message”:“404 Project Not Found”}

Same thing with v14.10.5 with a regular user.
9

We ended up using base64 to pass value in a variable. But may be it’s worth checking it out if there is some bug with using personal tokens.

- echo "COMPOSE=$(base64 -w0 docker-compose.yml)" >> build.env

10

This thread looks old, but maybe an answer may still be useful: in your curl example, you used an header named “TOKEN”. As per the documentation, the header needs to be named PERSONAL-TOKEN for PATs (including project tokens, group tokens a.s.o.) and JOB-TOKEN for CI_JOB_TOKEN.

Example, per your snippet:

      curl --location --output artifacts.zip
      "${CI_API_V4_URL}/projects/129/jobs/${CI_PARENT_JOB_ID}/artifacts"
      --header "JOB-TOKEN: ${CI_JOB_TOKEN}"
1 Like