How to use Security job artifacts in job dependencies?

GitLab CI/CD
1

I am attempting to do some post-processing on the Gitlab-provided security output. For experimentation, I am attempting to just ensure the artifacts are available for the post-processing job, however, it looks like only the container_scanning artifacts are available? Is there anything I’m obviously doing wrong?

I should also note that downloading the artifacts via Gitlab’s site/GUI works just fine.

We are using the gitlab.com, not self-hosted version.

Reduced (hopefully minimum-viable) .gitlab-ci.yml

stages:
  - test
  - security_report

include:
  - template: Jobs/SAST.gitlab-ci.yml
  - template: Jobs/Secret-Detection.gitlab-ci.yml
  - template: Jobs/Container-Scanning.gitlab-ci.yml

security_report:
  stage: security_report
  variables:
    GIT_STRATEGY: none
  dependencies:
    - semgrep-sast
    - secret_detection
    - container_scanning
  script:
    - ls

Actual Gitlab Runner output of security_report job:

Running with gitlab-runner 15.7.1 (6d480948)
  on ovh-cloud-runner eJatGgP2
Resolving secrets
00:00
Preparing the "docker" executor
00:02
Using Docker executor with image python:3.11 ...
Pulling docker image python:3.11 ...
Using docker image sha256:b44268c8cbc09471cdd06e2da2c3db6d41dd303b7a19560701945060bbda1d6a for python:3.11 with digest python@sha256:a3c0c6766535f85f18e7304d3a0111de5208d73935bcf1b024217005ad5ce195 ...
Preparing environment
00:00
Running on runner-ejatggp2-project-37442466-concurrent-0 via c2-15-bhs5...
Getting source from Git repository
00:01
Skipping Git repository setup
Skipping Git checkout
Skipping Git submodules setup
Downloading artifacts
00:00
Downloading artifacts for container_scanning (3632111085)...
Downloading artifacts from coordinator... ok        id=3632111085 responseStatus=200 OK token=64_kdyYt
Executing "step_script" stage of the job script
00:01
Using docker image sha256:b44268c8cbc09471cdd06e2da2c3db6d41dd303b7a19560701945060bbda1d6a for python:3.11 with digest python@sha256:a3c0c6766535f85f18e7304d3a0111de5208d73935bcf1b024217005ad5ce195 ...
$ ls
gl-container-scanning-report.json
gl-dependency-scanning-report.json
Cleaning up project directory and file based variables
00:00
Job succeeded

Expected:
I would expect additional lines in the output like:

...
Downloading artifacts for semgrep-sast (12341234)...
Downloading artifacts for secret_detection (09870987)...
Downloading artifacts for container_scanning (3632111085)...

...
$ ls
gl-container-scanning-report.json
gl-dependency-scanning-report.json
gl-secret-detection-report.json
gl-sast-report.json
...
2

Figured it out. Here is the artifacts section of the included YAMLs:

Jobs/Container-Scanning.gitlab-ci.yml

# ...
artifacts:
    reports:
      container_scanning: gl-container-scanning-report.json
      dependency_scanning: gl-dependency-scanning-report.json
    paths: [gl-container-scanning-report.json, gl-dependency-scanning-report.json]
# ...

Jobs/SAST.gitlab-ci.yml

# ...
artifacts:
    reports:
      sast: gl-sast-report.json
# ...

Jobs/Secret-Detection.gitlab-ci.yml

# ...
artifacts:
    reports:
      secret_detection: gl-secret-detection-report.json
# ...

The difference between the “working” Container-Scanning and the others is the lack of artifacts:paths.

To fix, the revised .gitlab-ci.yml looks more like:

stages:
  - test
  - security_report

include:
  - template: Jobs/SAST.gitlab-ci.yml
  - template: Jobs/Secret-Detection.gitlab-ci.yml
  - template: Jobs/Container-Scanning.gitlab-ci.yml

semgrep-sast:
  artifacts:
    paths: [gl-sast-report.json]

secret_detection:
  artifacts:
    paths: [gl-secret-detection-report.json]

security_report:
  stage: security_report
  variables:
    GIT_STRATEGY: none
  dependencies:
    - semgrep-sast
    - secret_detection
    - container_scanning
  script:
    - ls

The new output is like:

Running with gitlab-runner 15.7.1 (6d480948)
  on ovh-cloud-runner eJatGgP2
Resolving secrets
00:00
Preparing the "docker" executor
00:01
Using Docker executor with image python:3.11 ...
Pulling docker image python:3.11 ...
Using docker image sha256:b44268c8cbc09471cdd06e2da2c3db6d41dd303b7a19560701945060bbda1d6a for python:3.11 with digest python@sha256:a3c0c6766535f85f18e7304d3a0111de5208d73935bcf1b024217005ad5ce195 ...
Preparing environment
00:00
Running on runner-ejatggp2-project-37442466-concurrent-0 via c2-15-bhs5...
Getting source from Git repository
00:01
Skipping Git repository setup
Skipping Git checkout
Skipping Git submodules setup
Downloading artifacts
00:02
Downloading artifacts for container_scanning (3632708768)...
Downloading artifacts from coordinator... ok        id=3632708768 responseStatus=200 OK token=64_mynm4
Downloading artifacts for secret_detection (3632708763)...
Downloading artifacts from coordinator... ok        id=3632708763 responseStatus=200 OK token=64_mynm4
Downloading artifacts for semgrep-sast (3632708762)...
Downloading artifacts from coordinator... ok        id=3632708762 responseStatus=200 OK token=64_mynm4
Executing "step_script" stage of the job script
00:00
Using docker image sha256:b44268c8cbc09471cdd06e2da2c3db6d41dd303b7a19560701945060bbda1d6a for python:3.11 with digest python@sha256:a3c0c6766535f85f18e7304d3a0111de5208d73935bcf1b024217005ad5ce195 ...
$ ls
gl-container-scanning-report.json
gl-dependency-scanning-report.json
gl-sast-report.json
gl-secret-detection-report.json
Cleaning up project directory and file based variables
00:00
Job succeeded
1 Like