Validate Secret Detection job not failing when secrets have been detected

Hello,
For context I am attempting to make use of GitLab’s Security/Secret-Detection.gitlab-ci.yml template. The goal is to use GitLab secret detection to detect secrets then use a Validate Secret Detection job to query the resulting report artifact to see if there are any findings in the vulnerabilities section of the JSON.

Here is what the .gitlab-ci.yml includes:

include:
  - template: Security/Secret-Detection.gitlab-ci.yml

stages:
  - secret_detection

secret_detection:
  stage: secret_detection

Validate Secret Detection:
  stage: secret_detection
  needs:
    - "secret_detection"
  script:
    - |
      if [ -f "$SECRET_DETECTION_REPORT_FILE" ]; then
        if [ "$(jq ".vulnerabilities | length" $SECRET_DETECTION_REPORT_FILE)" -gt 0 ]; then
          echo "Vulnerabilities detected. Please analyze the artifact $SECRET_DETECTION_REPORT_FILE produced by the 'secret-detection' job."
          exit 80
        fi
      else
        echo "Artifact $SECRET_DETECTION_REPORT_FILE does not exist. The 'secret-detection' job likely didn't create one. Hence, no evaluation can be performed."
      fi

I am having trouble understanding why the validation job is passing when GitLab’s Secret Detection job has discovered a finding and added it to the report.
Below are some screenshots showing that things are working just not failing in the pipeline when expected.

Pipeline jobs running in the secret_detection stage:

Output from the Secret Detection runner:

Resulting report artifact:

Resulting report (after downloading it to my local machine):

Thank you.

Some context for understanding the question: The secrets scanning feature, amongst other security scanners, uses CI/CD to scan and collect the results. The scan reports are available as MR widgets and vulnerability dashboards to take action upon. The CI/CD jobs themselves will not fail when something is detected - this needs a more granular approach with matching the security vulnerability with severities and CVEs for example. MRs can be blocked from merging when security vulnerabilities such as secrets are detected, using scan result policies.

Downloading the JSON report manually could work as well, but needs the reports artifact in the job scope. You can expose the JSON report as manual artifact for example.

A similar example with IaC SAST reports is described in Fantastic Infrastructure as Code security attacks and how to find them and in combination with How to use Security job artifacts in job dependencies? the following (untested) configuration should work:

secret_detection:
  stage: secret_detection
  artifacts:
    paths:
      - gl-secret-detection-report.json
    reports:
      secret_detection: gl-secret-detection-report.json


Validate Secret Detection:
  stage: secret_detection
  needs:
    - "secret_detection"
  artifacts:
    paths:
      - gl-secret-detection-report.json

Alternatively to needs with artifacts you could also use different stages and dependencies to force the artifact download.

Thank you very much for the response and Information @dnsmichi!