Secret_detector fails, no artifacts are uploaded

I see the secret_dector finding issues with the code, but not uploading any details about the failure.

Output log:

time="2021-03-31T05:16:48Z" level=error msg="object not found"

exit status 1
Uploading artifacts for failed job
Uploading artifacts...
WARNING: gl-secret-detection-report.json: no matching files 
ERROR: No files to upload                          
Cleaning up file based variables
ERROR: Job failed: command terminated with exit code 1

This is an on prim gitlab
.gitlab-ci.yml

include:
  - project: devops/ci-jobs
    file: loadable/pipeline.yml
  - project: devops/ci-jobs
    file: targets/x64-next.yml

I didn’t find anything useful from the troubleshooting page:
Secret Detection | GitLab
This mentions downloading the artifacts, but no artifacts are uploaded for this test stage…

Hi @hickersonj
I can see from the Job output that the secret detector failed and exited with status 1. The error is also there level=error msg="object not found"
I would guess the report is not generated and that’s why it is not uploaded.

Thank you for the feedback.
Here is the full log, so I’m not quite sure why there is no output (I’ve removed the URL info from the log):

Running with gitlab-runner 13.8.0 (775dd39d)

[2](-/jobs/1149651#L2) on gitlab-runner-helm3-gitlab-runner-56b8bddf45-mz4gb 2qTJhKPf

[3](-/jobs/1149651#L3)Resolving secrets

00:00

[5](-/jobs/1149651#L5)Preparing the “kubernetes” executor

00:00

[6](-/jobs/1149651#L6)Using Kubernetes namespace: default

[7](-/jobs/1149651#L7)WARNING: Pulling GitLab Runner helper image from Docker Hub. Helper image is migrating to registry gitlab com, for more information see /runner/configuration/advanced-configuration.html#migrating-helper-image-to-registrygitlabcom

[8](-/jobs/1149651#L8)Using Kubernetes executor with image $SECURE_ANALYZERS_PREFIX/secrets:$SECRETS_ANALYZER_VERSION …

[10](-/jobs/1149651#L10)Preparing environment

00:04

[11](-/jobs/1149651#L11)Waiting for pod default/runner-2qtjhkpf-project-4009-concurrent-1pwtjp to be running, status is Pending

[12](-/jobs/1149651#L12)Running on runner-2qtjhkpf-project-4009-concurrent-1pwtjp via gitlab-runner-helm3-gitlab-runner-56b8bddf45-mz4gb…

[14](-/jobs/1149651#L14)Getting source from Git repository

00:54

[15](-/jobs/1149651#L15)Fetching changes with git depth set to 50…

[16](-/jobs/1149651#L16)Initialized empty Git repository in /builds/.git/

[17](-/jobs/1149651#L17)Created fresh repository.

[18](-/jobs/1149651#L18)Checking out f8950db7 as build-utils…

[19](-/jobs/1149651#L19)Skipping Git submodules setup

[21](-/jobs/1149651#L21)Downloading artifacts

00:27

[22](-/jobs/1149651#L22)Downloading artifacts for x64-next (1149647)…

[23](-/jobs/1149651#L23)Downloading artifacts from coordinator… ok id=1149647 responseStatus=200 OK token=hj-dncLV

[25](-/jobs/1149651#L25)Executing “step_script” stage of the job script

00:45

[26](-/jobs/1149651#L26)$ if [[ $CI_COMMIT_TAG ]]; then echo “Skipping Secret Detection for tags. No code changes have occurred.”; exit 0; fi

[27](-/jobs/1149651#L27)$ git fetch origin $CI_DEFAULT_BRANCH $CI_COMMIT_REF_NAME

[28](-/jobs/1149651#L28)From https ://builds

[29](-/jobs/1149651#L29) * branch master → FETCH_HEAD

[30](-/jobs/1149651#L30) * branch build-utils → FETCH_HEAD

[31](-/jobs/1149651#L31) * [new branch] master → origin/master

[32](-/jobs/1149651#L32)$ git log --left-right --cherry-pick --pretty=format:“%H” refs/remotes/origin/$CI_DEFAULT_BRANCH…refs/remotes/origin/$CI_COMMIT_REF_NAME > “$CI_COMMIT_SHA”_commit_list.txt

[33](-/jobs/1149651#L33)$ export SECRET_DETECTION_COMMITS_FILE=“$CI_COMMIT_SHA”_commit_list.txt

[34](-/jobs/1149651#L34)$ /analyzer run

[35](-/jobs/1149651#L35)[INFO] [secrets] [2021-03-31T05:02:18Z] GitLab secrets analyzer v3.17.0

[36](-/jobs/1149651#L36)[INFO] [secrets] [2021-03-31T05:02:18Z] Detecting project

[37](-/jobs/1149651#L37)[INFO] [secrets] [2021-03-31T05:02:18Z] Found project in /builds

[38](-/jobs/1149651#L38)[INFO] [secrets] [2021-03-31T05:02:18Z] Running analyzer

[39](-/jobs/1149651#L39)[DEBU] [secrets] [2021-03-31T05:02:18Z] /builds/.gitlab/secret-detection-ruleset.toml not found, ruleset support will be disabled.

[40](-/jobs/1149651#L40)[ERRO] [secrets] [2021-03-31T05:03:02Z] /usr/local/bin/gitleaks --report /tmp/gitleaks-138163939.json --path /builds --config-path /gitleaks.toml --leaks-exit-code 0 --commits f8950db73b172125c5944531cc3a78d4ac8d6e6a,7b627f4a0fdee9f83837898243ad6a103bec2ac8,5efbc78e77a97b446e8e8ab541666fbac0782334,b3312ac87e73ee3afb69f9a28d4e9ddb052b7b20,17aab9fe06eb7672c3666f5631069e02c0a0b65e,216f9ff44681456aa96a5b71bfda5d238278bc15,39d1f9695e071c1fd8a0f0dcedee053323b322df,dee40196bf5e770119ae19b397b15fa39437c3c5,5d7513fda83a16fe393d9968bbc8866b49e80ab6,9254979f9808581178fce24cb8ab73d3e2f216a3,fff837534d96b9ac151c11a5bd470eaff8f4edc1,d6f334038fcefcb9823af35e4481d222498f3c80,3948ce1debe63b43270332a13c192e408c9a9dff,02a6f3f9fb775767d9fc6af3b7aadd2e8ace91be,7ffc540252082617f717bed34cec9ef4bff86959,d1a1c66e09633e9aa6921fceb4476326c0ccc721,3c1e2c078aad874d94ec4bc31ee98fbd3f44b973,708a2a2fdff3d13ad0216cc5a85eefd36d85fca5,4d2d0faaf9b998e782ae516c2a302ffa796e86d8,16b569a315224a437a9998c8eb16062b03dc283f,7204a9693f184a6ce4fcff10c2f39b6588623ea7,b2af473d97746638398fa50cc6eed948f72763cc,6a119250707071dc6c58b2e38a04f8350fe99644,0a5a47f7d45ab6a7afc2fcebad2fdbbf8a94d700,fedc22fd2d8c9eef22e27b18c8a0cbefa371c761,4bac5464092324a7ec392b1d8159dc6e62f97fa2,0aaa965ff4f7cd4dd370c70c25d710632b9bfe2b,929890e309c569a0976b66c21ff7b4414214e854,7ff1f84df0cede9ed7861b8565158d479c6c83d0,c7ff6e6da477049b45e9ed63bef3fc3652c39fc3,ac49547ca78b48865f274410f8c08e627444f131,fbacf2be501891c6864336b01216a5a1a5a2a3ca,88554657c1bddcb4ebbd1d8ad0ea4d9e8922123f,602e083de5669c57ff25edda8bf9a3287a7a3433,c7edee9e56b23de80d9a5889b3219a1b92e95b18,198043da0174da0dd04626bd3d733323bdb3cc1a,6e79c4c441215854c853e5d1b15b273e4f5ffddd,73f0bd9710d15cd6beff8a6b1252f7efdc013e92,dd3e8703c7cad2df084c71ea2edebb37ecd8f6b5,7eb012598621eeaef1e17ead1ad92e3b7f33d743,7b9b8587b98128df07ba37fab9d163960820437f,988af813571425a09f6d2759e815f43c3886aaeb,77df9ee2884e636d0b535ff527de62537caeeabe,15b5e5d76d706ad255120cdceb0aede8d87a2889,df644cbf8f0464155477109419a5552c0880ae63,987f63d5b9e7571cef3f7736170b1113799aa0e8,fadb72f425af4b1569316c2c2202cdd8f58e0c5a,328211439fa04b11e86130c6f490e869777cf303,bb7dc773c552f7557ca6fd8fdfa6c1f4155245b1,760d8f1ca3fd570b584e77aaa4dc4932682a2b09,f8b2f5bceab0dcf69c187dfa034fbc9b53defea3,b03d088775f9705849d2fedb2af025ca3d25b4d7,c84c13f1ba789e4ed603ef1285e3d6a8c16eccee,1c349300d662913eb7d211f5fca3048cb9972b3a,517645940fecbc68edeb828f9245711685e2c53a,c4b937f20dafb5bc2ec6ffaebfd95472c5657424,5419b908e83550de6f6fcd224b25c8fcfa165c1d,00e367874803dab198c2deb311b112761e6333c6,070172ff6f5c02af970a504c80c60f8ae97bd900,2e8a8f31bf82c6c36b56dad07ad3cbcaa3c95d56,e5e8871f8aae2b83b3d2f339642fcdf4a2997e18,b3a8657cccb3d0956bc12544d2c00c5d8f9f7595,dac74c2446971d6cc9c91d0dd9f005e94401c03a,6cc3a18e8e27726281e9c2e62a866e3db0f34d52,960c96c2e37d8fecc5089c054043c080c574a7fc,e418e9e80e0457bf857763631df8733aac60ebb5,fab26025576b26f84587f119d62c16a192bbc9e4,42cd1870809bd1b622a0b466cfcd911b2a084436,426e4489d7b7c7e5c4d6c83e15a51d3eef3d91ee

[41](-/jobs/1149651#L41)time=“2021-03-31T05:02:18Z” level=info msg=“opening /builds\n”

[42](-/jobs/1149651#L42)time=“2021-03-31T05:03:01Z” level=info msg=“scan time: 43 seconds 900 milliseconds 390 microseconds”

[43](-/jobs/1149651#L43)time=“2021-03-31T05:03:01Z” level=error msg=“object not found”

[44](-/jobs/1149651#L44)exit status 1

[46](-/jobs/1149651#L46)Uploading artifacts for failed job

00:00

[47](-/jobs/1149651#L47)Uploading artifacts…

[48](-/jobs/1149651#L48)WARNING: gl-secret-detection-report.json: no matching files

\ 49ERROR: No files to upload

[51](-/jobs/1149651#L51)Cleaning up file based variables

00:00

[53](-/jobs/1149651#L53)ERROR: Job failed: command terminated with exit code 1

I found this issue that I think is related. It is mostly about the GIT_DEPTH variable. This seems like the secret detection is missing commits. Try to set the GIT_DEPTH variable to some greater number like 100.

That fixed it thanks!

However, that variable doesn’t seem to be clearly defined or at least what it should be set to. I finally got it to work by setting it to 10, initially I set it to 100 and that failed with a signal: killed. Assuming if too high it might be timing out or possibly using too much memory? Also if set to 0, the error is that it needs to be > 2.

Is 10 ok? How do I determine what is ok?

Thanks.

GIT_DEPTH sets the git clone --depth parameter when the project is cloned in Job

--depth <depth>
Create a shallow clone with a history truncated to the specified number of commits.

The default value for all projects created in GitLab 12.0 and later is 50. In projects created in older versions it is blank (unset). You can check the default value for your project in Settings → CI/CD → General pipelines in Git shallow clone field.

The Secret Detection docs also specify that the GIT_DEPTH must be higher than number of commits in Merge Request.

The following worked for me. Without GIT_CHECKOUT: "false", it was still failing.

    GIT_STRATEGY: clone
    GIT_CHECKOUT: "false" # https://docs.gitlab.com/ee/ci/runners/configure_runners.html#git-checkout
    GIT_DEPTH: "0"