Secrete Detector Fails

I used the Secret Detector in my CICD:

image: php7.4:latest
before_script:   
  - apt-get update -yqq  
  # Setup SSH deploy keys
  - 'which ssh-agent || ( apt-get install -qq openssh-client )'
  - eval $(ssh-agent -s)
  - ssh-add -D   
  - echo "$SSH_STAGE_PRIVATE_KEY" | tr -d '\r' | ssh-add -
  - mkdir -p ~/.ssh
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
  - composer install 
  test-job1:
     stage: test
     script:    
       - echo "TO DO"

include:
    - template: Security/Secret-Detection.gitlab-ci.yml

However, when the pipeline runs in my self-hosted Gitlab installation, the job secret_detection fails:

$ apt-get update -yqq
24/bin/sh: eval: line 108: apt-get: not found
26
Uploading artifacts for failed job
00:00
27Uploading artifacts...
28WARNING: gl-secret-detection-report.json: no matching files 
29ERROR: No files to upload

Hello gdm,

Is this your complete .gitlab-ci.yml file?

Based on your output it looks like the image used by the Secret-Detection template does not have apt-get and thus fails prior to running the scan.

It looks like the image is alpine based (I found this by running the following command as a part of my before_script)

$ cat /etc/os-release || true
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.13.2
PRETTY_NAME="Alpine Linux v3.13"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"

if you don’t need your default/global before_script to be used by the secret-detection job, change the nesting so it’s only where it’s needed.

Cheers,
Thomas

Hi @gdm
Globally-defined before_script and image are deprecated and shouldn’t be used.
Specify them per-job.

Thank @balonik. I changed a little bit by using anchors:

default:
  image: japitase/php-devops:latest
  stages:
    - build
    - test
    - quality
    - staging
    - production

  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - vendor/

# Install keys
.deploy-scripts: &deploy-scripts |
    echo "SSH stuff..."
    apt-get update -yqq      
    which ssh-agent || ( apt-get install -qq openssh-client )
    eval $(ssh-agent -s)
    ssh-add -D   
    echo "$SSH_STAGE_PRIVATE_KEY" | tr -d '\r' | ssh-add -
    mkdir -p ~/.ssh
    '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
    
.prepare: &prepare | 
    composer install 

test-job1:
  stage: test
  script:    
    - echo "TO DO"

quality-test:
  
  stage: quality
  script:   
    - *prepare     
    - ./vendor/bin/phpstan analyse -c ci/phpstan.neon --memory-limit=-1
  allow_failure: true

include:
    - template: Security/Secret-Detection.gitlab-ci.yml

deploy-to-stage:
  stage: staging   
  
  script:        
    - *deploy-scripts 

Now the secret-detection is run, but I got an error for the quality-test:

Checking cache for develop-1...

[19]No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
[20]Successfully extracted cache
[22]Executing "step_script" stage of the job script
[23]$ ./vendor/bin/phpstan analyse -c ci/phpstan.neon --memory-limit=-1
[24]/bin/bash: line 112: ./vendor/bin/phpstan: No such file or directory
[26]Cleaning up file based variables
[28]ERROR: Job failed: exit code 1

It seems that the prepare anchors is not getting executed.

@gdm this is most likely, because phpstan does not have execute permissions, try to add chmod +x ./vendor/bin/phpstan before the ./vendor/bin/phpstan analyse -c ci/phpstan.neon --memory-limit=-1

EDIT: actually there is no output from composer install, but I cannot see why it is not executed. maybe you are hitting a reserved word, try to change prepare to something else.

1 Like

Thank you. It works now. However, the last stage fails:

/bin/bash: line 119: [[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config: No such file or directory

[51]Cleaning up file based variables
[53](ERROR: Job failed: exit code 1

Remove the single quotes ' from the begging and end of that line

:woman_facepalming: I’am blind.
Thank you.