Gitlab shared runner can not connect to Docker daemon and leaks secrets


I have two problems:
1) Pipeline running on shared runner throws error

"failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial tcp connect: connection refused"

I tried to restart pipeline several times, same error happens, sometimes error message differs little bit. That’s other error message I get

Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?

And here is step it is supposed to run

  stage: dockerize
  image: docker:latest
    - docker:dind
    - docker build
      --build-arg DB_HOST=$PROD_DB_HOST
      --build-arg DB_USER=$PROD_DB_USER
    - develop
    - tags

It worked fine some days ago, but not it keeps throwing this error at build step

And other problem / question:

Sometimes when pipeline fails with this error it leaks all secrets into job history and I can’t find way how to clear job history. Someone suggested to delete whole project and make it again but that does not seem to be too good option (might cause all sorts of other issues)

Here is error message it throws sometimes

error during connect: Post http://docker:2375/v1.40/build?buildargs=%7B%DB_HOST%22%3A%22 ... all secrets leaked in here ... : context canceled

ERROR: Job failed: exit code 1

Both errors have happened on and also with managers 3, 5, and 6 shows all green. Maybe there is something I should change in my CI settings to get it working again?

1 Like

It seems docker updated their latest stable images and gitlab has not updated their runners yet, changing the images to be on the 18 major fixes these issues, eg:

image: docker:18-git

  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2

  - docker:18-dind

It worked like charm!
Thank you!

Also for anyone who want’s to delete pipelines / jobs to get rid of exposed secrets, seems like only way to delete them currently is through API:

Link to related issue:

1 Like

Thanks @ctorrisi the hotfix saves my day.:sweat_smile:

# .gitlab-ci.yml

@cheewai.lai Great to hear. The GitLab team are working on a more permanent fix and should resolve that issue once it is done.

Issue is being formally tracked on:

You delete the job log using the delete icon at the top of the log. Looks like a tiny trashcan.

You can delete an individual job’s log using the delete icon at the top of the log. There’s no need to go to the API unless you need to do bulk deletion.

1 Like