So I’m having a problem with using dind to build images with gitlab.
My runner is on kubernetes k8s cluster. The helm chart values.yml file content is this:
gitlabUrl: https://REDACTED
certsSecretName: gitlab-runner-ca
concurrent: 10
check_interval: 1
log_level: "debug"
rbac:
create: true
clusterWideAccess: true
serviceAccountName: gitlab-runner
rules:
- resources: ["configmaps", "pods", "pods/attach", "secrets", "services"]
verbs: ["get", "list", "watch", "create", "patch", "delete"]
- resources: ["secrets"]
verbs: ["get", "list", "watch", "create", "patch", "update", "delete"]
- resources: ["serviceAccounts"]
verbs: ["get"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create", "patch", "delete"]
runners:
secret: gitlab-runner-secret
config: |
[[runners]]
name = "Kubernetes GitLab Runner"
executor = "kubernetes"
shell = "bash"
[runners.kubernetes]
namespace = "gitlab-runner"
terminationGracePeriodSeconds = 5
privileged = true
allow_privilege_escalation = true
image = "alpine"
helper_image = "gitlab/gitlab-runner-helper:ubuntu-x86_64-latest"
ca_file = "gitlab-runner-ca"
[[runners.kubernetes.volumes.secret]]
name = "gitlab-runner-ca"
mount_path = "/etc/gitlab-runner/certs/"
[[runners.kubernetes.volumes.host_path]]
name = "docker"
mount_path = "/var/run/docker.sock"
read_only = true
host_path = "/var/run/docker.sock"
my pipeline ci cd code is this:
variables:
DOCKER_HOST: tcp://docker:2375 # also tried localhost here
DOCKER_DRIVER: overlay2
docker-build:
image: docker:23.0.6
stage: build
services:
- docker:23.0.6-dind
before_script:
- docker --version
- until docker info; do sleep 1; done
script:
- echo "Hello"
and the infamous error I get is this:
Client:
4650 Context: default
4651 Debug Mode: false
4652 Plugins:
4653 buildx: Docker Buildx (Docker Inc.)
4654 Version: v0.11.1
4655 Path: /usr/local/libexec/docker/cli-plugins/docker-buildx
4656 compose: Docker Compose (Docker Inc.)
4657 Version: v2.20.0
4658 Path: /usr/local/libexec/docker/cli-plugins/docker-compose
4659Server:
4660ERROR: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?
4661errors pretty printing info
Any help will be much appreciated
It looks like you’re telling the docker client to connect to the docker daemon at TCP host docker:2375 but you’re mounting the docker socket as a file at /var/run/docker.sock
In this case try not setting DOCKER_HOST since the default is for docker to look for the socket
Here is a working example for me (on kubernetes executor)
docker_build_job:
image: docker:23.0.2-alpine3.17
services:
- docker:23.0.2-dind-alpine3.17
stage: build
before_script:
- until docker info; do sleep 1; done;
script:
- docker build .
retry:
max: 2
# specify retry on certain conditions
# see https://docs.gitlab.com/ee/ci/yaml/index.html#retrywhen
when:
- runner_system_failure
- stuck_or_timeout_failure
I’ve tried that as well, I’ve tested even not mounting the socket and use the DOCKER_HOST only still the same error
I"ve tested those versions as well still the same error. Mind sharing your values.yaml? Could I be doing something wrong there 
So the answer was between the two answers. What I’ve done:
values.yml
gitlabUrl: https://REDACTED
certsSecretName: gitlab-runner-ca
concurrent: 10
check_interval: 1
log_level: "debug"
rbac:
create: true
clusterWideAccess: true
serviceAccountName: gitlab-runner
rules:
- resources: ["configmaps", "pods", "pods/attach", "secrets", "services"]
verbs: ["get", "list", "watch", "create", "patch", "delete"]
- resources: ["secrets"]
verbs: ["get", "list", "watch", "create", "patch", "update", "delete"]
- resources: ["serviceAccounts"]
verbs: ["get"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create", "patch", "delete"]
runners:
secret: gitlab-runner-secret
config: |
[[runners]]
name = "Kubernetes GitLab Runner"
executor = "kubernetes"
shell = "bash"
[runners.kubernetes]
namespace = "gitlab-runner"
terminationGracePeriodSeconds = 5
privileged = true
allow_privilege_escalation = true
image = "alpine"
ca_file = "gitlab-runner-ca"
[[runners.kubernetes.volumes.secret]]
name = "gitlab-runner-ca"
mount_path = "/etc/gitlab-runner/certs/"
pipeline:
variables:
DOCKER_TLS_CERTDIR: ""
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
docker-build:
image: docker:23.0.2-alpine3.17
services:
- docker:23.0.2-dind-alpine3.17
stage: build
before_script:
- docker --version
- until docker info; do sleep 1; done
script:
- echo "Hello"
Many thanks to @c.vandesande and @sjoukedv
2 Likes
Glad you got it working! FYI there’s a few solutions that work, I don’t think anyone is better than the other. One bug I encountered is that the dockerd service is often not ready by the time the jobs starts resulting in a failure, so I added a sleep timer.
This little pre_build_script looks to see if the “docker” command is in the build image, and if so, runs docker version up to 10 times. (Returns false if dockerd is not ready). Usually dockerd is up after 2-3 seconds, but randomly sometimes it takes more than 6 seconds
config: |
[[runners]]
pre_build_script = '''
# Docker is accessed via unix socket on k8s runners
unset DOCKER_HOST
unset DOCKER_CERT_PATH
unset DOCKER_TLS_VERIFY
# If docker CLI exists wait for dockerd to start
if command -v docker &> /dev/null; then
i=1; while [ $i -le 10 ]; do
echo "docker command found, waiting for dockerd service $i/10..."
docker version &> /dev/null && break
sleep 1
if [ $i -eq 10 ]; then
echo "WARNING docker cli detected but dockerd service not found, continuing build..."
fi
i=$(( i + 1 ))
done
fi
'''
2 Likes