Self-hosted gitlab-runner unable to reach docker daemon

Hi,
I’m trying to setup a gitlab-runner on my local machine to run a pipeline that uses docker-compose.
I followed the official instructions for ubuntu and installed gitlab runner with apt-get. I registered the runner as per these instructions. My config.toml looks like this:

concurrent = 1
check_interval = 0
shutdown_timeout = 0

[session_server]
session_timeout = 1800

[[runners]]
name = “fricktop”
url = “https://gitlab.com/
id = 20902875
token = “redacted”
executor = “docker”
[runners.custom_build_dir]
[runners.cache]
MaxUploadedArchiveSize = 0
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = “docker:stable”
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = [“/cache”]
shm_size = 0

and my pipeline stage looks like this:
pytest:
stage: test
image: docker/compose:1.29.2
tags:
- docker
services:
- docker:dind
before_script:
- docker-compose -f local.yml build
- docker-compose -f local.yml run --rm django python manage.py migrate
- docker-compose -f local.yml up -d
script:
- docker-compose -f local.yml run django pytest

Whenever I run the pipeline locally I get a warning that a health check is failing in the beginning:

*** WARNING: Service runner–project-0-concurrent-0-c0232bd429f46d0e-docker-0 probably didn’t start properly.

Health check error:
service “runner–project-0-concurrent-0-c0232bd429f46d0e-docker-0-wait-for-service” timeout

Health check container logs:
Service container logs:
2023-02-04T09:37:22.025657624Z Certificate request self-signature ok
2023-02-04T09:37:22.025687582Z subject=CN = docker:dind server
2023-02-04T09:37:22.038720652Z /certs/server/cert.pem: OK
2023-02-04T09:37:23.821173278Z Certificate request self-signature ok
2023-02-04T09:37:23.821188512Z subject=CN = docker:dind client
2023-02-04T09:37:23.833869082Z /certs/client/cert.pem: OK
2023-02-04T09:37:23.862344861Z ip: can’t find device ‘ip_tables’
2023-02-04T09:37:23.863255910Z ip_tables 32768 2 iptable_filter,iptable_nat
2023-02-04T09:37:23.863345974Z x_tables 53248 6 xt_conntrack,xt_MASQUERADE,xt_addrtype,iptable_filter,iptable_nat,ip_tables
2023-02-04T09:37:23.863665815Z modprobe: can’t change directory to ‘/lib/modules’: No such file or directory
2023-02-04T09:37:23.865024959Z mount: permission denied (are you root?)
2023-02-04T09:37:23.865071875Z Could not mount /sys/kernel/security.
2023-02-04T09:37:23.865080200Z AppArmor detection and --privileged mode might break.
2023-02-04T09:37:23.865872091Z mount: permission denied (are you root?)


Eventually the pipeline fails with this error:

docker.errors.DockerException: Error while fetching server API version: (‘Connection aborted.’, FileNotFoundError(2, ‘No such file or directory’))
ERROR: Job failed: exit code 255

I tried running docker info before the docker compose command and it appears that there is no connection to the docker daemon:

Server:
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Docker daemon is definitely running. I have also added gitlab-runner to the docker group. The socket exists and is accessible to the docker group.

Any hints/help would be greatly appreciated. :slight_smile:

1 Like

I increased the wait_for_services_timeout in the runner config to 120 and this seams to resolve the problem