I have been running all my pipelines using Project specific Runners, docker executors, all working fine without any issues.
As the runners take some time to setup the environment and installing dependencies, I’ve created our custom image with that setup already done.
This is in a separate project branch now using registry.gitlab.com// image instead of php:8.1
The runner is able to connect, pull the image but randomly, this pipeline fails with the common
ERROR: Job failed (system failure): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? (docker.go:659:120s)
Many people complain about this error, but all the errors are kind of related with people trying to use dind and that’s not my case.
- The runners never fails with this error when using php image
- The custom image has 1.5GB, about 3x the size of base php image
- Gitlab version: Gitlab.com
- Gitlab-Runner version: 15.3.0 (project specific runner)
- Knowing it was not a dind related problem, I’ve tried exposing the docker.sock as a volume or setting docker to connect via tcp port, without success.
- Tried setting
wait_for_services_timeout=120without change on runner behaviour
On the machine running the gitlab-runner, I can see it creates the container and when the error occurrs it stay there created.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 730cb7c95e9c e92b9b9879e3 "docker-php-entrypoi…" 43 hours ago Created runner-removedid-project-removedid2-concurrent-0-3a0792ab23b639c3-build-2
The container is from our custom image (e92b9b9879e3).
Gitlab-Runner relevant configurations
[[runners]] name = "supressed" url = "https://gitlab.com/" token = "supressed-again" executor = "docker" [runners.docker] tls_verify = false image = "docker:stable" privileged = true disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false volumes = ["/certs/client", "/cache"] shm_size = 0 wait_for_services_timeout=120
At this point, I don’t know what else to try.
Please feel free to ask for more information or run some testing.