Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? Error on Gitlab Cloud CI/CD with self-hosted runners

Hello,
I am using gitlab-runner version 14.4.0 and docker version 20.10.11 on Ubuntu 18.04.6 LTS The machine I am using for the runners is a powerful Supermicro server. Our Gitlab CI is on gitlab cloud (SAAS)
I have been receiving the following errors on Build stage jobs:

  1. ERROR: Job failed (system failure): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? (exec.go:66:120s)
  2. Error: Job failed (system failure): Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? (docker.go:708:120s)
  3. Preparation failed: adding cache volume: set volume permissions: create permission container for volume “runner-######-project-#####-concurrent-0-cache-##############”: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? (linux_set.go:90:120s)
  4. ERROR: Job failed (system failure): prepare environment: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? (docker.go:708:120s). Check Types of shells supported by GitLab Runner | GitLab for more information

The solutions I have tried so far:

  1. Added multi pull policy: pull_policy = [“always”, “if-not-present”] in config.toml for all runners
  2. Gave permission to gitlab-runner user for docker and sudo groups
  3. Tried chmod 666 /var/run/docker.sock
  4. systemctl docker enable & systemctl docker start
  5. restarted gitlab-runner and reloaded daemon
  6. Fresh installed the machine from scratch with Ubuntu 18.04.6 LTS, latest docker and gitlab-runner
    Nothing seemed to have solved the issue. Usually just restarting the jobs after the error gets the jobs running. But that is not a solution.
    I am new to this and any help is appreciated!
    Thank you
1 Like