Docker Runner errors in the first stage

Hello everybody,

I got all messages when the process is started.

Running with gitlab-runner 14.9.1 (bd40e3da)
  on Docker 2vMynvtP
Preparing the "docker" executor
Using Docker executor with image docker:19.03-dind ...
Starting service docker:19.03.12-dind ...
Pulling docker image docker:19.03.12-dind ...
Using docker image sha256:66dc2d45749a48592f4348fb3d567bdd65c9dbd5402a413b6d169619e32f6bd2 for docker:19.03.12-dind with digest docker@sha256:674f1f40ff7c8ac14f5d8b6b28d8fb1f182647ff75304d018003f1e21a0d8771 ...
Waiting for services to be up and running...
*** WARNING: Service runner-2vmynvtp-project-2-concurrent-0-9efc183ac995b1a6-docker-0 probably didn't start properly.
Health check error:
service "runner-2vmynvtp-project-2-concurrent-0-9efc183ac995b1a6-docker-0-wait-for-service" timeout
Health check container logs:
Service container logs:
2022-04-22T07:10:18.362432904Z time="2022-04-22T07:10:18.362206281Z" level=info msg="Starting up"
2022-04-22T07:10:18.364666128Z time="2022-04-22T07:10:18.363434607Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
2022-04-22T07:10:18.364680615Z time="2022-04-22T07:10:18.363997819Z" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting --tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]"
2022-04-22T07:10:18.366760711Z time="2022-04-22T07:10:18.365204403Z" level=info msg="libcontainerd: started new containerd process" pid=23
2022-04-22T07:10:18.366774537Z time="2022-04-22T07:10:18.365249999Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2022-04-22T07:10:18.366780448Z time="2022-04-22T07:10:18.365261650Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2022-04-22T07:10:18.366784666Z time="2022-04-22T07:10:18.365285134Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
2022-04-22T07:10:18.366789695Z time="2022-04-22T07:10:18.365298740Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2022-04-22T07:10:18.397011998Z time="2022-04-22T07:10:18.394439271Z" level=info msg="starting containerd" revision=7ad184331fa3e55e52b890ea95e65ba581ae3429 version=v1.2.13 
2022-04-22T07:10:18.397043547Z time="2022-04-22T07:10:18.394838256Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1 
2022-04-22T07:10:18.397049748Z time="2022-04-22T07:10:18.394926531Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1 
2022-04-22T07:10:18.397055148Z time="2022-04-22T07:10:18.395160208Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" 
2022-04-22T07:10:18.397061179Z time="2022-04-22T07:10:18.395173803Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1 
2022-04-22T07:10:18.401495274Z time="2022-04-22T07:10:18.401405787Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1" 
2022-04-22T07:10:18.401510342Z time="2022-04-22T07:10:18.401427088Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1 
2022-04-22T07:10:18.401552461Z time="2022-04-22T07:10:18.401518689Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1 
2022-04-22T07:10:18.401767664Z time="2022-04-22T07:10:18.401705017Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1 
2022-04-22T07:10:18.402103181Z time="2022-04-22T07:10:18.401894552Z" level=info msg="skip loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1 
2022-04-22T07:10:18.402115063Z time="2022-04-22T07:10:18.401905081Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1 
2022-04-22T07:10:18.402129751Z time="2022-04-22T07:10:18.401936729Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" 
2022-04-22T07:10:18.402146842Z time="2022-04-22T07:10:18.401942891Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" 
2022-04-22T07:10:18.402151611Z time="2022-04-22T07:10:18.401948311Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1" 
2022-04-22T07:10:18.404073352Z time="2022-04-22T07:10:18.404022658Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1 
2022-04-22T07:10:18.404086266Z time="2022-04-22T07:10:18.404043006Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1 
2022-04-22T07:10:18.404127533Z time="2022-04-22T07:10:18.404079253Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1 
2022-04-22T07:10:18.404135218Z time="2022-04-22T07:10:18.404089281Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1 
2022-04-22T07:10:18.404139867Z time="2022-04-22T07:10:18.404097267Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1 
2022-04-22T07:10:18.404143994Z time="2022-04-22T07:10:18.404106193Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1 
2022-04-22T07:10:18.404174401Z time="2022-04-22T07:10:18.404115581Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1 
2022-04-22T07:10:18.404180412Z time="2022-04-22T07:10:18.404125389Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1 
2022-04-22T07:10:18.404184631Z time="2022-04-22T07:10:18.404139335Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1 
2022-04-22T07:10:18.404188547Z time="2022-04-22T07:10:18.404148432Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1 
2022-04-22T07:10:18.404322608Z time="2022-04-22T07:10:18.404276011Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2 
2022-04-22T07:10:18.404410953Z time="2022-04-22T07:10:18.404367982Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1 
2022-04-22T07:10:18.405183606Z time="2022-04-22T07:10:18.404659849Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1 
2022-04-22T07:10:18.405195159Z time="2022-04-22T07:10:18.404684594Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1 
2022-04-22T07:10:18.405560190Z time="2022-04-22T07:10:18.404714890Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405570600Z time="2022-04-22T07:10:18.404725100Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405574808Z time="2022-04-22T07:10:18.404734107Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405578475Z time="2022-04-22T07:10:18.404741591Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405582012Z time="2022-04-22T07:10:18.404749736Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405585358Z time="2022-04-22T07:10:18.404758362Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405588693Z time="2022-04-22T07:10:18.404766528Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405592541Z time="2022-04-22T07:10:18.404774763Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405596728Z time="2022-04-22T07:10:18.404781916Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1 
2022-04-22T07:10:18.405600756Z time="2022-04-22T07:10:18.404941675Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405605033Z time="2022-04-22T07:10:18.404954529Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405608941Z time="2022-04-22T07:10:18.404963155Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405612999Z time="2022-04-22T07:10:18.404973975Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1 
2022-04-22T07:10:18.405616866Z time="2022-04-22T07:10:18.405201710Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock" 
2022-04-22T07:10:18.405620473Z time="2022-04-22T07:10:18.405248648Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock" 
2022-04-22T07:10:18.405624861Z time="2022-04-22T07:10:18.405256854Z" level=info msg="containerd successfully booted in 0.011461s" 
2022-04-22T07:10:18.412195699Z time="2022-04-22T07:10:18.411861054Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2022-04-22T07:10:18.412210777Z time="2022-04-22T07:10:18.411888064Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2022-04-22T07:10:18.412216328Z time="2022-04-22T07:10:18.411912009Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
2022-04-22T07:10:18.412229522Z time="2022-04-22T07:10:18.411924302Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2022-04-22T07:10:18.412687528Z time="2022-04-22T07:10:18.412601668Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2022-04-22T07:10:18.412700252Z time="2022-04-22T07:10:18.412620634Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2022-04-22T07:10:18.412706234Z time="2022-04-22T07:10:18.412636452Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] <nil>}" module=grpc
2022-04-22T07:10:18.412711804Z time="2022-04-22T07:10:18.412647272Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2022-04-22T07:10:18.438728106Z time="2022-04-22T07:10:18.438599245Z" level=warning msg="Your kernel does not support cgroup blkio weight"
2022-04-22T07:10:18.438748535Z time="2022-04-22T07:10:18.438639711Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
2022-04-22T07:10:18.439221118Z time="2022-04-22T07:10:18.438858880Z" level=info msg="Loading containers: start."
2022-04-22T07:10:18.452040912Z time="2022-04-22T07:10:18.451929304Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge                200704  1 br_netfilter\nstp                    16384  1 bridge\nllc                    16384  2 bridge,stp\nip: can't find device 'br_netfilter'\nbr_netfilter           24576  0 \nbridge                200704  1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
2022-04-22T07:10:18.453613100Z time="2022-04-22T07:10:18.453536928Z" level=warning msg="Running iptables --wait -t nat -L -n failed with message: `modprobe: can't change directory to '/lib/modules': No such file or directory\niptables v1.8.4 (legacy): can't initialize iptables table `nat': Table does not exist (do you need to insmod?)\nPerhaps iptables or your kernel needs to be upgraded.`, error: exit status 3"
2022-04-22T07:10:18.483629628Z time="2022-04-22T07:10:18.483525143Z" level=info msg="stopping event stream following graceful shutdown" error="<nil>" module=libcontainerd namespace=moby
2022-04-22T07:10:18.483896617Z time="2022-04-22T07:10:18.483797822Z" level=info msg="stopping healthcheck following graceful shutdown" module=libcontainerd
2022-04-22T07:10:18.483907517Z time="2022-04-22T07:10:18.483814193Z" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby
2022-04-22T07:10:19.486070420Z failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables -t nat -N DOCKER: modprobe: can't change directory to '/lib/modules': No such file or directory
2022-04-22T07:10:19.486100617Z iptables v1.8.4 (legacy): can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
2022-04-22T07:10:19.486106307Z Perhaps iptables or your kernel needs to be upgraded.
2022-04-22T07:10:19.486110585Z  (exit status 3)
*********

This is my .gitlab-ci.yml file:

image: docker:19.03.12

variables:
  DOCKER_HOST: tcp://docker:2375
  DOCKER_TLS_CERTDIR: ""

build image:
  image: docker:19.03-dind
  services:
    - docker:19.03.12-dind
  script:
    - echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER $CI_REGISTRY --password-stdin
    - docker build -t $CI_REGISTRY_IMAGE .
    - docker push $CI_REGISTRY_IMAGE

Solved.

You must set “/certs/client” in volumes configuration in /etc/gitlab-runner/config.toml like this:

concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "Docker"
  url = "https://XX.XXX.com"
  token = "XX"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "alpine"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/certs/client","/cache"]
    shm_size = 0

In .gitlab-ci.yml change:

variables:
  DOCKER_TLS_CERTDIR: "/certs"

build image:
  image: docker:dind
  services:
    - docker:dind
  script:
    - echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER $CI_REGISTRY --password-stdin
    - docker build -t $CI_REGISTRY_IMAGE .
    - docker push $CI_REGISTRY_IMAGE

I found this blog post here