Bug in runner caused get out of CI minutes

Hello,

I’m a happy community GitLab user. I’ve a project using free runners and few days ago a build was stuck because a malfunction in the runner. The fact is that, because this bug, I’m out of CI minutes at the beginning of month…

are there a way to solve this issue so I can continue using the runners as usual guys? I really appreciate all the free tier you give but I understand this is caused because a malfunction.

Here the data:

Duration: 19359 minutes 13 seconds
Timeout: 1h (from project)
Runner: shared-runners-manager-6.gitlab.com (#380987)

And the output of the build:

Running with gitlab-runner 12.1.0 (de7731dd)
  on docker-auto-scale ed2dce3a
Using Docker executor with image docker:latest ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:f20e73ae0611ce447dbbd26d8b62eb87a966a7e5b54c32d071d601e411d78d9d for docker:dind ...
Waiting for services to be up and running...

*** WARNING: Service runner-ed2dce3a-project-1660093-concurrent-0-docker-0 probably didn't start properly.

Health check error:
service "runner-ed2dce3a-project-1660093-concurrent-0-docker-0-wait-for-service" timeout

Health check container logs:


Service container logs:
2019-08-23T00:21:12.735033310Z time="2019-08-23T00:21:12.734812602Z" level=info msg="Starting up"
2019-08-23T00:21:12.736668276Z time="2019-08-23T00:21:12.736562725Z" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
2019-08-23T00:21:12.737041301Z time="2019-08-23T00:21:12.736975430Z" level=warning msg="[!] DON'T BIND ON ANY IP ADDRESS WITHOUT setting --tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING [!]"
2019-08-23T00:21:12.739779664Z time="2019-08-23T00:21:12.739698091Z" level=info msg="libcontainerd: started new containerd process" pid=17
2019-08-23T00:21:12.739915750Z time="2019-08-23T00:21:12.739859005Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2019-08-23T00:21:12.740024414Z time="2019-08-23T00:21:12.739943362Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2019-08-23T00:21:12.740139002Z time="2019-08-23T00:21:12.740063683Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] }" module=grpc
2019-08-23T00:21:12.740262814Z time="2019-08-23T00:21:12.740186768Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2019-08-23T00:21:12.740473913Z time="2019-08-23T00:21:12.740378914Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000126630, CONNECTING" module=grpc
2019-08-23T00:21:12.760673852Z time="2019-08-23T00:21:12.760518933Z" level=info msg="starting containerd" revision=894b81a4b802e4eb2a91d1ce216b8817763c29fb version=v1.2.6 
2019-08-23T00:21:12.761254522Z time="2019-08-23T00:21:12.761179565Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1 
2019-08-23T00:21:12.770639185Z time="2019-08-23T00:21:12.769803809Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1 
2019-08-23T00:21:12.770660576Z time="2019-08-23T00:21:12.770148809Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" 
2019-08-23T00:21:12.770667328Z time="2019-08-23T00:21:12.770166911Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1 
2019-08-23T00:21:12.782642762Z time="2019-08-23T00:21:12.782405067Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1" 
2019-08-23T00:21:12.783417983Z time="2019-08-23T00:21:12.782442157Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1 
2019-08-23T00:21:12.784369659Z time="2019-08-23T00:21:12.783628033Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1 
2019-08-23T00:21:12.784381488Z time="2019-08-23T00:21:12.783821609Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1 
2019-08-23T00:21:12.784385559Z time="2019-08-23T00:21:12.784114971Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter" 
2019-08-23T00:21:12.784399668Z time="2019-08-23T00:21:12.784127730Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1 
2019-08-23T00:21:12.784404509Z time="2019-08-23T00:21:12.784180833Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" 
2019-08-23T00:21:12.784410139Z time="2019-08-23T00:21:12.784191912Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "ip: can't find device 'aufs'\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n": exit status 1" 
2019-08-23T00:21:12.784413911Z time="2019-08-23T00:21:12.784199362Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter" 
2019-08-23T00:21:12.795714318Z time="2019-08-23T00:21:12.794244228Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1 
2019-08-23T00:21:12.795743406Z time="2019-08-23T00:21:12.794293849Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1 
2019-08-23T00:21:12.795748383Z time="2019-08-23T00:21:12.794331872Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1 
2019-08-23T00:21:12.795751896Z time="2019-08-23T00:21:12.794345672Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1 
2019-08-23T00:21:12.795755289Z time="2019-08-23T00:21:12.794356600Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1 
2019-08-23T00:21:12.795758599Z time="2019-08-23T00:21:12.794369979Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1 
2019-08-23T00:21:12.795761835Z time="2019-08-23T00:21:12.794383758Z" level=info msg="loading plugin "io.containerd.service.v1.leases-service"..." type=io.containerd.service.v1 
2019-08-23T00:21:12.795764946Z time="2019-08-23T00:21:12.794395099Z" level=info msg="loading plugin "io.containerd.service.v1.namespaces-service"..." type=io.containerd.service.v1 
2019-08-23T00:21:12.795768042Z time="2019-08-23T00:21:12.794405490Z" level=info msg="loading plugin "io.containerd.service.v1.snapshots-service"..." type=io.containerd.service.v1 
2019-08-23T00:21:12.795771050Z time="2019-08-23T00:21:12.794417089Z" level=info msg="loading plugin "io.containerd.runtime.v1.linux"..." type=io.containerd.runtime.v1 
2019-08-23T00:21:12.795774138Z time="2019-08-23T00:21:12.794673717Z" level=info msg="loading plugin "io.containerd.runtime.v2.task"..." type=io.containerd.runtime.v2 
2019-08-23T00:21:12.795777296Z time="2019-08-23T00:21:12.794801204Z" level=info msg="loading plugin "io.containerd.monitor.v1.cgroups"..." type=io.containerd.monitor.v1 
2019-08-23T00:21:12.795788846Z time="2019-08-23T00:21:12.795223140Z" level=info msg="loading plugin "io.containerd.service.v1.tasks-service"..." type=io.containerd.service.v1 
2019-08-23T00:21:12.795792355Z time="2019-08-23T00:21:12.795259467Z" level=info msg="loading plugin "io.containerd.internal.v1.restart"..." type=io.containerd.internal.v1 
2019-08-23T00:21:12.795796278Z time="2019-08-23T00:21:12.795309656Z" level=info msg="loading plugin "io.containerd.grpc.v1.containers"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.795799446Z time="2019-08-23T00:21:12.795323453Z" level=info msg="loading plugin "io.containerd.grpc.v1.content"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.795802439Z time="2019-08-23T00:21:12.795335545Z" level=info msg="loading plugin "io.containerd.grpc.v1.diff"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.795805436Z time="2019-08-23T00:21:12.795346878Z" level=info msg="loading plugin "io.containerd.grpc.v1.events"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.795808527Z time="2019-08-23T00:21:12.795357223Z" level=info msg="loading plugin "io.containerd.grpc.v1.healthcheck"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.795811511Z time="2019-08-23T00:21:12.795367312Z" level=info msg="loading plugin "io.containerd.grpc.v1.images"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.795814480Z time="2019-08-23T00:21:12.795377730Z" level=info msg="loading plugin "io.containerd.grpc.v1.leases"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.795817558Z time="2019-08-23T00:21:12.795387282Z" level=info msg="loading plugin "io.containerd.grpc.v1.namespaces"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.795820562Z time="2019-08-23T00:21:12.795396953Z" level=info msg="loading plugin "io.containerd.internal.v1.opt"..." type=io.containerd.internal.v1 
2019-08-23T00:21:12.798695946Z time="2019-08-23T00:21:12.797822832Z" level=info msg="loading plugin "io.containerd.grpc.v1.snapshots"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.798710152Z time="2019-08-23T00:21:12.797851214Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.798713770Z time="2019-08-23T00:21:12.797864745Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.798717068Z time="2019-08-23T00:21:12.797875679Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1 
2019-08-23T00:21:12.798720218Z time="2019-08-23T00:21:12.798140969Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock" 
2019-08-23T00:21:12.798723265Z time="2019-08-23T00:21:12.798219291Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock" 
2019-08-23T00:21:12.798726260Z time="2019-08-23T00:21:12.798231285Z" level=info msg="containerd successfully booted in 0.039484s" 
2019-08-23T00:21:12.801622538Z time="2019-08-23T00:21:12.801514427Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc000126630, READY" module=grpc
2019-08-23T00:21:12.834693482Z time="2019-08-23T00:21:12.822142434Z" level=info msg="Setting the storage driver from the $DOCKER_DRIVER environment variable (overlay)"
2019-08-23T00:21:12.834732983Z time="2019-08-23T00:21:12.822433005Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2019-08-23T00:21:12.834737728Z time="2019-08-23T00:21:12.822447986Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2019-08-23T00:21:12.834741319Z time="2019-08-23T00:21:12.822475266Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] }" module=grpc
2019-08-23T00:21:12.834745057Z time="2019-08-23T00:21:12.822486497Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2019-08-23T00:21:12.834748356Z time="2019-08-23T00:21:12.822548883Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0006d8a00, CONNECTING" module=grpc
2019-08-23T00:21:12.834751717Z time="2019-08-23T00:21:12.823189206Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0006d8a00, READY" module=grpc
2019-08-23T00:21:12.834754716Z time="2019-08-23T00:21:12.828859197Z" level=info msg="parsed scheme: \"unix\"" module=grpc
2019-08-23T00:21:12.834757897Z time="2019-08-23T00:21:12.828879327Z" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
2019-08-23T00:21:12.834760953Z time="2019-08-23T00:21:12.828902527Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}] }" module=grpc
2019-08-23T00:21:12.834764211Z time="2019-08-23T00:21:12.828913346Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
2019-08-23T00:21:12.834767391Z time="2019-08-23T00:21:12.828953359Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0006d9010, CONNECTING" module=grpc
2019-08-23T00:21:12.836682538Z time="2019-08-23T00:21:12.835226299Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc0006d9010, READY" module=grpc
2019-08-23T00:21:12.836695370Z time="2019-08-23T00:21:12.836167404Z" level=warning msg="[graphdriver] WARNING: the overlay storage-driver is deprecated, and will be removed in a future release"
2019-08-23T00:21:12.941824035Z time="2019-08-23T00:21:12.921303438Z" level=info msg="Loading containers: start."
2019-08-23T00:21:12.941853725Z time="2019-08-23T00:21:12.940433888Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: ip: can't find device 'bridge'\nbridge                167936  1 br_netfilter\nstp                    16384  1 bridge\nllc                    16384  2 bridge,stp\nip: can't find device 'br_netfilter'\nbr_netfilter           24576  0 \nbridge                167936  1 br_netfilter\nmodprobe: can't change directory to '/lib/modules': No such file or directory\n, error: exit status 1"
2019-08-23T00:21:12.992990388Z time="2019-08-23T00:21:12.948976400Z" level=warning msg="Running modprobe nf_nat failed with message: `ip: can't find device 'nf_nat'\nnf_nat_ipv4            16384  2 ipt_MASQUERADE,iptable_nat\nnf_nat                 32768  1 nf_nat_ipv4\nnf_conntrack          135168  5 ipt_MASQUERADE,nf_conntrack_netlink,nf_nat_ipv4,nf_nat,xt_conntrack\nlibcrc32c              16384  2 nf_nat,nf_conntrack\nmodprobe: can't change directory to '/lib/modules': No such file or directory`, error: exit status 1"
2019-08-23T00:21:12.994178002Z time="2019-08-23T00:21:12.978241215Z" level=warning msg="Running modprobe xt_conntrack failed with message: `ip: can't find device 'xt_conntrack'\nxt_conntrack           16384  2 \nnf_conntrack          135168  5 ipt_MASQUERADE,nf_conntrack_netlink,nf_nat_ipv4,nf_nat,xt_conntrack\nmodprobe: can't change directory to '/lib/modules': No such file or directory`, error: exit status 1"
2019-08-23T00:21:13.101101071Z time="2019-08-23T00:21:13.100934829Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.18.0.0/16. Daemon option --bip can be used to set a preferred IP address"
2019-08-23T00:21:13.168702111Z time="2019-08-23T00:21:13.168507830Z" level=info msg="Loading containers: done."
2019-08-23T00:21:13.188656030Z time="2019-08-23T00:21:13.188060401Z" level=info msg="Docker daemon" commit=74b1e89e8a graphdriver(s)=overlay version=19.03.1
2019-08-23T00:21:13.188692355Z time="2019-08-23T00:21:13.188280708Z" level=info msg="Daemon has completed initialization"
2019-08-23T00:21:13.243349561Z time="2019-08-23T00:21:13.242952739Z" level=info msg="API listen on [::]:2375"
2019-08-23T00:21:13.243398003Z time="2019-08-23T00:21:13.243054562Z" level=info msg="API listen on /var/run/docker.sock"