Hello there,
I have an issue since few days related to GitlabCI and a docker build.
A job that was running fine for months suddenly start to fail because of networking issues. And I can’t figure what is going on.
It was setup based on the official guide in DinD ( https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-executor)
# Build docker image and push on gitlab registry
docker_build:
stage: release
tags:
- docker
image: docker:stable
services:
- docker:dind
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
script:
# build
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker build -t $DOCKER_IMG_TAG .
- docker push $DOCKER_IMG_TAG
And the image by itself is a quite simple and standard ruby website.
But since few days, things happening during the docker-build step seems to have lot of networking issue. I see random error from both apt and bundle:
Retrying download gem from https://rubygems.org/ due to error (2/4): Gem::RemoteFetcher::UnknownHostError timed out (https://rubygems.org/gems/rake-12.3.2.gem)
Retrying download gem from https://rubygems.org/ due to error (3/4): Gem::RemoteFetcher::UnknownHostError timed out (https://rubygems.org/gems/rake-12.3.2.gem)
The job that was usually taking 2-3 minutes, now take up to 30-40 min and randomly fails. Building the image locally works fine and doesnt show any warning or networking slow down, the issue occurs only in Gitlab-CI (shared-runner).
So there is some connectivity but really slow and degraded.
I originally suspected issue with rubygems or a recent update of docker. But even with older version of both, I still have the same issue on gitlab-ci. Job which were working fine one month ago now fails …
Did anything change recently on gitlab-ci runners which could cause networking issues?