GitLab-CI job hangs while downloading

GitLab CI/CD
1

Hello,

GitLab-CI job hangs while downloading Alpine packages index.

Sample:

Any hint?

Thanks!

2

There are problems right now: https://status.gitlab.com/

Intermittent networking issues with some shared runner jobsPartial Service Disruption
Incident Status
Partial Service Disruption
Components
CI/CD - GitLab SaaS Shared Runners
Locations
Google Compute Engine

1 Like
3

Had same issue on private gitlab installation (ce version). After investigating we found out that problem comes from docker bridge network.

Currently we fixed it by changing MTU setting of docker daemon. Its help us to override settings for all newly created interfaces at runner`s host.
My /etc/docker/daemon.json:

{
   "metrics-addr": "0.0.0.0:9323",
   "experimental": true,
   "max-concurrent-downloads": 50,
   "max-concurrent-uploads": 50,
   "mtu": 1440
}

MTU of system interface is 1460, but for bridge network its was created with MTU 1500.

More details can be founded at Gitlab Runner Failing to Download files (#28225) · Issues · GitLab.org / gitlab-runner · GitLab and Gitlab Runner Failing to Download files (#28225) · Issues · GitLab.org / gitlab-runner · GitLab

4

Currently works configuring GitLabCI variable FF_NETWORK_PER_BUILD:1 as described in gitlab-com/gl-infra/production#5590