Gitlab CI get timeout from debian.org

Hi,

Since yesterday I’m getting a lot of timeout from pulling packages from debian.org and causing the pipeline fail.

Here is some examples:

E: Failed to fetch http://security.debian.org/debian-security/pool/updates/main/o/openssl/libssl-dev_1.1.1k-1%2bdeb11u1_amd64.deb  Connection timed out [IP: 151.101.2.132 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Step 3/44 : RUN apt-get update -yqq     && pecl channel-update pecl.php.net
 ---> Running in 42e47c73a36b
W: Failed to fetch http://deb.debian.org/debian/dists/bullseye/InRelease  Connection timed out [IP: 151.101.250.132 80]
W: Failed to fetch http://security.debian.org/debian-security/dists/bullseye-security/InRelease  Connection timed out [IP: 151.101.66.132 80]
W: Failed to fetch http://deb.debian.org/debian/dists/bullseye-updates/InRelease  Connection timed out [IP: 151.101.250.132 80]
W: Some index files failed to download. They have been ignored, or old ones used instead.

Sometimes it can connect to debian.org but sometimes not, seems a stability issue.

I’m usign a php7.4-fpm docker image.

Anyone having the same problem?

2 Likes

It’s seems that was a stability issue, the pipeline ran successfully.

1 Like

This problem happens actually to me, also:

+ apt-get update
Get:1 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Get:2 http://deb.debian.org/debian bullseye InRelease [113 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8178 kB]
Get:5 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [2088 B]
Err:1 http://security.debian.org/debian-security bullseye-security InRelease
  Connection timed out [IP: 151.101.202.132 80]
Fetched 8332 kB in 30s (277 kB/s)
Reading package lists...
W: Failed to fetch http://security.debian.org/debian-security/dists/bullseye-security/InRelease  Connection timed out [IP: 151.101.202.132 80]
W: Some index files failed to download. They have been ignored, or old ones used instead.
+ apt-get install -y --no-install-recommends build-essential git gcc libev-dev
Reading package lists...
Building dependency tree...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
 perl : Depends: perl-base (= 5.32.1-4) but 5.32.1-4+deb11u1 is to be installed
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c set -ex     && BUILD_DEPS="     build-essential     git     gcc     libev-dev     "     && apt-get update && apt-get install -y --no-install-recommends $BUILD_DEPS     && pip install --no-cache-dir -r requirements-dev.txt     && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false $BUILD_DEPS     && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100

Hello,

I am also having this problem on all my pipelines for about two days.

This is a huge problem because I can no longer deploy anything using my pipelines.

Sometimes the problem is temporarily resolved for 1 of the steps of my pipeline but it’s very random. It seems to be a network issue on the shared runner.

If anybody has an idea of what to do, it would help a lot.

EDIT: As a workaround, using host network host while building the docker images fixes the timeout problems. So instead of:

docker build ...

Do:

docker build --network host

Hope it helps.

2 Likes

Can confirm that adding --network host fixes the issue! Saved my day, Thank you

Edit: Also using the php7.4-fpm docker image

1 Like

That’s the way it works also for me, probably not intented, but as a workaround.
Thanks for the hint.

1 Like

docker build --network host works for me as well! Thank you guys.

There’s an open incident under investigation: 2021-09-26: Intermittent networking issues with some shared runner jobs (#5590) · Issues · GitLab.com / GitLab Infrastructure Team / production · GitLab

2 Likes

Same issue here.

6 [3/18] RUN apt-get update;     apt-get --no-install-recommends install -...
#6 0.287 Get:1 http://deb.debian.org/debian bullseye InRelease [113 kB]
#6 0.467 Get:2 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
#6 30.33 Err:1 http://deb.debian.org/debian bullseye InRelease
#6 30.33   Connection timed out [IP: 199.232.166.132 80]
#6 30.34 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
#6 30.49 Err:2 http://security.debian.org/debian-security bullseye-security InRelease
#6 30.49   Connection timed out [IP: 151.101.66.132 80]
#6 60.37 Err:3 http://deb.debian.org/debian bullseye-updates InRelease
#6 60.37   Connection timed out [IP: 199.232.166.132 80]
#6 60.37 Reading package lists...
#6 60.38 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye/InRelease  Connection timed out [IP: 199.232.166.132 80]
#6 60.38 W: Failed to fetch http://security.debian.org/debian-security/dists/bullseye-security/InRelease  Connection timed out [IP: 151.101.66.132 80]
#6 60.39 W: Failed to fetch http://deb.debian.org/debian/dists/bullseye-updates/InRelease  Connection timed out [IP: 199.232.166.132 80]
#6 60.39 W: Some index files failed to download. They have been ignored, or old ones used instead.
#6 60.39 Reading package lists...
#6 60.40 Building dependency tree...
#6 60.41 Reading state information...
#6 60.41 Package git is not available, but is referred to by another package.
#6 60.41 This may mean that the package is missing, has been obsoleted, or
#6 60.41 is only available from another source
#6 60.41 
#6 60.41 E: Unable to locate package net-tools
#6 60.41 E: Unable to locate package iputils-ping
#6 60.41 E: Unable to locate package traceroute
#6 60.41 E: Unable to locate package libzip-dev
#6 60.41 E: Unable to locate package zip
#6 60.41 E: Unable to locate package unzip
#6 60.41 E: Package 'git' has no installation candidate
#6 DONE 60.5s