Network issue with the Gitlab runners and *.githubusercontent.com

GitLab CI/CD
1

Problem to solve

My CI job uses dind to build a node-16 project, which includes bcrypt and sharp, both would download a precompiled binary, but fails downloading.
It’s a unchanged gitlab-ci.yaml file and no new dependencies were added since the last successful release.

Steps to reproduce

I have added some network debugging to my Dockerfile:
RUN set +e;
echo “=== proxy env ===”; env | grep -i proxy || echo “(no proxy vars set)”;
echo “=== DNS resolution (getent) ===”;
for h in ``registry.npmjs.org`` ``github.com`` ``objects.githubusercontent.com`` ``codeload.github.com``; do
printf – “-- %s –\n” “$h”; getent hosts “$h” || echo “DNS FAIL: $h”;
done;
FMT=‘dns=%{time_namelookup}s connect=%{time_connect}s tls=%{time_appconnect}s ttfb=%{time_starttransfer}s total=%{time_total}s code=%{http_code} redirects=%{num_redirects} final=%{url_effective}\n’;
echo “=== HTTPS reachability (connect 10s / total 30s) ===”;
for u in ``https://registry.npmjs.org/`` ``https://github.com/`` ``https://objects.githubusercontent.com/`` ``https://codeload.github.com/``; do
printf – “-- %s –\n” “$u”;
curl -sS -o /dev/null --connect-timeout 10 --max-time 30 -w “$FMT” “$u” || echo “CURL FAIL: $u”;
done;
echo “=== bcrypt prebuilt binary (follow redirects, total 60s) ===”;
curl -sSL -o /dev/null --connect-timeout 10 --max-time 60 -w “$FMT”
https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.0.1/bcrypt_lib-v5.0.1-napi-v3-linux-x64-glibc.tar.gz``
|| echo “BCRYPT BINARY DOWNLOAD FAILED/TIMED OUT (this is the install hang)”;
echo “=== end diagnostics ===”; true

And here is the output:

=== proxy env ===

(no proxy vars set)

=== DNS resolution (getent) ===

-- registry.npmjs.org --

2606:4700::6810:422 registry.npmjs.org

2606:4700::6810:a22 registry.npmjs.org

2606:4700::6810:722 registry.npmjs.org

2606:4700::6810:b22 registry.npmjs.org

2606:4700::6810:22 registry.npmjs.org

2606:4700::6810:622 registry.npmjs.org

2606:4700::6810:122 registry.npmjs.org

2606:4700::6810:822 registry.npmjs.org

2606:4700::6810:322 registry.npmjs.org

2606:4700::6810:222 registry.npmjs.org

2606:4700::6810:522 registry.npmjs.org

2606:4700::6810:922 registry.npmjs.org

-- github.com --

140.82.112.4    github.com

-- objects.githubusercontent.com --

185.199.108.133 objects.githubusercontent.com

185.199.109.133 objects.githubusercontent.com

185.199.110.133 objects.githubusercontent.com

185.199.111.133 objects.githubusercontent.com

-- codeload.github.com --

140.82.112.9    codeload.github.com

=== HTTPS reachability (connect 10s / total 30s) ===

-- https://registry.npmjs.org/ --

dns=0.002252s connect=0.013408s tls=3.312879s ttfb=3.328951s total=3.329023s code=200 redirects=0 final=https://registry.npmjs.org/

-- https://github.com/ --

dns=0.002485s connect=0.015592s tls=0.050412s ttfb=0.064478s total=0.128467s code=200 redirects=0 final=https://github.com/

-- https://objects.githubusercontent.com/ --

curl: (28) Operation timed out after 10001 milliseconds with 0 out of 0 bytes received

dns=0.023388s connect=0.036323s tls=0.000000s ttfb=0.000000s total=10.001776s code=000 redirects=0 final=https://objects.githubusercontent.com/

CURL FAIL: https://objects.githubusercontent.com/

-- https://codeload.github.com/ --

dns=0.002285s connect=0.014799s tls=0.054203s ttfb=0.069381s total=0.069416s code=301 redirects=0 final=https://codeload.github.com/

=== bcrypt prebuilt binary (follow redirects, total 60s) ===

curl: (28) Operation timed out after 10002 milliseconds with 0 out of 0 bytes received

dns=0.005183s connect=0.029512s tls=0.047174s ttfb=0.060383s total=10.062516s code=302 redirects=1 final=https://release-assets.githubusercontent.com/github-production-release-asset/611333/cfe7f880-802a-11eb-9cd7-9c683269205d?sp=r&sv=2018-11-09&sr=b&spr=https&se=2026-06-02T11%3A28%3A33Z&rscd=attachment%3B+filename%3Dbcrypt_lib-v5.0.1-napi-v3-linux-x64-glibc.tar.gz&rsct=application%2Foctet-stream&skoid=96c2d410-5711-43a1-aedd-ab1947aa7ab0&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skt=2026-06-02T10%3A27%3A43Z&ske=2026-06-02T11%3A28%3A33Z&sks=b&skv=2018-11-09&sig=GHG040ysMtULq%2Bj8irXrTh1ffoeiEvo%2Fu0l9ZrcmRxs%3D&jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmVsZWFzZS1hc3NldHMuZ2l0aHVidXNlcmNvbnRlbnQuY29tIiwia2V5Ijoia2V5MSIsImV4cCI6MTc4MDM5NzE3NSwibmJmIjoxNzgwMzk2ODc1LCJwYXRoIjoicmVsZWFzZWFzc2V0cHJvZHVjdGlvbi5ibG9iLmNvcmUud2luZG93cy5uZXQifQ.Z3FSas8mZMWteASePtQoxaxpL53Q11ve7hwWEjOYAgQ&response-content-disposition=attachment%3B%20filename%3Dbcrypt_lib-v5.0.1-napi-v3-linux-x64-glibc.tar.gz&response-content-type=application%2Foctet-stream

BCRYPT BINARY DOWNLOAD FAILED/TIMED OUT (this is the install hang)

Versions

Please select whether options apply, and add the version information.

  • Self-managed
  • GitLab.com SaaS
  • Dedicated
  • Self-hosted Runners
2

Setting the MTU on the dind service as suggested here works:

services:
- name: docker:20.10.11-dind
command: ["–mtu=1360"]