Can not login to gitlab registry with runner (via $CI_REGISTRY_PASSWORD), causes HTTP 401

Problem to solve

I try to build a CI component for building docker images. After the build, I want to push the image into the gitlab registry to make the image accessable by other jobs/stages. Therefore I try to login/authenticate the runner into the gitlab registry with

echo -n $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY

in the .gitlab-ci.yml. When the job is executed the login fails with

Get "https://<gitlab registry url>/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

I’ve checked the logs of my gitlab instance. So the command causes the following log entry

{"content_type":"application/json","correlation_id":"...","duration_ms":0,"host":"<gitlab registry url>","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"xxx.xxx.xxx.xxx:yyyy","remote_ip":"xxx.xxx.xxx.xxx","status":401,"system":"http","time":"2024-04-27T13:51:44.372+02:00","ttfb_ms":0,"uri":"/v2/","user_agent":"Docker-Client/25.0.5 (linux)","written_bytes":87}

But when I login/authenticate in the gitlab registry from the same maschine the runner is hosted via terminal, everything works just fine.

Why gives the gitlab instance/registry a HTTP 401 only when the runner tries to login? Has anybody experinced the same or similar problem and found a solution?
Thanks in advanced.

Configuration

I use a self hosted gitlab instance behind an external nginx. The registry has an indipendent url and is configured behind nginx too. The runner is hosted on a different VM. Every server is hosted by netcup (just in case it is important to know).

Runner config:

[[runners]]
  name = "Runner"
  url = "gitlab-instance-url"
  token = "xxxxxxxxx"
  token_obtained_at = xxxxxxx
  token_expires_at = xxxxxxx
  executor = "docker"
  clone_url = "gitlab-instance-url"
  [runners.cache]
    Type = "s3"
    Path = "path"
    Shared = true
    MaxUploadedArchiveSize = 0
    [runners.cache.s3]
      ServerAddress = "minio-instance-url"
      AccessKey = "xxxx"
      SecretKey = "xxxxx"
      BucketName = "gitlab-cache"
  [runners.docker]
    tls_verify = false
    image = "docker:stable"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/certs/client", "/cache"]
    shm_size = 0
    network_mtu = 0

Versions

Please select whether options apply, and add the version information.

  • Self-managed
  • GitLab.com SaaS
  • Self-hosted Runners

Versions

  • GitLab : 16.11.1
  • GitLab Runner: 16.9.0