Unable to pull image from gitlab registry for one project (all other repositories are fine)

How to Use GitLab
, ,
1

I have a weird problem with one of the repositories:

  • I’m running instance of gitlab-ce on premise
  • I have a repository that builds docker image with some build tools
  • this image is published to gitlab registry on the same host
  • all other projects use this image in GitLab CI for their business
  • the typical job with this image would look like this:
#
# publish stage
#
publish-pkg-dev:
  extends:
    - .debug
  stage: publish
  tags:
    - docker
  image: registry.example.com/foo/build-tools:latest
  variables:
    PKG_VERSION: 0.0.0-${CI_PIPELINE_ID}
    RELEASE: "0"
  script:
    - make pkg
   - ...
  • This woks fine for all projects but one and this is the error I’m getting:
ERROR: Job failed: failed to pull image "registry.example.com/foo/build-tools:latest" with specified policies [always]: Error response from daemon: pull access denied for registry.example.com/foo/build-tools, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (manager.go:237:0s)
  • On the gitlab side I see these errors in the log:
2023-02-16_21:04:39.68601 {"content_type":"application/json","correlation_id":"01GSE0P984C2V9MXQV6R0D6A32","duration_ms":1,"host":"registry.example.com","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:50328","remote_ip":"2620:18:6000:aa10:9476:92ff:fe74:822c","status":401,"system":"http","time":"2023-02-16T21:04:39.685Z","ttfb_ms":1,"uri":"/v2/","user_agent":"docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.4.0-137-generic os/linux arch/amd64 UpstreamClient(Go-http-client/1.1)","written_bytes":87}
2023-02-16_21:04:39.96323 {"content_type":"application/json","correlation_id":"01GSE0P9GSS5R5A4264KRYGEBD","duration_ms":2,"host":"registry.example.com","level":"info","method":"HEAD","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:50334","remote_ip":"2620:18:6000:aa10:9476:92ff:fe74:822c","status":401,"system":"http","time":"2023-02-16T21:04:39.963Z","ttfb_ms":1,"uri":"/v2/foo/build-tools/manifests/latest","user_agent":"docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.4.0-137-generic os/linux arch/amd64 UpstreamClient(Go-http-client/1.1)","written_bytes":158}
2023-02-16_21:04:40.08954 {"content_type":"application/json","correlation_id":"01GSE0P9MD2KYQ256QB37CCY3D","duration_ms":11,"host":"registry.example.com","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:50350","remote_ip":"2620:18:6000:aa10:9476:92ff:fe74:822c","status":401,"system":"http","time":"2023-02-16T21:04:40.088Z","ttfb_ms":11,"uri":"/v2/foo/build-tools/manifests/latest","user_agent":"docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.4.0-137-generic os/linux arch/amd64 UpstreamClient(Go-http-client/1.1)","written_bytes":158}
  • All project share same pull of runners
  • I also double- and triple- checked settings of:
    • the build-tool project
    • the project in question
    • compared with settings of other project that work fine and couldn’t find any discrepancy
  • All I can tell right now that this is an anomaly and possibly a bug

More details on the project:

  • It was originally imported from github (as few others)
  • It was renamed from it’s original name (as few others)

My question is what else I can check in order to get this project working?