I use the GitLab.com free plan to host certain projects for my clients in separate namespaces.
I have some base Docker images hosted in a private repository’s container registry in GitLab. I reference this image from other projects in other namespaces. Furthermore, I’m the sole developer in those namespaces, so It’s clear I have access rights to these docker images on that other repository.
Now, for some reason, I see an error in one project when it tries to pull the docker image:
Failed to pull image with policy "always": Error response from daemon: pull access denied for registry.gitlab.com/<url-to-image>, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (manager.go:203:0s)
This didn’t happen 10 days ago, but happens now. For no particular reason. There was no change regarding the CI setup. And it only seems to happen in that one project. I use it exactly in the same way as 10 days ago, when it worked.
I already tried switching to a different Docker image published in the same container registry as the one failing. But this fails too.
So, the main questions now are
- Were there any significant GitLab CI changes since May 23 that might cause this?
- Is there any limitation how often a project can pull images from the container registry in GitLab?
- Are there any other limitations I might need to know of that could cause the issue?
I have the same user permissions in all projects, in the ones where it still works and the single one, where it doesn’t. I can also pull the image locally.
Any inputs what to look for?
Hi @renestalder did you fix this in the end? I’ve had the same problem with images pulled from Docker Hub.
The only workaround I found was to set the project to public, while restricting all access to the features except the container registry to members only. I found no real solution.
Oh, that’s interesting, thanks. It hadn’t occurred to me that project visibility might be relevant here.
Facing the same problem without the possibility to make the project public, I took the following approach:
Create a deploy key with “read registry” permission in the repo for the “source” image
Pull this image in the pipeline of your target repo (docker login/docker pull/docker logout) using the deploy key created above
Now you have this image available locally in your build environment and can access it.
In my case I needed the Docker image located in source repo’s registry as the base image for my Dockerfile in the target repo. The code in the target repo’s pipeline looks basically like this:
- docker login -u "$BASE_IMAGE_TOKEN_USER" -p "$BASE_IMAGE_TOKEN_PASSWORD" "$CI_REGISTRY"
- docker pull "$CI_REGISTRY/my_source_repo:my_source_image_tag"
- docker logout
BASE_IMAGE_TOKEN_PASSWORD contain the deploy key created in the source repo stored in CI/CD variables.
Do you have a link to the issue? I seem to be having the same problem and could not work around it yet.
Same issue here. As mentioned by @renestalder I also had to set the source repo with the docker image
Public and restrict access to the features
Only Project Members. See following screenshot:
@mkind Your solution works! Here is a my challenge -
- When you do a docker pull - where is the image at this point ?
- I want to use the above as my base image and add few other stuff on top of this image. docker build and docker are giving errors
Can you please help me , Thanks
I solved the problem in Gitlab.com 16.2 via allowing access to my second project CI_JOB_TOKEN
this can be done in your base project in Settings > CI/CD > Token Access
you will need to add full notion of you “client” project, i.e. GROUP/SUBGROUP/PROJECT_NAME
after that in client project use docker login via token
docker login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY
It actually works without
docker login after granting access
can be used as base image for inherit flows
@oleksiy I am also using 16.2. I am not able to make it work. Here are my steps -
- Say I have a group/Project where I am maintaining all my images. I used manage and granted access to the other group ( say group B) where I need to access the images. It is giving me errors. However the project ( where I am hosting the images) is showing in shared projects list ( from group B)
Any thoughts as where I am messing it up.
This thread saved my sanity and mental health.
My case was the following.
I am maintaining a repository with common CI/CD assets like jobs shared across other projects and custom images for job runners.
I created a custom image for one of my runners (pre-built with some dependencies).
The image happily pulls locally with no credentials, but It simply failed to pull in a CI/CD context with the following message in the runner’s log:
“ERROR: Job failed: failed to pull image “registry.gitlab.com/redacted-group/redacted-sub-group/redacted-project/sonar-scanner-runner:latest” with specified policies [always]: Error response from daemon: pull access denied for registry.gitlab.com/redacted-group/redacted-sub-group/redacted-project/sonar-scanner-runner:latest repository does not exist or may require ‘docker login’: denied: requested access to the resource is denied (manager.go:237:0s)”
I finally solved this issue at the repository where I pushed the sonar-scanner-runner:latest image.
Go to “Settings” → “CI/CD” → Token Access
In the Token Access section, “Limit access to this project” should be switched on by default as a security measure.
If true, just include all the projects allowed to access the project.
Include all groups and subgroups like this:
Many thanks to Oleksiy for helping me out with this.
I tried to solve this with ChatGPT, Bard, Claude, and prayers to the eternal Force with a zero result for over 3 hours.
Finally - I settled with the old good way - Googling and digging into forums.
This is a highly kind reminder that a skilled developer is worth 100 more than any LLM.