Can't access Gitlab container registry from Kubernetes

Hi there,

I am just about to start to work with the GitLab container registry.
We have a somewhat special approach, an Omnibus install, which does not user the built in nginx but rather an apache acting as a reverse proxy.

So https://gitlab.example.com gives access to the gitlab frontend, whereas https://gitlab-registry.example.com proxies for http://localhost:5000 opening the docker repository.

Everything works quite nice. I can:

  • login to the registry docker login gitlab-registry.example.com
  • pull and push images
  • Build images in CI-Pipelines (using kaniko) and push them to the registry

Now I am trying to pull the images into my kubernets application.
I followed the steps to autenticate in the pod using imagePullSecrets.

Nevertheless I fail, inspecting the status gives me:

$ kubectl describe pod
[…]
Failed to pull image “gitlab-registry.example.com::latest”: rpc error: code = Unknown desc = Error response from daemon: Head “https://gitlab-registry.example.com/v2//manifests/latest”: denied: access forbidden

Trying to access the V2-API with curl and the credentials gives me the same access forbidden error.
Also trying to connect phpstorm with the GitLab container registry (with Access Token) gives me this error.

So I have no Idea what to do.
The settings I changed in gitlab.rb are:

registry_external_url 'https://gitlab-registry.example.com'
gitlab_rails['registry_enabled'] = true
registry['enable'] = true

And the relevant apache config lines are:

        ProxyRequests Off
        SSLProxyEngine On
        ProxyPreserveHost on

        ProxyPass / http://localhost:5000/ nocanon
        ProxyPassReverse / http://localhost:5000/ nocanon

        AllowEncodedSlashes NoDecode

But maybe these should be different.

I hope someone has an idea.

Best regards and thanks in advance Willi