Failed to pull image in Kubernetes but succeeded with "docker pull" command with the same credential from gitlab registry

Replace this template with your information

Describe your question in as much detail as possible:
I have docker image in gitlab registry. I login in with “docker login registyr.gitlab.com” and have the credential in my account directory such as ~/.docker/config.json after “docker login” command.

The file looks like below

$ cat ~/.docker/config.json
{
	"auths": {
		"registry.gitlab.com": {
			"auth": "Y3Jxxxxxxxxx="
		}
	},
	"HttpHeaders": {
		"User-Agent": "Docker-Client/19.03.6 (linux)"
	}
}

After login command, I can pull successfully my docker image from gitlab registry. The command I used is

$ docker pull registry.gitlab.com/crong.moim/xxxs/xxx-fileupload

I want to use gitlab regitry with my kubernetes deployment. First I created kubernetes secret with docker login information

$ kubectl create secret generic regcred \
    --from-file=.dockerconfigjson=/home/psw/.docker/config.json \
    --type=kubernetes.io/dockerconfigjson

And I made the deployment yaml for my sample service.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-xxxxx-fileupload
  namespace: xxxxx-op
spec:
  replicas: 3
  selector:
    matchLabels:
      app: xxxxx-fileupload
  template:
    metadata:
      name: xxxxx-fileupload-pod
      labels:
        app: xxxxx-fileupload
    spec:
      containers:
      - name: container-xxxxx-fileupload
        image: registry.gitlab.com/crong.moim/xxxxx/xxxxx-fileupload:latest
        ports:
        - containerPort: 8003
      imagePullSecrets:
      - name: regcred
---
apiVersion: v1
kind: Service
metadata:
  name: service-xxxxx-fileupload
  namespace: xxxxx-op
spec:
  ports:
    - name: web-port
      port: 8003
      targetPort: 8003
  selector:
    app: xxxxx-fileupload
  type: ClusterIP

However, while applying above deployment, I got errors.

$ kubectl describe pod deployment-xxxxx-fileupload-6cd9496fd6-6cpfn -n xxxxx-op
......
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age               From               Message
  ----     ------     ----              ----               -------
  Normal   Scheduled  20s               default-scheduler  Successfully assigned petus-op/deployment-petus-fileupload-6cd9496fd6-6cpfn to psw-sub
  Normal   BackOff    15s               kubelet, psw-sub   Back-off pulling image "registry.gitlab.com/crong.moim/petus/petus-fileupload:latest"
  Warning  Failed     15s               kubelet, psw-sub   Error: ImagePullBackOff
  Normal   Pulling    3s (x2 over 17s)  kubelet, psw-sub   Pulling image "registry.gitlab.com/crong.moim/petus/petus-fileupload:latest"
  Warning  Failed     2s (x2 over 16s)  kubelet, psw-sub   Failed to pull image "registry.gitlab.com/crong.moim/petus/petus-fileupload:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://registry.gitlab.com/v2/crong.moim/petus/petus-fileupload/manifests/latest: denied: access forbidden
  Warning  Failed     2s (x2 over 16s)  kubelet, psw-sub   Error: ErrImagePull

I DID succeed in pulling image in kubernetes deployment last week on the same server machines. But I re installed kubernetes cluster to study it again on the same machines with the same recipe I used last week.
I don’t know why it failed this time. What shell I do for debug?

I also tried to create the credential with the command below

$ kubectl create secret docker-registry <name> --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL

But I’ve got the errors also.

Thank you.

It is absolutely my fault. gitlab is no guilty.

My deployment is in “xxxxx-op” namespace, but I created the registry secret in the “default” namespace. That was the problem.

Gitlab is innocent.

Thanks Gitlab.