Can't pull docker image on already deployed pod

We have setup gitlab pipeline and k8s deploy using gitlab agent on our cluster.
The deployment is always working fine when executed by the pipeline scripts.

But sometimes, after a certain time, the cluster can’t pull again the image with this error :

Failed to pull image "": 
failed to pull and unpack image "": 
failed to resolve reference "[](": 
failed to authorize: failed to fetch oauth token: unexpected status: 401 Unauthorized

It happens after node restart or reprovisionned (a new node), sometimes on pod restart too, only on specific pods (3 pods actually). The secret (with dockerconfigjson entry) is still there.

We are using gitlab v16.3 and gitlab-agent v16.4.0

1 Like

I finally found out what was the problem !
The “not working” projects had not the same auto-devops configuration.

If you choose : “Continuous deployment to production” the generated deploy token is not the same.

The good working solution is this one (on the picture) :