Kaniko fails to authenticate to gitlab.com registry

Hi there.

I’m currently having trouble using kaniko to push my docker image to this project’s docker registry.

As you can see on this merge request, I’ve been trying to introduce a “publish” step in my CI that reproduces almost exactly the doc for using kaniko with gitlab-CI, the only difference is that I’m fetching my binary as an artifact from a previous step, and that my dockerfile is located at $CI_PROJECT_DIR/docker/Dockerfile :

  stage: publish
    - job: build
      artifacts: true
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/docker/Dockerfile --destination $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

Whatever I try to do, I’m always getting an authentication error:

error checking push permissions -- make sure you entered the correct tag name, 
and that you are authenticated correctly, and try again: 
checking push permission for "registry.gitlab.com/neuware/xkcd-bot:f4137e95": 
POST https://registry.gitlab.com/v2/neuware/xkcd-bot/blobs/uploads/: 
UNAUTHORIZED: authentication required; 
[map[Action:pull Class: Name:neuware/xkcd-bot Type:repository] map[Action:push Class: Name:neuware/xkcd-bot Type:repository]]

Is there something broken with the current kaniko release, or the doc, or gitlab’s registry, or am I missing something obvious?

I tried multiple things to reproduce this issue, including building and pushing the same image, using the same version of kaniko executor on my laptop, and I didn’t get any error.

I tried replacing the docker configuration (/kaniko/.docker/config.json) with the auth information I’m using on my laptop : it just won’t work in the CI. I’m always getting this “unauthorized” error message no matter what.

I had this working in my pipeline with this setup. it stopped working. Seems kaniko no longer allows username/password directly in the .docker/config.json file. It must not use the base64 encoded value of those combined. gitlab registry works like dockerhub. Here is the kaniko info on auth for dockerhub:

so it seems Building images with kaniko and GitLab CI/CD | GitLab is a little out of date. As soon as I updated this to base64 auth value, it started working again. heres my line for making the .docker/config.json:
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"auth\":\"$(echo -n ${CI_REGISTRY_USER}:${CI_REGISTRY_PASSWORD} | base64)\"}}}" > /kaniko/.docker/config.json