"docker login" AUTH conditionally by "local gitlab-runner" vs. "gitlab-CI-pipeline"

Hi experts!

I’m experimenting with running gitlab-runner locally. My project is to build a docker image.

The basic steps of a docker build (to simplify) is:

  1. docker build (using Dockerfile)
  2. docker login (to Gitlab registry)
  3. docker push (to Gitlab registry)

Currently this is all done in a single job.

  stage: build
  - "docker build ..."
  - "docker login ..."
  - "docker push ..."

The key focus for this question is in step 2), which in the gitlab-CI land looks like:

docker login --username gitlab-ci-token --password $CI_JOB_TOKEN <CI_REGISTRY_URL>

If we want to run gitlab-runner locally however, we do not have access to the CI_JOB_TOKEN magic, and instead should be auth’ing using the developer’s personal username + access token.

My goal is to have the gitlab-ci.yml dynamically handle this complexity for the developer.

fwiw, this is a simplified snippet of how I’m running gitlab-runner locally:

sudo gitlab-runner \
  --debug \
  --log-level debug \
  exec docker \
    --docker-privileged \
    --docker-volumes /var/run/docker.sock:/var/run/docker.sock \
    --pre-clone-script 'echo; echo; echo $PWD; ls -l "$PWD"; echo; echo' \

At first I tried separating out the “docker login” script from the job, into its own stage called docker_login, which has two conditional jobs, 'twas something like:

  stage: docker_login
  script: "docker login --username gitlab-ci-token --password $CI_JOB_TOKEN $CI_REGISTRY"
      - $DOCKER_USERNAME == null

  stage: docker_login
  script: "docker login --username $DOCKER_USERNAME --password $DOCKER_PERSONAL_ACCESS_TOKEN $CI_REGISTRY"

, there I’m relying on a wrapper script that defines these ENV vars and passes them in. Leveraging “only”, you can see the simple logic – if those ENVs exist, we’re a local gitlab-runner, so login using the user access token, else login using the gitlab CI_JOB_TOKEN magic. (excuse the hacks, eventually to transition to: https://docs.docker.com/engine/reference/commandline/login/#credentials-store)

In theory this could work … EXCEPT … that gitlab-runner can only run one job at a time. https://gitlab.com/gitlab-org/gitlab-runner/issues/2797 … Even if we ran the login job first, then the build job second, this is docker-in-docker, so that first “docker login” gets blown away :frowning:

A “HACKY” way to deal with this is to have the developer comment out the docker login line, and hack it for their personal login in order to push into the registry. But (A) that’s annoying to remember and do, and (B) we risk accidentally checking that in and breaking the real CI (counter-productive).

Another thought I had was to have the local wrapper script force the docker login using one of these flags

--pre-build-script 'docker login --username $DOCKER_USERNAME --password $DOCKER_PERSONAL_ACCESS_TOKEN $CI_REGISTRY' \
--post-build-script 'docker login --username $DOCKER_USERNAME --password $DOCKER_PERSONAL_ACCESS_TOKEN $CI_REGISTRY' \

The “post build” for some reason didn’t work, but the “pre build” did! … whether this is advisable or not is uncertain. Further, even with this solution, I need to decide how to conditionally login via the CI_JOB_TOKEN magic using pre-build stage jobs, OR to do it in the main build job and allow failures … (doesn’t seem good either).

I hope I’ve made clear enough the problem and goal.
Has anyone dealt with this problem before and has any proposal/solutions?

I’m not sure if I understood the problem correctly, but won’t it possible to pass --env CI_JOB_TOKEN to the local gitlab-runner?

Unfortunately not. A user’s personal access token needs to auth against their own username, not the gitlab-ci-token username which is required for the CI_JOB_TOKEN magic.

(of course, one could change the docker login to use ENV variables for both the username and password – but I’m not sure as of yet how gitlab CI proper can be configured to set an ENV var)

Just tried my experiment of having a pre-stage do the “docker login” before a subsequent stage runs “docker push” - the AUTH is lost, so that doesn’t work. Darn.