Hi experts!
I’m experimenting with running gitlab-runner locally. My project is to build a docker image.
The basic steps of a docker build (to simplify) is:
- docker build (using Dockerfile)
- docker login (to Gitlab registry)
- docker push (to Gitlab registry)
Currently this is all done in a single job.
build-docker:
stage: build
script:
- "docker build ..."
- "docker login ..."
- "docker push ..."
The key focus for this question is in step 2), which in the gitlab-CI land looks like:
docker login --username gitlab-ci-token --password $CI_JOB_TOKEN <CI_REGISTRY_URL>
If we want to run gitlab-runner locally however, we do not have access to the CI_JOB_TOKEN magic, and instead should be auth’ing using the developer’s personal username + access token.
My goal is to have the gitlab-ci.yml dynamically handle this complexity for the developer.
fwiw, this is a simplified snippet of how I’m running gitlab-runner locally:
sudo gitlab-runner \
--debug \
--log-level debug \
exec docker \
--env DOCKER_USERNAME="$DOCKER_USERNAME" \
--env DOCKER_PERSONAL_ACCESS_TOKEN="$DOCKER_PERSONAL_ACCESS_TOKEN" \
--docker-privileged \
--docker-volumes /var/run/docker.sock:/var/run/docker.sock \
--pre-clone-script 'echo; echo; echo $PWD; ls -l "$PWD"; echo; echo' \
$JOB_TO_RUN
At first I tried separating out the “docker login” script from the job, into its own stage called docker_login, which has two conditional jobs, 'twas something like:
docker-login-gitlab-ci:
stage: docker_login
script: "docker login --username gitlab-ci-token --password $CI_JOB_TOKEN $CI_REGISTRY"
only:
variables:
- $DOCKER_USERNAME == null
- $DOCKER_PERSONAL_ACCESS_TOKEN == null
docker-login-local:
stage: docker_login
script: "docker login --username $DOCKER_USERNAME --password $DOCKER_PERSONAL_ACCESS_TOKEN $CI_REGISTRY"
only:
variables:
- $DOCKER_USERNAME
- $DOCKER_PERSONAL_ACCESS_TOKEN
, there I’m relying on a wrapper script that defines these ENV vars and passes them in. Leveraging “only”, you can see the simple logic – if those ENVs exist, we’re a local gitlab-runner, so login using the user access token, else login using the gitlab CI_JOB_TOKEN magic. (excuse the hacks, eventually to transition to: https://docs.docker.com/engine/reference/commandline/login/#credentials-store)
In theory this could work … EXCEPT … that gitlab-runner can only run one job at a time. https://gitlab.com/gitlab-org/gitlab-runner/issues/2797 … Even if we ran the login job first, then the build job second, this is docker-in-docker, so that first “docker login” gets blown away 
A “HACKY” way to deal with this is to have the developer comment out the docker login line, and hack it for their personal login in order to push into the registry. But (A) that’s annoying to remember and do, and (B) we risk accidentally checking that in and breaking the real CI (counter-productive).
Another thought I had was to have the local wrapper script force the docker login using one of these flags
--pre-build-script 'docker login --username $DOCKER_USERNAME --password $DOCKER_PERSONAL_ACCESS_TOKEN $CI_REGISTRY' \
--post-build-script 'docker login --username $DOCKER_USERNAME --password $DOCKER_PERSONAL_ACCESS_TOKEN $CI_REGISTRY' \
The “post build” for some reason didn’t work, but the “pre build” did! … whether this is advisable or not is uncertain. Further, even with this solution, I need to decide how to conditionally login via the CI_JOB_TOKEN magic using pre-build stage jobs, OR to do it in the main build job and allow failures … (doesn’t seem good either).
I hope I’ve made clear enough the problem and goal.
Has anyone dealt with this problem before and has any proposal/solutions?