Pipeline fails during Docker Build accessing GitLab Package Repo, 401 Credentials error

Quick Background

I am trying to build a containerized, command-line-tool. I could not get my local package to build inside the docker image, so I decided to install it from the GitLab Package Registry. (a previous step in the pipeline builds & pushes the package to package registry)

I have a number of internal packages, so the below error was happening when I tried to build the package with a python setup.py install type command as well. I cannot seem to access the GitLab package registry from inside the Docker build.

Pipeline fails during Docker Build accessing GitLab Package Repo

My Docker file works locally when I use build_args with my GitLab PIP token, but fails when executed during the CI/CD Pipeline with 401 Credentials not correct error.

Here is the error message:

Step 10/13 : RUN pip install leadroll-jobs --extra-index-url https://__token__:${GITLAB_PIP_TOKEN}@gitlab.com/api/v4/projects/26541953/packages/pypi/simple --no-deps
 ---> Running in e6e6ddbda117
Looking in indexes: https://pypi.org/simple, https://__token__:****@gitlab.com/api/v4/projects/26541953/packages/pypi/simple
WARNING: 401 Error, Credentials not correct for https://gitlab.com/api/v4/projects/26541953/packages/pypi/simple/leadroll-jobs/
ERROR: Could not find a version that satisfies the requirement leadroll-jobs (from versions: none)
ERROR: No matching distribution found for leadroll-jobs
WARNING: 401 Error, Credentials not correct for https://gitlab.com/api/v4/projects/26541953/packages/pypi/simple/pip/

The .gitlab-ci.yml snippet:


docker-build-master:
  image: docker:latest
  stage: Package
  services:
    - docker:dind

  needs:
    - test
    - job: build-all
      artifacts: true

  before_script:
   - docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY_PASSWORD $CI_REGISTRY

  script:
    - echo "${GITLAB_PIP_TOKEN}"
    - echo "${CI_REGISTRY_IMAGE}"
    - echo "${CI_COMMIT_TAG}"
    - export MY_TAG=${CI_COMMIT_TAG#*v}
    - echo ${MY_TAG}
    - docker build --pull -t $CI_REGISTRY_IMAGE .
    - docker build --build-arg GITLAB_PIP_TOKEN=${GITLAB_PIP_TOKEN}  --build-arg MY_TAG=${MY_TAG} --tag $CI_REGISTRY_IMAGE:latest --tag $CI_REGISTRY_IMAGE:${MY_TAG} .
    - docker push $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:${MY_TAG}

  only:
    - master

My Dockerfile

FROM python:3.9-slim

ARG GITLAB_PIP_TOKEN
ARG MY_TAG
ENV GITLAB_PIP_TOKEN=$GITLAB_PIP_TOKEN

RUN useradd --create-home --shell /bin/bash app_user

WORKDIR /home/app_user

COPY requirements.txt ./
RUN echo $MY_TAG

RUN pip install --no-cache-dir --upgrade pip
RUN pip install leadroll-jobs --extra-index-url https://__token__:${GITLAB_PIP_TOKEN}@gitlab.com/api/v4/projects/26541953/packages/pypi/simple --no-deps
RUN pip install --no-cache-dir -r requirements.txt

USER app_user
CMD ["bash"]

As you can see from the error message above, I am getting a credential error.

This works when I run the docker command locally:

docker build -t myapp . --build-arg GITLAB_PIP_TOKEN=$GITLAB_PIP_TOKEN