Test Jobs Not Running in Build Job Container

Replace this template with your information

Describe your question in as much detail as possible:

• I was expecting the build job to build a container that would then be used for subsequent steps (testing and deployment).
• Instead, after the build job completes, the test job fails because the python command is not found.
• I was able to work around this by employing a before_script and a default image, but now the “build” process happens before every job. It works, but is not efficient and is not how I expected this to work.

stages:
  - build
  - test

build-job:
  only:
    - merge_requests
  stage: build
  image: repo.io/python:3.8.13-alpine3.15
  script:
    - echo "Installing from Nexus..."
    - python -m pip install .[test] --extra-index-url https://$USER:$PASS@repo.io/repository/libs-python-local/simple/ --no-input
    - echo "Install complete."

unit-test-job:
  only:
    - merge_requests
  stage: test
  script:
    - python -m pytest

Hi @joshweaver23

Yeah, that’s not really how this works, each job and stage is independent, so if you want build to create a container to run test, then you need to state that explicitly.

It’s not clear to me quite what would be most useful for you here, that depends a bit on the rest of your CI config, but there are two obvious ways for you to approach this, IMO.

Anchors (or extends)

If you only want to install dependencies sometimes, then YAML anchors or extends might be easiest:

stages:
  - test

.needs-python-libs: &needs-python-libs
  image: repo.io/python:3.8.13-alpine3.15
  before_script:
    - echo "Installing from Nexus..."
    - python -m pip install .[test] --extra-index-url https://$USER:$PASS@repo.io/repository/libs-python-local/simple/ --no-input
    - echo "Install complete."

unit-test-job:
  <<: *.needs-python-libs
  only:
    - merge_requests
  stage: test
  script:
    - python -m pytest

Using the container registry

If you need the dependencies on every other CI job, it would be easier to use the container registry for your project:

stages:
    - build
    - test

build-image:
    stage: build
    image: docker:stable
    services:
        - docker:dind
    script:
        - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
        # Next line assumes that there is a Dockerfile in the project root
        - docker build -t $CI_REGISTRY/GROUP/PROJECT/$CI_DEFAULT_BRANCH:latest .
        - docker push $CI_REGISTRY/GROUP/PROJECT/$CI_DEFAULT_BRANCH:latest
    rules:
        - changes:
              - Dockerfile
          when: always
        - when: never

test:
    stage: test
    image: registry.gitlab.com/GROUP/PROJECT/$CI_DEFAULT_BRANCH:latest
    only:
        - merge_requests
    script:
        - python -m pytest

It sounds like the container registry might solve this for me, thank you.

@snim2 What would the purpose of a “build” stage be? It’s obviously not what I was expecting or hoping for it to be, but maybe it is useful for something (the docs have build, test, and deploy so I feel like it has some importance).

In my example, the build stage builds the Docker container. If you mean what is the purpose in general, rather than in my examples, then strictly speaking there isn’t one! You can have as many stages as you want, and name them as you wish (with occasional restrictions, e.g. for GitLab pages). So, it’s entirely up to you to choose a convention.

Generally, I prefer to use prepare for something like building a Docker image, then build for building the SUT (if it needs building, but if I’m using a dynamic language, there may not be a build stage at all). I usually have lint (for anything that can be tested before building the app) and test for unit tests, etc. If the project has a deployment, then I’d usually have a deploy stage and after that verify, to make sure that the deployed environment is correct (e.g. returning HTTP 200 OK, if it’s a web app).

You’ll see lots of different practices around though.

Got it. Thank you so much for your responses.