CI not running for pushes to master

I recently restructured my CI configuration, making changes to some script in jobs, moving some stuff into and out of templates, but nothing changing with stages or job dependencies. Now something strange is happening: the pipeline does not run when pushes are made to master, only those pushed to other branches. According to my project’s CI lint tool, there are no syntax errors, and it shows the following visualization:

The “lint” tab reports “Only policy: branches, tags” for all jobs. I think this is the default value, since I didn’t set it explicitly with only or rules.

Version: 13.8.4-ee (we get EE for free as a large scientific organisation, but no commercial support)
We’re self-hosted. Runners are GitLab runners, not sure what version.

Any help figuring this out, or suggestions for stuff to try, would be appreciated!
Here is my CI file:

## GitLab CI configuration.

stages:
  - build
  - test
  - docs
  - deploy

###############
# Build stage #
###############

.template/build/manylinux: &template-build-manylinux
  stage: build
  image: quay.io/pypa/manylinux2014_x86_64
  script:
    - yum install -y suitesparse-devel
    - ${PYBIN}/pip install -r requirements-build.txt
    - ${PYBIN}/pip wheel . --no-deps -w wheelhouse
    - auditwheel repair wheelhouse/*.whl
  artifacts:
    paths:
      - wheelhouse

build/manylinux/3.8:
  <<: *template-build-manylinux
  needs: []
  variables:
    PYBIN: /opt/python/cp38-cp38/bin

build/manylinux/3.9:
  <<: *template-build-manylinux
  needs: []
  variables:
    PYBIN: /opt/python/cp39-cp39/bin

##############
# Test stage #
##############

.template/test/ubuntu: &template-test-ubuntu
  stage: test
  before_script:
    - pip3 install -r requirements-test.txt

test/3.8/functional:
  <<: *template-test-ubuntu
  needs:
    - build/manylinux/3.8
  image: python:3.8
  script:
    - pip3 install wheelhouse/finesse-*-cp38-cp38-manylinux2014_x86_64.whl
    - pytest tests/functional

test/3.9/functional:
  <<: *template-test-ubuntu
  needs:
    - build/manylinux/3.9
  image: python:3.9
  script:
    - pip3 install wheelhouse/finesse-*-cp39-cp39-manylinux2014_x86_64.whl
    - pytest tests/functional

test/3.8/validation:
  <<: *template-test-ubuntu
  needs:
    - build/manylinux/3.8
  image: python:3.8
  script:
    - pip3 install wheelhouse/finesse-*-cp38-cp38-manylinux2014_x86_64.whl
    - pytest tests/validation

test/3.9/validation:
  <<: *template-test-ubuntu
  needs:
    - build/manylinux/3.9
  image: python:3.9
  script:
    - pip3 install wheelhouse/finesse-*-cp39-cp39-manylinux2014_x86_64.whl
    - pytest tests/validation

# Test coverage report generation. 
#
# Note that this performs its own build because creating coverage report involves rebuilding the
# Cython extensions with special flags anyway.
test/coverage:
  stage: test
  needs: []
  image: quay.io/pypa/manylinux2014_x86_64
  variables:
    PYBIN: /opt/python/cp39-cp39/bin
  script:
    - yum install -y suitesparse-devel
    - ${PYBIN}/pip install -r requirements-build.txt
    - ${PYBIN}/pip install -r requirements-test.txt
    # Install editable mode so in-place pyx files are picked up by coverage.
    - ${PYBIN}/pip install -e .
    # Build Cython extensions with coverage support.
    - ${PYBIN}/python setup.py build_ext --force --inplace --define CYTHON_TRACE --define CYTHON_TRACE_NOGIL
    # Run all tests and generate Cobertura XML formatted coverage report.
    - ${PYBIN}/pytest --cov=finesse --cov-config=setup.cfg --cov-report=xml tests
    # Generate human readable coverage report.
    - ${PYBIN}/pycobertura show coverage.xml  # Necessary so the GitLab CI regex picks up the total.
  coverage: '/^TOTAL\s+.*\s+(\d+\.?\d*)%/'  # Regex to allow GitLab to extract and display coverage.
  artifacts:
    reports:
      # Displays coverage in MR diffs.
      cobertura: coverage.xml

##############
# Docs stage #
##############

docs/html:
  stage: docs
  needs:
    - build/manylinux/3.9
  image: python:3.9
  script:
    - pip install -r requirements-doc.txt
    # Sphinx autodoc needs to be able to import finesse and finesse_sphinx.
    - pip install wheelhouse/finesse-*-cp39-cp39-manylinux2014_x86_64.whl
    - cd docs
    - make html
    - cd ..
  artifacts:
    paths:
      - docs/build/html/

################
# Deploy stage #
################

# Publish the documentation only for changes to master, and only when all tests pass.
pages:
  stage: deploy
  needs:
    # Everything else must have passed. Grab artifacts only from the doc build job.
    - job: test/3.8/functional
      artifacts: false
    - job: test/3.9/functional
      artifacts: false
    - job: test/3.8/validation
      artifacts: false
    - job: test/3.8/validation
      artifacts: false
    - job: test/coverage
      artifacts: false
    - job: docs/html
      artifacts: true
  script:
    - mv docs/build/html public
  artifacts:
    paths:
      - public

I found the cause: the last job needs keyword made nothing run. Removing this made it work. Still don’t know why it wasn’t working with this keyword in, but at least I’ve got a fix now.

(In any case I didn’t need to have that complicated needs setup because it was already in a later stage and would only run if the previous stages passed).

1 Like