Python coverage and pytest not producing anything in CI

GitLab CI/CD
1

I am having two issues when running python coverage in the gitlab CI where i do not really have a starting point to work on gettimg them fixed and so i wanted to ask here if anyone might know how to get going with that.

The first is a new issue that appeared yesterday when i wanted to update some of the dependencies of one of our projects.

Since then the job that previously worked has coverage (and pytest itself) not do anything when called in the CI. No output and no error messages.

However running coverage run --help produces the expected output.

This is a cut down version of the .gitlab-ci.yml that we are using.

workflow:
    rules:
        - if: $CI_COMMIT_REF_PROTECTED == "true"
        - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

default:
    tags:
        # Make you job be executed in a shared runner that has CVMFS mounted
        - cvmfs
    before_script:
        - set +e
        - alias python3="python"
        - sudo mv CommonAnalysisFramework ..
        - export CAFPATH="$PWD/.."
        - ln -s templates/UserSettingsGuideline.py UserSettings.py
        - source $CAFPATH/CommonAnalysisFramework/${SETUP_SCRIPT}

variables:
    SETUP_SCRIPT: setup_root628.sh

image: atlas/centos7-atlasos-dev

stages:
  - build
  - run
  - check
  - lint

test:
    stage: check
    before_script:
        - set +e
        - alias python3="python"
        - sudo mv CommonAnalysisFramework ..
        - export CAFPATH="$PWD/.."
        - ln -s templates/UserSettingsGuideline.py UserSettings.py
        - source $CAFPATH/CommonAnalysisFramework/${SETUP_SCRIPT}
        - source /cvmfs/sft.cern.ch/lcg/releases/LCG_104/pytest/7.4.0/x86_64-centos7-gcc12-opt/pytest-env.sh
        - source /cvmfs/sft.cern.ch/lcg/releases/LCG_104/pytest_cov/3.0.0/x86_64-centos7-gcc12-opt/pytest_cov-env.sh
        - unalias python3
    script:
        - set +e
        - echo " in main script"
        - coverage run --branch --source="." --omit="*config*,test/*.py,*UserSettings*,templates/*,*__init__.py" -m pytest test
        - coverage report -m --ignore-errors
        - coverage html --ignore-errors
    artifacts:
        paths:
            - .coverage
            - htmlcov

and this is the output of the job

Getting source from Git repository
00:02
Fetching changes...
Initialized empty Git repository in /builds/atlas-germany-dresden-vbs-group/CAF-Scripts/.git/
Created fresh repository.
Checking out 3f716e4e as detached HEAD (ref is Root628)...
Skipping Git submodules setup
Downloading artifacts
00:01
Downloading artifacts for compile (32909051)...
Downloading artifacts from coordinator... ok        host=gitlab.cern.ch OK
Downloading artifacts for run_USGuideline (32909052)...
Downloading artifacts from coordinator... ok        host=gitlab.cern.ch OK
Executing "step_script" stage of the job script
00:16
$ # INFO: Lowering limit of file descriptors for backwards compatibility. ffi: https://cern.ch/gitlab-runners-limit-file-descriptors # collapsed multi-line command
$ set +e
$ alias python3="python"
$ sudo mv CommonAnalysisFramework ..
$ export CAFPATH="$PWD/.."
$ ln -s templates/UserSettingsGuideline.py UserSettings.py
$ source $CAFPATH/CommonAnalysisFramework/${SETUP_SCRIPT}
lsetup               lsetup <tool1> [ <tool2> ...] (see lsetup -h):
 lsetup asetup        (or asetup) to setup an Athena release
 lsetup astyle        ATLAS style macros
 lsetup atlantis      Atlantis: event display
 lsetup eiclient      Event Index 
 lsetup emi           EMI: grid middleware user interface 
 lsetup ganga         Ganga: job definition and management client
 lsetup lcgenv        lcgenv: setup tools from cvmfs SFT repository
 lsetup panda         Panda: Production ANd Distributed Analysis
 lsetup pyami         pyAMI: ATLAS Metadata Interface python client
 lsetup root          ROOT data processing framework
 lsetup rucio         distributed data management system client
 lsetup scikit        python data analysis ecosystem
 lsetup views         Set up a full LCG release
 lsetup xcache        XRootD local proxy cache
 lsetup xrootd        XRootD data access
advancedTools        advanced tools menu
diagnostics          diagnostic tools menu
helpMe               more help
printMenu            show this menu
showVersions         show versions of installed software
 
06 Mar 2023 
centos7: setupATLAS is python3 environment by default (same as setupATLAS -3).
         If you need the previous python2 environment, do setupATLAS -2.
 	 We strongly encourage API users to migrate their scripts to python3.
          * * * This environment has been setup as python3. * * *
$ source /cvmfs/sft.cern.ch/lcg/releases/LCG_104/pytest/7.4.0/x86_64-centos7-gcc12-opt/pytest-env.sh
$ source /cvmfs/sft.cern.ch/lcg/releases/LCG_104/pytest_cov/3.0.0/x86_64-centos7-gcc12-opt/pytest_cov-env.sh
$ unalias python3
$ set +e
$ echo " in main script"
 in main script
$ coverage run --branch --source="." --omit="*config*,test/*.py,*UserSettings*,templates/*,*__init__.py" -m pytest test
$ coverage report -m --ignore-errors
No data to report.
$ coverage html --ignore-errors
No data to report.
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: command terminated with exit code 1

Where the noteworthy thing for me is the lack of any output after coverage run --branch --source="." --omit="*config*,test/*.py,*UserSettings*,templates/*,*__init__.py" -m pytest test

And another issue that i had previously (cant check if it still exists before this above is now fixed)
is that coverage always returned with an exit code of 0 even when there were test failures and thus the job did not show as failed. As i could not find any resources on that elsewhere and could not reproduce it locally i wanted to check if this maybe might be due to something with the CI.