The artifact gets saved just fine and looks normal when I download it, but I don’t get any visualization as described in the documentation linked above. I just updated the gitlab runner to V14.0.0 thinking that might be the problem, it’s not.
I have spent nearly 2 days trying to get this simple task to work. I see in posts that I have searched that there are some feature flags to enable to get this to work, but from what I can tell they are no longer needed. I don’t have any sort of regex pattern setup as from my understanding that is only for printing the coverage info to stdout.
I’m sure I am overlooking something trivial and I really need a sanity check here as I have already spent way more time on this than I can afford.
I just wanted to make sure you’re looking on the Changes Tab of the MR and the tests ran/cobertura.xml file was created as part of that MR’s latest pipeline? I should also call out there there have been fixed for automatic path correction in the 13.8 and 13.9 versions so if you are not running that on a self managed issue there could be problems with the paths.
I am in fact not using self-hosted gitlab, and I am looking at the changes tab of the MR. I double checked and the cobertura-coverage.xml is getting generated in that MR pipeline.
After looking through the link you provided on automatic path correction I think the cobertura-coverage.xml file might be the issue. Since we have a monorepo for this project the pipeline jobs are running on different directories of the repository (by that I mean the test pipeline has a job for running the tests in $PROJECT_ROOT/feature1, another job in $PROJECT_ROOT/feature2, etc. Because of this (and the fact that the test files are no in the root dir) the .xml generated shows the following
I am going to attempt modifying these values in the job script to match the latter output. Please advise if you can think of anything else. Thank you for your help!
I am still not seeing code overage in the MR diff view, I am also still not seeing a code coverage % in the CI => Jobs page either.
If you have any ideas or a course of action you would recommend please let me know as I am running out of ideas. I was considering opening an issue, but I want to make 100% sure that this isn’t a user error before I do. Thank you for your guidance.
Hi @Btripp1986! I noticed filename="sdk/src/contexts/ThemeProvider.tsx" from the sed modified XML. Just to confirm, is it correct that the full path (relative to the project root) of the class ThemeProvider.tsx, is sdk/src/contexts/ThemeProvider.tsx which includes sdk/?
Yes, the full path to ThemeProvider.tsx is <project-root>/sdk/src/contexts/ThemeProvider.tsx
After reading @jheimbuck_gl comment, I know I do not have the RegEx patter correct, so I assume that is the reason for coverage % not showing in the Jobs page. Would this affect the visualization in the MR diff, or is that feature unconcerned with the RegEx pattern?
No it won’t affect the MR diff visualization. They rely on 2 different artifacts.
When on the MR diff page, can you please check in your browser’s console if it made any requests to something like .../-/merge_requests/:mr_id/coverage_reports.json? And if it did, can you check if the JSON response contained any map of filenames?
My team has changed its git strategy and we are now committing directly to the default branch, so coverage visualization is no longer required for us(no MRs).
I did get the coverage % to show up in the jobs page, which will be enough for our purposes.
If we change our git strategy again I will be implementing the coverage visualization, in which case I will refer back to this thread.
I thank you and James for your help. You folks are great and I preach your product to everyone I know.
Glad @Btripp1986 & @iamricecake posted on this topic. I’m running a self-hosted 13.12.3 setup. I’ve got pipelines uploading cobertura artifacts (I can download them ok). Am also working through how to get the visualization working in the MR pages. Checking the console when loading the diffs page, I see the GET request for https://gitlab…stevens.edu/rt…/ioif/-/merge_requests/32/coverage_reports.json with response SC 204. But Chrome reports the resource size of 0 (the content-length header not shown). Suggests to me that whatever processing is returning the payload of the request has determined no content should be provided.
Any chance this is a function of the paths defined in the , <class filename=> attributes not being correct? [FYI: I now have a sed script to do as Ben indicated to ‘massage’ these paths but not used on this particular set of uploaded artifacts] - as these cobertura formated coverage xml files are created by the pytest-cov plugin during a pytest launched by tox, so the paths are tox testenv locations.
Looking at the log files generated in our installation both at upload time and this morning when trying to get the MR diffs visualization working, I don’t see any details revealing any issues with retrieving, using the coverage_reports.json.
What log file(s) might reveal issues with generating this coverage_reports.json payload?
MTIA for any feedback, insights.
some more details: the file downloaded via the ‘Download tests:cobertura artifacts’ menu choice in the pipeline’s Artifacts menu is the aggregation of the 3 different cobertura formatted coverage reports created, uploaded by the pipeline’s tests job. And in my Chrome console, there is a 2nd GET request for the coverage_reports.json doc. And this request’s response has a json object. But there are only 2 files with coverage lines, counts being returned. In the downloaded coverage XML, there are other elements in the same instance. Which have the same valid paths [1 out of 3 uploaded coverage XML files have tox testenv paths, but not this one so, as indicated by the Gitlab docs on auto path correction, the source files are ‘discoverable’ by the coverage paths. No idea yet as to why the other elements not being in the response.
The other 2 instances have no filename instances returned in the response body. Which is possibly explained by these 2 instances having incorrect tox testenv paths: <class branch-rate="0" complexity="0" filename=".tox/py37-graphdb/lib/python3.7/site-packages/ioif_core/Exceptions.py"
Hey, @SteveHespelt, have you had any luck figuring this out? I am on the same version of self-hosted gitlab as you are, and cannot for the life of me figure out how to get code coverage working. I’ve tried everything from massaging the paths in the cobertura xml files to enabling a feature flag. Nothing seems to work.
Hi @aronweiler, I’ve not. I resorted to downloading the htmlcov artifacts I generate via pytest --cov-report=html:{envtmpdir}{/}htmlcov at the same time I’m generating the Cobertura xml file via --cov-report=xml:test_reports/coverage.xml in my tox usage. Tox commands entry then zips up the htmlcov directory and mv the zip over to where the artifacts.paths entry for the job’s zip files. Better than nothing, my boss likes to review those occasionally.
I am using on prem gitlab v16.04. I have set up pipeline as below
# .gitlab-ci.yml
image: node:12
# Define the stages of the pipeline
stages:
- test
cache:
paths:
- node_modules/
# Define the job that will run tests
test:
stage: test
script:
- npm install
- npm run test -- --coverage --verbose --coverageReporters=cobertura
artifacts:
reports:
coverage_report:
coverage_format: cobertura
path: coverage/cobertura-coverage.xml
# Add coverage to code coverage statistics
coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
# Define when this job should be executed
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"'
I am trying to get test coverage visualizer in diff view of the merge request working. but i cannot see it. I can see the artifact for each job in the pipeline when run. I can see the cobertura-coverage.xml file in the artifact
