Cobertura coverage report not showing in MR's diff

GitLab CI/CD
, , ,
1

I have recently added the cobertura coverage report to my repository, but it still does not show the coverage in an MR’s diff.

Here is the job of my .gitlab-ci.yml that generates the coverage report:

coverage-report:
  stage: coverage
  script:
    - tox -e coverage-report
  coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
  artifacts:
    name: "coverage"
    paths:
      - public/coverage
    expire_in: 1 week
    reports:
      cobertura: public/coverage/coverage.xml
    expose_as: "coverage"

And here is my tox.ini:

[tox]
envlist =
    coverage-report
minversion = 3.4

[testenv:coverage-report]
basepython = python2.7-32
skip_install = True
deps =
  coverage
commands =
  coverage run -m pytest -s -vv -x --junitxml=public/test-report.xml tests/
  coverage report
  coverage html
  coverage xml

I am pretty sure everything goes well with the report because not only does its XML exist under public/coverage (which I can see through the published artifacts), but the coverage % summary also shows up in the job and MR. But the coverage still does not show up in the MR’s diff. I also tried opening the Network tab of my browser and look for the merge_requests/26/coverage_reports.json HTTP request, and that is coming up empty (more specifically, the response is {"files":{}}), which I do not think is supposed to be happening.

I am using Python 2.7-32 and Coverage.py to get the report. My GitLab is self-hosted with version 14.9.5-ee. Here is a link to download my coverage.xml. It is not the complete coverage, but it shows 2 files which show up in the MR’s diff but have no coverage information.

2

Hi,

you are using the deprecated version for code coverage with cobertura that will certainly fail if you update to 15.0. See: Deprecations by version > 15.0 > artifacts:reports:cobertura keyword

A bit strange is that the 14.10 docs say it was removed with 14.10. Maybe a documentation error.

But certainly if you update now you won’t have to think about this later. So why not give ot a try?
See artifacts:reports:coverage_report

This would be your new job definition if i did no mistake while copy&pasting.

coverage-report:
  stage: coverage
  script:
    - tox -e coverage-report
  coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/'
  artifacts:
    expose_as: "coverage"
    name: "coverage"
    expire_in: 1 week
    paths:
      - public/coverage
    reports:
      coverage_report:
        coverage_format: cobertura
        path: public/coverage/coverage.xml
3

I too am having a difficult time giving up on getting this to work (Gitlab 15.6 CE Omnibus install). And seeing a recent Gitlab forum post got me inspired to try again. So, I have the few Cobertura <source> elements as paths relative to the project root - eg. a/b/c if a is a top-level directory in the project/repo. And I have python source files that show up as XML elements with attribute filename=“pkg/mod1.py” and since a/b/c/pkg/mod1.py is a valid file path in the repo & mod1.py is modified in the commit that triggers the pipeline, I was hoping to see some coverage visualization in the MR “Changes” view. But no such luck.

I then downloaded from the job that created the *-coverage.xml (actually 3 of them are found via the artifacts:reports:coverage_report’s wildcard path in my .gitlab-ci.yml). So per the docs, all 3 are uploaded. When I download the cobertura-coverage.xml.gz from the pipeline’s artifacts, I notice that the resulting XML file has 3 prologs (ie. <?xml version="1.0" ?>) each with the 1 element. Does this possibly affect the Gitlab visualization parser & thus, could be the root cause for why the visualization doesn’t render?

Firefox 111.0.1 chokes on loading the cobertura-coverage.xml with 3 prologs so perhaps? And the file still needs a root element with cardinality of 1 (firefox also complains if there is a 2nd root element - the 2nd element). I read the “How test coverage visualization works” section as being that my use case is supported: multiple Cobertura files can be specified, uploaded and no mention that I need to manipulate the XML files (ie. I could combine them in a script, create a root element with 1 or more elements but I don’t know if the Gitlab parser will process that ok. However, I read the DTD at cobertura/coverage-04.dtd at master · cobertura/cobertura · GitHub to indicate only 1 element is allowed (no so no explicit root element).

The docs for artifacts:reports:coverage_report indicates only 1 report per job. I read this as only 1 such block in the .gitlab-ci.yml but that the path value can contain wildcards so that multiple xml files might be uploaded for the 1 report definition. Have I misunderstood? And maybe only 1 XML file can be uploaded despite the presence of “files” in the same doc section and the “including wildcard paths”…“combines it together” documentation?

Any suggestions to the correct general approach? Not knowing what the parser is doing, I thought I’d ask…MTIA for any feedback.

additional info: long after (hours) the pipeline associated with the MR completes, I go to the specific MR URL, once the overview page loads I bring up Chrome’s dev tools, network view to watch requests. I then select the “Changes” view, there is a series of requests, one of which is a XHR https://gitlab.our-FQDN/org-name/proj-name/-/merge_requests/49/coverage_reports.json. But the response is minimal:

{"files":{}}

Searching through the *.log files under /var/log/gitlab/ modified yesterday, i do see what appears to be a successful upload of the coverage_report artifact:

{"severity":"INFO","time":"2023-04-06T18:38:08.109Z","correlation_id":"01GXBXS5N1C62WX6925QMPXPV2","project_id":5,"pipeline_id":1030,"pipeline_artifact_id":64,"message":"Created code coverage for pipeline."}

That was the time of yesterday’s only pipeline completing so it appears, well, correlated :slight_smile:

Now the big question is, what causes that API response to have an empty “files” object? XML Parsing error?

1 Like