Code coverage visualisation in Merge Requests

Replace this template with your information

Readin https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html it looks like it is possible to make use of code coverage visualisation in merge requests.

Unfortunately, I can’t get any coverage report in my MRs.

My configuration:

image: node:12-alpine

test:
  script:
    - yarn install
    - yarn test:ci
  artifacts:
    reports:
      cobertura: coverage/jest/cobertura-coverage.xml
    expire_in: 1 week
  when: always
  only:
     - merge_requests

In a job report:

Uploading artifacts...
coverage/jest/cobertura-coverage.xml: found 1 matching files and directories 
Uploading artifacts as "cobertura" to coordinator... ok  id=**** responseStatus=201 Created token=****

When coverage is dropping there is no visualisation for it, the same is when coverage increases.

What is wrong here and what can be improved?

1 Like

Hi @orthodoz - Thanks for posting!

If you can upload a bit of the cobertura-coverage.xml file or provide a link to the project if public I can take a look. I did want to call out there’s a known limitation that the parser does not support the <source> element. It is assumed that the filename of a class element contains the full path relative to the project root.

Hopefully this helps!

-James H, GitLab Product Manager, Verify:Testing

1 Like

Hi James, thanks for your reply!

It is a private project, therefore, unfortunately, I can’t share any details like that.

I can share following from my cobertura-coverage.xml:

  <sources>
    <source>/home/app/project-folder</source>
  </sources>
<packages>
    <package name="src" line-rate="0" branch-rate="1">
      <classes>
        <class name="App.tsx" filename="src/App.tsx" line-rate="0" branch-rate="1">
          <methods>
            <method name="(anonymous_0)" hits="0" signature="()V">
              <lines>
                <line number="5" hits="0"/>
              </lines>
            </method>
          </methods>
          <lines>
            <line number="5" hits="0" branch="false"/>
            <line number="6" hits="0" branch="false"/>
          </lines>
        </class>

Does it help or I need to provide more info?

Thanks!

That helps @orthodoz - If the git root of your project is “project-folder” the feature should be working as intended. If it is not you’ll need get the path more clearly defined.

-James H, GitLab Product Manager, Verify:Testing

Yes, @jheimbuck_gl, project-folder is a root, in a root folder, I have coverage folder with jest/cobertura-coverage.xml in it.

Still, no luck to obtain any Cobertura results for tests.

@orthodoz - i just want to confirm you are looking at the changes tab on the MR as that is where this information appears? Here’s an example from the GitLab project.

Beyond that i’m stumped but I’ve asked the team to take a look to make sure i’m not missing something.

-James H, GitLab Product Manager, Verify:Testing

@jheimbuck_gl - I am facing a similar issue on a self-hosted GitLab instance (version 13.2.4). I am essentially using the gradle example configuration from the code coverage example documentation. Like @orthodoz I see that the cobertura-coverage.xml file is uploaded and I can download it from the pipeline (although it is cobertura-coverage.xml.gz at that point). In my cobertura-coverage.xml file, the source tag is set to src/main/java and each of the filename attributes on the class elements also starts with src/main/java.

I tried to search through some of the source code and debug a little, but I haven’t really done any work with Vue or Ruby, so progress is a little slow. On the browser side, stepping through some of the JS code, it seems that endpointCoverage is not set and therefore fetchCoverageFiles is never called for me, but I have not been able to trace that back to what condition is missing on the backend.

Here are a few things I have tried without success:

  1. gradle example exactly as shown in the documentation
  2. change the XML file name to match the “default” name shown in the ruby code
  3. upload the XML file as a plain artifact and a report artifact
  4. running in a branch pipeline vs merge (detached) pipeline
  5. both the source and target branches having a coverage report

Below is a snippet from the logs of the job uploading the artifact:

$ python3 /opt/cover2cover.py build/jacoco/jacoco.xml src/main/java > build/cobertura-coverage.xml
$ python3 /opt/source2filename.py build/cobertura-coverage.xml
src/main/java
Uploading artifacts for successful job
Uploading artifacts...
build/cobertura-coverage.xml: found 1 matching files and directories
Uploading artifacts as "cobertura" to coordinator... ok id=5413 responseStatus=201 Created token=******
Job succeeded

Unfortunately, I also cannot provide access to my project as it is proprietary, but I would be happy to provide what additional information I can.

I solved my problem. It turns out the 13.2.4 documentation mentions enabling the :coverage_report_view feature flag, but the newer documentation I was looking at does not. I enabled the feature flag and it started working!

1 Like

@jmkelm08 - Sorry for the delay and i’m glad you got it working!

-James Heimbuck, GitLab Product Manager, Verify:Testing

We have a similar setup with the same python job converting jacoco → cobertura. We also have the feature active but still cannot see the coverage being visualised? :frowning:

Cobertura sample:

$ cat target/site/cobertura.xml
<coverage timestamp="1630678857.926" line-rate="0.8213507625272332" branch-rate="0.4375" complexity="164.0">
    <sources>        
       <source>./apm-core/apm-core-app/src/main/java</source>
        ...
    </sources>
    <packages>
        <package name="com.fiserv.apm.core.app.controller.operation.result" line-rate="0.42105263157894735" branch-rate="0.45454545454545453" complexity="39.0">
....    
    </packages>
    <classes>
        <class name="com.fiserv.apm.core.app.controller.operation.result.BuildProcessingSubsequent" filename="./apm-core/apm-core-app/src/main/java/com/fiserv/apm/core/app/controller/operation/result/BuildProcessingSubsequent.java" line-rate="0.1" branch-rate="0.25" complexity="7.0">
            <methods>
                <method name="&lt;init&gt;" signature="()V" line-rate="1.0" branch-rate="0.0" complexity="1.0">
                    <lines>
                        <line number="22" hits="1" branch="false"/>
                    </lines>
                </method>
                <method name="transform" signature="(Lcom/fiserv/apm/core/app/workspace/WsSubsequentTransaction5BuildProcessing;)Lcom/fiserv/apm/core/app/workspace/WsSubsequentTransaction6UpdateTransaction;" line-rate="0.0" branch-rate="0.0" complexity="2.0">
                    <lines>
                        <line number="27" hits="0" branch="false"/>
                        <line number="29" hits="0" branch="true" condition-coverage="0% (0/2)">
                            <conditions>
                                <condition number="0" type="jump" coverage="0%"/>
                            </conditions>
                        </line>
                        <line number="30" hits="0" branch="false"/>
                        <line number="32" hits="0" branch="false"/>
                        <line number="34" hits="0" branch="false"/>
                    </lines>
                </method>
                                ...

Here ‘./apm-fns/apm-fns-swish/apm-fns-swish-app/src/main/java/com/fiserv/apm/fns/swish/app/model/response/ReturnAndVoidAppResponse.java’ is the full file path from the repository root.

Configuration:

irb(main):001:0> Feature.enabled?(:coverage_report_view)
=> true

Result:

Artifact uploaded:

Due to configuration:

  artifacts:
    reports:
      cobertura: target/site/cobertura.xml

This is on Gitlab 13.9.

Any ideas what else to look into? @jheimbuck_gl

@ambition-consulting - Thanks for the post and welcome to the forum!

Things look good to me based on your post. There could be a problem similar to what was reported in this issue but I haven’t had a chance to dig into it to be sure. Some users also note it just takes some time to parse the file if very large.

If you expand the lines shown do any lines include the visualization?

-James H, GitLab Product Manager, Verify:Testing

I have checked the other lines and older merge requests - nothing visual there either. Also, I checked the html source: <div class="diff-td line-coverage left-side new"></div>

I guess this means that coverage is successfully turned on. I will check the pointing to the root of the repository - maybe the ~60 entries are confusing processing. But since we are on 13.9 this should be an issue anymore, right?

I worked quite hard on getting coverage to work and currently publish the jacoco report on gitlab pages for master builds. That’s better than nothing, but doesn’t help with code reviewing. Having the code tested is really important for our definition of done - and currently too difficult to identify.

@ambition-consulting - I’ll ping the team to see if there’s something i’m missing. The published report is certainly a good step in the meantime.

You might also consider using the Coverage Check Approval Rule if you are on a Premium license to enforce code coverage cannot decrease.

-James H, GitLab Product Manager, Verify:Testing

@jheimbuck_gl Yes we have the premium license, but we do not want to enforce yet… too much untested code slipped through review. :stuck_out_tongue:

1 Like

@ambition-consulting had any luck?
I am facing sam issue

Was there ever any resolution to this? We have the same problem here- no code coverage for Cobertura reports. I’ve tried everything, from fixing the paths in the cobertura.xml to ensuring that the coverage_report_view is enabled.

To flesh out my issue some more, here are some details.

GitLab Version: 13.12

Cobertura file snipped with no path alterations:

<?xml version="1.0" encoding="utf-8"?>
<coverage line-rate="0.8332999999999999" branch-rate="0.75" version="1.9" timestamp="1666643932" lines-covered="5" lines-valid="6" branches-covered="3" branches-valid="4">
  <sources>
    <source>C:\</source>
  </sources>
  <packages>
    <package name="TestLib" line-rate="0.8332999999999999" branch-rate="0.75" complexity="4">
      <classes>
        <class name="TestLib.MyTestClass" filename="b\Ventilation\sandbox\code-coverage-spike\TestLib\MyTestClass.cs" line-rate="0.8332999999999999" branch-rate="0.75" complexity="4">
          <methods>
          .............

Cobertura file with path alteration (no trailing slash on source):

<?xml version="1.0" encoding="utf-8"?>
<coverage line-rate="0.8332999999999999" branch-rate="0.75" version="1.9" timestamp="1666639721" lines-covered="5" lines-valid="6" branches-covered="3" branches-valid="4">
  <sources>
    C:\b\Ventilation\sandbox\code-coverage-spike
  </sources>
  <packages>
    <package name="TestLib" line-rate="0.8332999999999999" branch-rate="0.75" complexity="4">
      <classes>
        <class name="TestLib.MyTestClass" filename="TestLib\MyTestClass.cs" line-rate="0.8332999999999999" branch-rate="0.75" complexity="4">
          <methods>
          ............

Cobertura file with path alterations, plus trailing slash on source:

<?xml version="1.0" encoding="utf-8"?>
<coverage line-rate="0.8332999999999999" branch-rate="0.75" version="1.9" timestamp="1666644321" lines-covered="5" lines-valid="6" branches-covered="3" branches-valid="4">
  <sources>
    C:\b\Ventilation\sandbox\code-coverage-spike\
  </sources>
  <packages>
    <package name="TestLib" line-rate="0.8332999999999999" branch-rate="0.75" complexity="4">
      <classes>
        <class name="TestLib.MyTestClass" filename="TestLib\MyTestClass.cs" line-rate="0.8332999999999999" branch-rate="0.75" complexity="4">
          <methods>
          ................

The relevant portion of my gitlab-ci file looks like this:

  artifacts:
    name: Unit Test Results
    paths:
      - $CI_PROJECT_DIR\cobertura\*\coverage.cobertura.xml
      - $CI_PROJECT_DIR\junit\*-junit-test-result.xml
    reports:
      cobertura: $CI_PROJECT_DIR\cobertura\*\coverage.cobertura.xml
      junit: $CI_PROJECT_DIR\junit\*-junit-test-result.xml      

Also, the artifacts are successfully uploaded at the end of the job.

It would be very helpful to have some guidance on what I am doing wrong here, or what the underlying issue is.