Enabling cobertura

I am trying to enable cobertura reports in a self-hosted instance.

Following the docs, I’ve enabled the flag in the console:

Feature.all.map {|f| [f.name, f.state]}
=> [["multiple_merge_request_assignees", :on], ["coverage_report_view", :on]]

Looking at the job logs I have confirmed that the report xml is generated, and that it’s also specified in the ci yaml:

  artifacts:
    paths:
      - cov.html/
    reports:
      cobertura: cov.xml
      junit: tests.xml
    expire_in: 1 week

Further, the gitlab version is 12.10.1, and the runner version is 12.9.0.

Still I don’t seem to see the coverage report in the merge request. What am I doing wrong?

Sources that I used are:

• https://docs.gitlab.com/ee/ci/yaml/#artifactsreportscobertura
• https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html#enabling-the-feature
• https://docs.gitlab.com/ee/administration/troubleshooting/gitlab_rails_cheat_sheet.html

@anton_akhmerov thanks for the post! I’m not certain if you are trying to see the coverage value and diff on the Merge Request page OR the test coverage visualization in the Merge Request Diff?

If it is the first one double check your regex in the CI/CD settings. If the latter I believe you’ve setup everything correctly and we’d be interested to get a glimpse of what you are seeing in the MR diff view.

Thanks!

James H - GitLab Product Manager

It’s the latter, and somehow I see absolutely nothing in the diff view

From what I can see from comparing with the local run, the cobertura xml is correctly produced and has meaningful content.

@anton_akhmerov - i’m assuming you have a code coverage report for the target branch as well but can you confirm?

I confirm. Here’s the latest master pipeline uploading coverage report, and here the MR pipeline.

I double-checked in the console that the feature flag is enabled, and the diff seems to show no coverage.

Relevant exerpt from .gitlab-ci.yml (junit does work):

    reports:
      cobertura: cov.xml
      junit: tests.xml

@anton_akhmerov could you get add the cov.xml as an artifact in the .gitlab-ci.yml so we can download it and take a look? the current thinking is the format isn’t correct based on what we can see in the current setup but we’d like to confirm. thanks!

Sure thing!

It’s a generic python coverage report.

.xml extension is prohibited by discourse settings, so I’ll link the file instead: here.

@anton_akhmerov - Thanks for the file it was helpful. We have run into some cases, like yours, in which the implementation for the code coverage doesn’t handle absolute paths well. We have an issue open and a documented workaround for coverlet is noted in there. I’m not sure if a similar workaround approach would work here for you or not.

You can follow along on the issue as we make progress here and we will use your case to validate the fix before shipping it.

-James H - GitLab Product Manager

That’s helpful, thanks! In the interim, can you explain what kind of path structure does gitlab expect? I’ve tried a couple, but to no avail so far.

More trial and error and it’s done

@anton_akhmerov that’s fantastic!! We’d love to see what you did here or get a link so we can document the workaround until the issue noted is done.

Thanks!

-James H - GitLab Product Manager

These substitutions worked for me (this is with pytest-cov). I am not sure whether it’s the only choice, but since I couldn’t find documentation on what path structure gitlab expects, I tried until something worked.

Hi @anton_akhmerov, I’m struggling with the same setup (python with coverage/pytest-cov) but couldn’t find the proper strings to replace in the report file yet.
Tried to apply your substitutions but no luck. It would be nice to have the precise lines before and after your replacement, so that I would have a working reference to compare with.

Could you share them too? Thanks.

Please see the CI and test configuration in our repository. Hope that helps.

@anton_akhmerov’s workaround worked for us. Thank you.

In our case we are generating coverage in ephemeral containers with a path that GitLab couldn’t reference. So what Anton did is update the “source” to be the the root-level src dir of the code, and also insert this src dir into every “filename” in coverage.xml. With that in place, GitLab started showing the coverage inline in the MRs.

We used pytest:

pytest tests --cov --cov-report xml:coverage.xml --cov-config=.coveragerc --cov-branch --junitxml=codecov.xml

–cov-report xml:coverage.xml is required. Then in CI added the sed commands:

  variables:
    SRC_DIR: src
  script:
    - make tox
    - sed -i "s=<source>.*${SRC_DIR}</source>=<source>./${SRC_DIR}</source>=g" coverage.xml
    - sed -i "s;filename=\";filename=\"${SRC_DIR}/;g" coverage.xml

Hey there!

I’m also confronted with not seeing any visual indication of coverage (produced by jest) in Merge Request Diffs. The file is properly uploaded as artifacts:reports:junit and Feature.enable(:coverage_report_view) has been executed in the rails console.

In my case the root of the covered files is the repository itself (not a sub-directory like src). I’ve tried patching the cobertura xml to no avail:

<source></source>
<source>.</source>
<source>./</source>

and

<class name="App.tsx" filename="./App.tsx" line-rate="0.6315999999999999" branch-rate="0.5832999999999999">
<class name="App.tsx" filename="App.tsx" line-rate="0.6315999999999999" branch-rate="0.5832999999999999">

any ideas?

@rodneyrehm did you mean artifacts:reports:cobertura?

I’m having the same problem. I tried different <source> and filename paths to no avail. How can we determine what path the code coverage parser is expecting?

I got it to work after merging and having it run on target branch.

<source>./</source>

and

<class name="appointment_request_decorator" filename="app/decorators/appointment_request_decorator.rb" line-rate="0.82" branch-rate="0" complexity="0">

Thanks for the workaround, never would have guessed that.

For me this seems to work directly in a new branch:

    - sed -i "s=<source>.*</source>=<source>./</source>=g" coverage.$MODULENAME.xml
    - sed -i "s;filename=\";filename=\"${SRC_DIR}/;g" /coverage.$MODULENAME.xml

  variables:
    SRC_DIR: backend/$MODULENAME/$MODULENAME # needed by MR test coverage

Since coverage 5.0b2 you can now use relative path with this configuration file .coveragerc:

[run]
relative_files = True

Then run pytest in the CI with the --cov option pointing to the current directory:

pytest --cov ./ --cov-report xml:coverage.xml --cov-config=.coveragerc --cov-branch -ra

Reference: