Cobertura coverage not showing in diff or jobs

I am trying to get test coverage visualization for our monorepo as described here Test Coverage Visualization

We are using a self-managed gitlab runner with the docker+machine executor hosted on AWS EC2 instances. The job from the gitlab-ci.yml is below

sdk:
  stage: test
  needs: []
  artifacts:
    when: always
    reports:
      cobertura: /builds/vividcloud/autoguide/amr/sdk/coverage/cobertura-coverage.xml
  <<: *main
  <<: *tests
  <<: *rules
  <<: *tags

The artifact gets saved just fine and looks normal when I download it, but I don’t get any visualization as described in the documentation linked above. I just updated the gitlab runner to V14.0.0 thinking that might be the problem, it’s not.

I have spent nearly 2 days trying to get this simple task to work. I see in posts that I have searched that there are some feature flags to enable to get this to work, but from what I can tell they are no longer needed. I don’t have any sort of regex pattern setup as from my understanding that is only for printing the coverage info to stdout.

I’m sure I am overlooking something trivial and I really need a sanity check here as I have already spent way more time on this than I can afford.

Thanks in advance for any help!

Hi @Btripp1986 thanks for the post!

I just wanted to make sure you’re looking on the Changes Tab of the MR and the tests ran/cobertura.xml file was created as part of that MR’s latest pipeline? I should also call out there there have been fixed for automatic path correction in the 13.8 and 13.9 versions so if you are not running that on a self managed issue there could be problems with the paths.

Hopefully this helps!

-James H, GitLab Product Manager, Verify:Testing

1 Like

Thanks for the reply @jheimbuck_gl

I am in fact not using self-hosted gitlab, and I am looking at the changes tab of the MR. I double checked and the cobertura-coverage.xml is getting generated in that MR pipeline.

After looking through the link you provided on automatic path correction I think the cobertura-coverage.xml file might be the issue. Since we have a monorepo for this project the pipeline jobs are running on different directories of the repository (by that I mean the test pipeline has a job for running the tests in $PROJECT_ROOT/feature1, another job in $PROJECT_ROOT/feature2, etc. Because of this (and the fact that the test files are no in the root dir) the .xml generated shows the following

<source>/builds/mycompany/mygroup/myproject/feature1</source>
...
...
<class name="Banner.tsx" filename="src/components/Banner/Banner.tsx" line-rate="0.23079999999999998" branch-rate="0">

From my understanding of the documentation you linked, I think gitlab is expecting

<source>/builds/mycompany/mygroup/myproject</source>
...
...
<class name="Banner.tsx" filename="feature1/src/components/Banner/Banner.tsx" line-rate="0.23079999999999998" branch-rate="0">

I am going to attempt modifying these values in the job script to match the latter output. Please advise if you can think of anything else. Thank you for your help!

1 Like

@Btripp1986 I think you’re on the right track with that approach.

-James H, GitLab Product Manager, Verify:Testing

1 Like

@jheimbuck_gl I have added an after script containing the following

- sed -i 's;filename=";filename="feature1/;g'
- sed -i 's;myproject/feature1;myproject;g'

I have downloaded the report artifact and it looks right now

<source>/builds/mycompany/mygroup/myrepo</source>
...
...
<class name="ThemeProvider.tsx" filename="sdk/src/contexts/ThemeProvider.tsx" line-rate="1" branch-rate="1">

I am still not seeing code overage in the MR diff view, I am also still not seeing a code coverage % in the CI => Jobs page either.

If you have any ideas or a course of action you would recommend please let me know as I am running out of ideas. I was considering opening an issue, but I want to make 100% sure that this isn’t a user error before I do. Thank you for your guidance.

@Btripp1986 - Thanks for the update!

I’ll poke around a bit with the team to see what ideas they have about your latest .

For code coverage on the Jobs page you’ll need to parse the coverage which you can find information about in the documentation.

-James H, GitLab Product Manager, Verify:Testing

1 Like

Hi @Btripp1986! I noticed filename="sdk/src/contexts/ThemeProvider.tsx" from the sed modified XML. Just to confirm, is it correct that the full path (relative to the project root) of the class ThemeProvider.tsx, is sdk/src/contexts/ThemeProvider.tsx which includes sdk/?

-Erick B, Senior Backend Engineer, Verify:Testing

Thanks for looking at this.

Yes, the full path to ThemeProvider.tsx is <project-root>/sdk/src/contexts/ThemeProvider.tsx

After reading @jheimbuck_gl comment, I know I do not have the RegEx patter correct, so I assume that is the reason for coverage % not showing in the Jobs page. Would this affect the visualization in the MR diff, or is that feature unconcerned with the RegEx pattern?

No it won’t affect the MR diff visualization. They rely on 2 different artifacts.

When on the MR diff page, can you please check in your browser’s console if it made any requests to something like .../-/merge_requests/:mr_id/coverage_reports.json? And if it did, can you check if the JSON response contained any map of filenames?

Sorry for the late response @iamricecake

My team has changed its git strategy and we are now committing directly to the default branch, so coverage visualization is no longer required for us(no MRs).

I did get the coverage % to show up in the jobs page, which will be enough for our purposes.

If we change our git strategy again I will be implementing the coverage visualization, in which case I will refer back to this thread.

I thank you and James for your help. You folks are great and I preach your product to everyone I know.

1 Like