You can specify one or more coverage reports to collect, including wildcard paths. GitLab then takes the coverage information in all the files and combines it together.
My pipeline consist of the following stages:
On the 'unit tests’ stage I have a job to expose a code coverage report (R1) and on the ‘integration tests’ stage I also have a job with a report (R2).
And it seems in an MR the only R1 is shown. When I expose R1+R2 from the same job on the ‘integration tests’ stage I see a merged result in an MR. But when reports come from different stages I see only the 1st one (R1).
Is that expected?.. Any way to have a merged result from different stages?
The thing is that ‘integration tests’ stage runs not always in my pipeline, so I can’t expose both the reports from the later stage.
Please show your .gitlab-ci.yml (at least the parts dealing with the coverage reports), it’s much easier to discuss actual code than it is to deal in abstracts.
Things I’d check:
When both reports run, do they have the same structure? That is, do the paths all match up exactly so that the reports can be combined?
When both reports run, do their outputs have differentfilenames, or is it possible one report is overwriting the other? Since the reports are going to be combined, I don’t think it’s safe to assume they’re kept separate by the analysis tool.
Despite the text you quoted, all of the examples on the page you referenced have just a single stage exposing the coverage report, with reports from previous stages being combined within the CI job(s) themselves. So it’s possible the “combine reports from multiple stages” functionality is either broken or was never implemented.
You could probably work around it yourself, though, by adding a final R3 stage that takes the report from R1, adds in the report from R2 iff it was run, and then exports the results as the final artifacts:reports:cobertura output. The Maven example looks like a pretty good template for that.
There are several open or incomplete issues/merges in the GitLab tracker that make me think that text about combining reports from multiple stages may be a bit optimistic. Combining them yourself and exporting only a single artifacts:reports:cobertura may be the best solution.
Make sure that the report-combining stage depends: on the prior ones, so that it’ll wait for them to finish before assembling the final report. (You’ll have to look up how to do that correctly when some prior stages may be optional, I’m not sure offhand.)
Filenames are different, both the reports have the same structure. If I remove R1 stage, the R2 is displayed correctly. But once R1 is in place - R2 is not taken into the account.
Unfortunately, R3 workaround doesn’t look like a solution in my case. ‘Integration tests’ job is manual. And I would like to see a coverage report in an MR for either R1 only (in case R2 is not run at all) or for both the R1+R2 (in case int tests were run). So the R3, in that case, would depend on R1 & R2, and would never be run if R2 is also not run.
Anyway, let me check the reports structure again and issues/merges you’ve mentioned. Will update the thread in case I find anything new. Thanks
Yeah, I read over the code for processing coverage reports (it’s in ci/parsers/coverage/cobertura.rb), and I no longer think that would matter. It seems to reference the files only as abstract blobs, I don’t even think they have filenames anymore by the time they’ve gotten to the parser.
It still isn’t clear to me where that parser would be run on more than one file, though. Admittedly, it’s not clear to me that it’s definitely not, either. But the code there certainly appears to only process a single report input. If it’s meant to run multiple times and be aggregated, that code is clearly elsewhere.
I think that should be workable, personally. Here’s what you could probably do.
Since integration_tests are manual, the report assembly clearly can’t be part of that stage exclusively; you want it to run even when they’re not. So you’ll need another stage for the report processing. The examples call it visualization, so why not? (They also mention that’s a non-default stage that has to be defined, though, so keep that in mind.)