Local machine to gitlab code coverage using gcov/lcov and gtest

Replace this template with your information

*I have a C++ project set up in gitlab. It will support both windows and linux. I am right now exploring the code coverage of this C++ project. To try, I am exploring GCOV/LCOV and Google test using GCC compiler. I used GitHub - QianYizhou/gtest-cmake-gcov-example: A sample project using GoogleTest with CMake as the tutorial. I installed Ubuntu virtual machine, mounted my gitlab clone project from Windows to Linux. I installed Google test, LCOV in Ubuntu, did generate the code coverage report as per that tutorial.

So it proved in my local virtual machine. I tried to integrate into the pipeline. In my local machine, the cmake files were generated to a build folder. But in my pipeline, I am not sure where to go and give following command in the yml file

cd build && make test
cd build && make coverage_TEST_NAME //To check the coverage

Thanks for taking the time to be thorough in your request, it really helps! :blush:*


I’ve tried to understand and reproduce the problem. The following content is the story from beginning to the end where I could make the linked project work, but only with additional modifications.

Project tests

I assume you want to run the linked project on your GitLab instance/account. I did the following to import the project into my GitLab.com account:

  1. Cloned the project locally
  2. Edited the Git configuration to update the remote origin to be my GitLab project
  3. Changed the default branch to main locally
  4. Push the project - with HTTPS and PAT, this will automatically create a private project
  5. Update the project settings visibility to public
  6. Change the default branch to main and protect it in the settings
git clone https://github.com/QianYizhou/gtest-cmake-gcov-example.git

cd gtest-cmake-gcov-example

vim .git/config

[remote "origin"]
        url = https://gitlab.com/gitlab-de/use-cases/coverage-reports/gitlab-test-coverage-googletest-cmake-gcov.git
        fetch = +refs/heads/*:refs/remotes/origin/*

git checkout -b main
git push 

Local build

After that, I’ve looked into the CMake build steps, and replicated the tutorial. I have cmake/make/etc. installed using Homebrew on macOS.

mkdir build
cd build 
cmake .. -DCMAKE_BUILD_TYPE=Debug  -Dtest=ON

CMake Error at CMakeLists.txt:69 (add_subdirectory):
  The source directory


  does not contain a CMakeLists.txt file.

Problem is that the source code is modified and points to gtest/googletest that is not a valid Git submodule anymore. Not sure what got broken here, but my fixes are done like this:

cd gtest
git rm -r googletest
git commit -av -m "Purge broken googletest directory"

git submodule add https://github.com/google/googletest.git googletest

git commit -av -m "Add google/googletest as submodule"                             ─╯
[master 73903e8] Add google/googletest as submodule
 2 files changed, 4 insertions(+)
 create mode 100644 .gitmodules
 create mode 160000 gtest/googletest

git submodule init
git submodule update 

Next error, did not have lcov installed.

mkdir build
cd build 
cmake .. -DCMAKE_BUILD_TYPE=Debug .. -Dtest=ON

CMake Error at cmake/CodeCoverage.cmake:98 (MESSAGE):
  lcov not found! Aborting...
Call Stack (most recent call first):
  test/testfoo/CMakeLists.txt:20 (SETUP_TARGET_FOR_COVERAGE)

Fixing this with brew install lcov. Taking a note that this needs to be installed in CI/CD builds too.

Trying again.

mkdir build
cd build 
cmake .. -DCMAKE_BUILD_TYPE=Debug -Dtest=ON
cd ..

make -C build

Looks good, it builds.

There are no tests yet …

make test -C build                                                                 ─╯
Running tests...
Test project /Users/mfriedrich/dev/test/gtest-cmake-gcov-example/build
No tests were found!!!

Needed lots of trial and error because the README of the project is not correct. Figured that the remaining existing test case is testFoo.

make coverage_testFoo -C build

ll build/coverage_testFoo_dir

It generates some coverage report as HTML.

Coverage report in GitLab CI/CD

In order to integrate into GitLab CI/CD, the coverage report needs to be provided as Cobertura XML report, Test coverage visualization | GitLab

I’ve googled for gitlab cicd googletest coverage report and found a better tutorial in Generating Code Coverage Report Using GNU Gcov & Lcov. | by Naveen Kashyap | Medium which also explains how gcov and lcov are being invoked.

The GitLab documentation provides an example for C/C++ Cobertura reports. Combining this knowledge, let’s see whether the example project uses gcovr already, similar to the GitLab docs example.

╰─ grep -r gcovr .                                                                    ─╯
./cmake/CodeCoverage.cmake:FIND_PROGRAM( GCOVR_PATH gcovr PATHS ${CMAKE_SOURCE_DIR}/tests)
./cmake/CodeCoverage.cmake:		MESSAGE(FATAL_ERROR "gcovr not found! Aborting...")
./cmake/CodeCoverage.cmake:		# Running gcovr
./cmake/CodeCoverage.cmake:		COMMENT "Running gcovr to produce Cobertura code coverage report."

But it isn’t called anywhere, so there is no gcovr being installed yet.

brew install gcovr

The function is prepared, similar to the coverage function. I’ve compare the parameters and invokes for both CMake functions, and added a second make target into CMakeLists.txt.

I’ve had to patch the function to not check for Python (rconv packages requires this explicitly on macOS and also Debian/Ubuntu), and then integrate it as CMakeLists.txt target into test/testFoo/CMakeLists.txt

vim cmake/CodeCoverage.cmake

# Param _targetname     The name of new the custom make target
# Param _testrunner     The name of the target which runs the tests
# Param _outputname     cobertura output is generated as _outputname.xml
# Optional fourth parameter is passed as arguments to _testrunner
#   Pass them in list form, e.g.: "-j;2" for -j 2
FUNCTION(SETUP_TARGET_FOR_COVERAGE_COBERTURA _targetname _testrunner _outputname)

                MESSAGE(FATAL_ERROR "gcovr not found! Aborting...")


                # Run tests
                ${_testrunner} ${ARGV3}

                # Running gcovr
                COMMAND ${GCOVR_PATH} -x -r ${CMAKE_SOURCE_DIR} -e '${CMAKE_SOURCE_DIR}/tests/'-e'${CMAKE_SOURCE_DIR}/build/'  -o ${_outputname}.xml
                COMMENT "Running gcovr to produce Cobertura code coverage report."

        # Show info where to find the report
                COMMAND ;
                COMMENT "Cobertura code coverage report saved in ${_outputname}.xml."

vim test/testFoo/CMakeLists.txt 

    coverage_cobertura_${BIN_NAME}  # Name for custom target.
    ${BIN_NAME}         # Name of the test driver executable that runs the tests.
    # NOTE! This should always have a ZERO as exit code
    # otherwise the coverage generation will not complete.
    coverage_cobertura_${BIN_NAME}            # Name of output file.
cd build 
cmake .. -DCMAKE_BUILD_TYPE=Debug -Dtest=ON

make coverage_cobertura_testFoo

This results in a new cobertura report file which should match what GitLab CI/CD expects.


Ok, finally. Let’s try to model the learnt knowledge into GitLab CI/CD jobs and execution steps.

GitLab CI/CD config

I did it in multiple steps:

  1. Define the build steps to get a coverage report from the source in a new job and script section
  2. Research how the Cobertura reports are configured and add the reports section to the job
  3. Add a build image, debian:bullseye-slim is a small Debian image
  4. Use before_script to install the build tools
  5. Install Git and initialize the submodule

You can follow the learning curve in this MR: Add GitLab CI/CD config for Cobertura Coverage reports (!1) · Merge requests · Developer Evangelism at GitLab / use-cases / Coverage Reports / gitlab-test-coverage-googletest-cmake-gcov · GitLab

  MAKE_TARGET: coverage_cobertura_testFoo

  image: debian:bullseye-slim
    - apt update && apt install -y git cmake g++ make gcovr lcov
    - git submodule init && git submodule update 
    - mkdir build && cd build
    - cmake .. -DCMAKE_BUILD_TYPE=Debug .. -Dtest=ON
    - make $MAKE_TARGET
  coverage: /^\s*lines:\s*\d+.\d+\%/
    expire_in: 2 days
        coverage_format: cobertura
        path: build/$MAKE_TARGET.xml

# https://docs.gitlab.com/ee/ci/testing/test_coverage_visualization.html#cc-example

The MR ran successfully.

Test Coverage Visualization

To visualize the test coverage in libfoo/foo.cpp, I’ve made some edits there, updating the style and pushing to the same MR.


Using GoogleTest in a C++ together with Cobertura test coverage reports in GitLab works. The example project provides many useful resources but is a bit outdated, and CMake adds more complexity to it than needed.

The project fork in Developer Evangelism at GitLab / use-cases / Coverage Reports / gitlab-test-coverage-googletest-cmake-gcov · GitLab got updated documentation on how to run and use, including GitLab CI/CD configuration. CMake and C++ knowledge is required - one of my past OSS projects allowed me to learn it for many years.

Hope this learning journey helps to get you started, I enjoyed learning new features in GitLab and also how Google Test and CMake work together.