Hi,
I’ve tried to understand and reproduce the problem. The following content is the story from beginning to the end where I could make the linked project work, but only with additional modifications.
Project tests
I assume you want to run the linked project on your GitLab instance/account. I did the following to import the project into my GitLab.com account:
- Cloned the project locally
- Edited the Git configuration to update the remote origin to be my GitLab project
- Changed the default branch to
main locally
- Push the project - with HTTPS and PAT, this will automatically create a private project
- Update the project settings visibility to public
- Change the default branch to
main and protect it in the settings
git clone https://github.com/QianYizhou/gtest-cmake-gcov-example.git
cd gtest-cmake-gcov-example
vim .git/config
[remote "origin"]
url = https://gitlab.com/gitlab-de/use-cases/coverage-reports/gitlab-test-coverage-googletest-cmake-gcov.git
fetch = +refs/heads/*:refs/remotes/origin/*
git checkout -b main
git push
Local build
After that, I’ve looked into the CMake build steps, and replicated the tutorial. I have cmake/make/etc. installed using Homebrew on macOS.
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug -Dtest=ON
CMake Error at CMakeLists.txt:69 (add_subdirectory):
The source directory
/Users/mfriedrich/dev/test/gtest-cmake-gcov-example/gtest/googletest
does not contain a CMakeLists.txt file.
Problem is that the source code is modified and points to gtest/googletest that is not a valid Git submodule anymore. Not sure what got broken here, but my fixes are done like this:
cd gtest
git rm -r googletest
git commit -av -m "Purge broken googletest directory"
git submodule add https://github.com/google/googletest.git googletest
git commit -av -m "Add google/googletest as submodule" ─╯
[master 73903e8] Add google/googletest as submodule
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 gtest/googletest
git submodule init
git submodule update
Next error, did not have lcov installed.
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug .. -Dtest=ON
CMake Error at cmake/CodeCoverage.cmake:98 (MESSAGE):
lcov not found! Aborting...
Call Stack (most recent call first):
test/testfoo/CMakeLists.txt:20 (SETUP_TARGET_FOR_COVERAGE)
Fixing this with brew install lcov. Taking a note that this needs to be installed in CI/CD builds too.
Trying again.
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug -Dtest=ON
cd ..
make -C build
Looks good, it builds.
There are no tests yet …
make test -C build ─╯
Running tests...
Test project /Users/mfriedrich/dev/test/gtest-cmake-gcov-example/build
No tests were found!!!
Needed lots of trial and error because the README of the project is not correct. Figured that the remaining existing test case is testFoo.
make coverage_testFoo -C build
ll build/coverage_testFoo_dir
It generates some coverage report as HTML.
Coverage report in GitLab CI/CD
In order to integrate into GitLab CI/CD, the coverage report needs to be provided as Cobertura XML report, Test coverage visualization | GitLab
I’ve googled for gitlab cicd googletest coverage report and found a better tutorial in Generating Code Coverage Report Using GNU Gcov & Lcov. | by Naveen Kashyap | Medium which also explains how gcov and lcov are being invoked.
The GitLab documentation provides an example for C/C++ Cobertura reports. Combining this knowledge, let’s see whether the example project uses gcovr already, similar to the GitLab docs example.
╰─ grep -r gcovr . ─╯
./cmake/CodeCoverage.cmake:FIND_PROGRAM( GCOVR_PATH gcovr PATHS ${CMAKE_SOURCE_DIR}/tests)
./cmake/CodeCoverage.cmake: MESSAGE(FATAL_ERROR "gcovr not found! Aborting...")
./cmake/CodeCoverage.cmake: # Running gcovr
./cmake/CodeCoverage.cmake: COMMENT "Running gcovr to produce Cobertura code coverage report."
But it isn’t called anywhere, so there is no gcovr being installed yet.
brew install gcovr
The function is prepared, similar to the coverage function. I’ve compare the parameters and invokes for both CMake functions, and added a second make target into CMakeLists.txt.
I’ve had to patch the function to not check for Python (rconv packages requires this explicitly on macOS and also Debian/Ubuntu), and then integrate it as CMakeLists.txt target into test/testFoo/CMakeLists.txt
vim cmake/CodeCoverage.cmake
# Param _targetname The name of new the custom make target
# Param _testrunner The name of the target which runs the tests
# Param _outputname cobertura output is generated as _outputname.xml
# Optional fourth parameter is passed as arguments to _testrunner
# Pass them in list form, e.g.: "-j;2" for -j 2
FUNCTION(SETUP_TARGET_FOR_COVERAGE_COBERTURA _targetname _testrunner _outputname)
IF(NOT GCOVR_PATH)
MESSAGE(FATAL_ERROR "gcovr not found! Aborting...")
ENDIF() # NOT GCOVR_PATH
ADD_CUSTOM_TARGET(${_targetname}
# Run tests
${_testrunner} ${ARGV3}
# Running gcovr
COMMAND ${GCOVR_PATH} -x -r ${CMAKE_SOURCE_DIR} -e '${CMAKE_SOURCE_DIR}/tests/'-e'${CMAKE_SOURCE_DIR}/build/' -o ${_outputname}.xml
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
COMMENT "Running gcovr to produce Cobertura code coverage report."
)
# Show info where to find the report
ADD_CUSTOM_COMMAND(TARGET ${_targetname} POST_BUILD
COMMAND ;
COMMENT "Cobertura code coverage report saved in ${_outputname}.xml."
)
ENDFUNCTION() # SETUP_TARGET_FOR_COVERAGE_COBERTURA
vim test/testFoo/CMakeLists.txt
SETUP_TARGET_FOR_COVERAGE_COBERTURA(
coverage_cobertura_${BIN_NAME} # Name for custom target.
${BIN_NAME} # Name of the test driver executable that runs the tests.
# NOTE! This should always have a ZERO as exit code
# otherwise the coverage generation will not complete.
coverage_cobertura_${BIN_NAME} # Name of output file.
)
cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug -Dtest=ON
make coverage_cobertura_testFoo
This results in a new cobertura report file which should match what GitLab CI/CD expects.
build/coverage_cobertura_testFoo.xml
Ok, finally. Let’s try to model the learnt knowledge into GitLab CI/CD jobs and execution steps.
GitLab CI/CD config
I did it in multiple steps:
- Define the build steps to get a coverage report from the source in a new job and
script section
- Research how the Cobertura reports are configured and add the
reports section to the job
- Add a build image,
debian:bullseye-slim is a small Debian image
- Use
before_script to install the build tools
- Install Git and initialize the submodule
You can follow the learning curve in this MR: Add GitLab CI/CD config for Cobertura Coverage reports (!1) · Merge requests · Developer Evangelism at GitLab / use-cases / Coverage Reports / gitlab-test-coverage-googletest-cmake-gcov · GitLab
variables:
MAKE_TARGET: coverage_cobertura_testFoo
gtest-cobertura:
image: debian:bullseye-slim
before_script:
- apt update && apt install -y git cmake g++ make gcovr lcov
- git submodule init && git submodule update
script:
- mkdir build && cd build
- cmake .. -DCMAKE_BUILD_TYPE=Debug .. -Dtest=ON
- make $MAKE_TARGET
coverage: /^\s*lines:\s*\d+.\d+\%/
artifacts:
name: ${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHA}
expire_in: 2 days
reports:
coverage_report:
coverage_format: cobertura
path: build/$MAKE_TARGET.xml
# https://docs.gitlab.com/ee/ci/testing/test_coverage_visualization.html#cc-example
The MR ran successfully.
Test Coverage Visualization
To visualize the test coverage in libfoo/foo.cpp, I’ve made some edits there, updating the style and pushing to the same MR.
Conclusion
Using GoogleTest in a C++ together with Cobertura test coverage reports in GitLab works. The example project provides many useful resources but is a bit outdated, and CMake adds more complexity to it than needed.
The project fork in Developer Evangelism at GitLab / use-cases / Coverage Reports / gitlab-test-coverage-googletest-cmake-gcov · GitLab got updated documentation on how to run and use, including GitLab CI/CD configuration. CMake and C++ knowledge is required - one of my past OSS projects allowed me to learn it for many years.
Hope this learning journey helps to get you started, I enjoyed learning new features in GitLab and also how Google Test and CMake work together.
Cheers,
Michael
*
