Hello @jheimbuck_gl sorry if I didn’t explained well, my English can make things difficult haha
The quality report on the pipeline page is working. The problem is with the widget on the MR page.
Quick answer to your question is: nothing that depends on the latest pipeline in the master branch to use as baseline is working:
- code quality widget on the MR page
- coverage difference on the MR page
- pipeline badge used on main page of the project (it will be always as unknown after maven-release-plugin).
I will try to explain the whole context because maybe it will trigger some insights on your side.
Its a pipeline with build, test, quality and release stages.
Global Cache
cache:
paths:
- .m2/repository
Global Variables
variables:
MAVEN_CLI_OPTS: "-s settings.xml --batch-mode --errors --show-version"
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true -Dgit-link=$CI_PROJECT_URL"
Build Stage
- build job that runs only for MRs.
- This is irrelevant since it doesn’t produces any artifacts. I’m performing my tests with this job commented.
Test Stage
- test job that runs in MRs and master.
.jacoco-coverage: &jacoco-coverage |
if [ $JACOCO_ENABLED == 'true' ]; then
JACOCO_COVERAGE=$(awk -F"," '{ instructions += $4 + $5; covered += $5; line_condition_cov += $7 + $9; line_condition_total += $6 + $7 + $8 + $9 } END { print covered, "/", instructions, "bytecode instructions covered"; print 100*(line_condition_cov/line_condition_total), "% covered" }' $JACOCO_PATH)
echo $JACOCO_COVERAGE > coverage.txt
cat coverage.txt
fi
.set-jacoco-path: &set-jacoco-path
- finalJaCoCoPath=$JACOCO_PATH
- finalJaCoCoPath=${finalJaCoCoPath/\.csv/.xml}
- echo Using report file for JaCoCo located in $finalJaCoCoPath
test:
stage: test
needs: []
extends:
- .default-before-script # This only configures maven settings.xml and basic configurations
- .tests-tags # This adds tags to run in the correct runner
- .except-ci-user # This is to skip the job from running in our GitLab CI User (using except.variables for the User ID)
- .mvn-cleanup # This is to cleanup maven environment removing SNAPSHOT artifacts before being cached.
coverage: /\d+.?\d* \% covered/
script:
- su $TEST_EXECUTION_USER -c "mvn $MAVEN_CLI_OPTS -U clean verify"
- *jacoco-coverage
- *set-jacoco-path
- cp $finalJaCoCoPath ./ 2>/dev/null # This will write the jacoco.xml file (will be converted in the coverage-report job).
only:
refs:
- master
- merge_requests
artifacts:
paths:
- jacoco.xml
- coverage.txt
reports:
junit:
- "**/target/surefire-reports/TEST-*.xml"
- Produces junit report, a jacoco.xml artifact and a coverage.txt artifact.
Quality Stage
- coverage-report job that runs in MRs and master to convert jacoco.xml artifact into a valid cobertura.xml artifact:
- I added this yesterday and its not working right now. It produces the corrrect covertura.xml but I can’t see the coverage on the MR diff page. I will show its configuration but let’s ignore this for now and focus on quality widget and coverage difference.
coverage-report:
stage: quality
image: haynes/jacoco2cobertura:1.0.4
allow_failure: true
script:
# convert report from jacoco to cobertura
- 'python /opt/cover2cover.py jacoco.xml src/main/java > cobertura.xml'
# read the <source></source> tag and prepend the path to every filename attribute
- 'python /opt/source2filename.py cobertura.xml'
needs: ["test"]
extends:
- .except-ci-user
dependencies:
- test
artifacts:
reports:
cobertura: cobertura.xml
paths:
- cobertura.xml
only:
refs:
- master
- merge_requests
variables:
- $JACOCO_ENABLED == "true"
- code_quality job that runs only for MRs and master branch. This runs sonar scanner and creates the codeclimate file as gl-code-quality-report.json artifact:
variables:
SONAR_QUBE_ENABLED: "true"
SONAR_QUBE_QUERY_WAIT: 1000
SONAR_QUBE_MAX_RETRY: 150
.load-jacoco-path: &load-jacoco-path
- finalJaCoCoPath=$CI_PROJECT_DIR/$JACOCO_PATH
- finalJaCoCoPath=${finalJaCoCoPath/\.csv/.xml}
- echo Using report file for JaCoCo located in $finalJaCoCoPath
.run-for-master: &run-for-master |
if [[ $CI_COMMIT_BRANCH == 'master' ]]; then
su $TEST_EXECUTION_USER -c "mvn $MAVEN_CLI_OPTS verify sonar:sonar <parameters>"
fi
.run-for-features: &run-for-features |
if [[ $CI_COMMIT_BRANCH != 'master' ]]; then
su $TEST_EXECUTION_USER -c "mvn $MAVEN_CLI_OPTS verify sonar:sonar <parameters>"
fi
code_quality:
stage: quality
needs: []
variables:
GIT_DEPTH: 0
cache:
key: "${CI_JOB_NAME}"
when: 'always'
paths:
- .sonar/cache
- .m2/repository
extends:
- .default-before-script
- .tests-tags
allow_failure: true
script:
- *load-jacoco-path
- *run-for-features
- *run-for-master
after_script:
- mv codeclimate.json gl-code-quality-report.json 2>/dev/null # The plugin we use is generating codeclimate.json instead of gitlab's file pattern.
artifacts:
reports:
codequality: gl-code-quality-report.json
paths:
- gl-code-quality-report.json
rules:
- if: '$SONAR_QUBE_ENABLED != "true"'
when: never
- if: '$CODE_QUALITY_DISABLED'
when: never
- if: '$GITLAB_USER_ID == "9" || $GITLAB_USER_EMAIL == "<REDACTED>"'
when: never
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH == "master"'
- when: never
It generates the gl-code-quality-report.json file.
Release Stage
- release job that runs only for master.
This job performs the maven-release-plugin and creates 2 commits that doesn’t runs any job. Commits are generated using the user with User Id 9. It’s ignored by rules in other stages or by the extension of .except-ci-user.
The widget only works if I comment this job and then commits on master, wait for artifact generation and then create another MR. If master runs with this job it will have 2 automatic commits without running any jobs.
This job needs to be divided into two jobs since its performing the release and create images. I create image job creation that runs only for tags but this doesn’t matter for this current issue, I think.
release:
stage: release
needs: ["test"]
extends:
- .default-before-script
- .deploys-tags
- .only-master-except-schedules
- .except-ci-user
- .mvn-cleanup
script:
- su $TEST_EXECUTION_USER -c "git checkout -B $CI_BUILD_REF_NAME"
- *upgrade-versions # Automatic bumps version based on conventional commits.
- *update-snapshots # Remove snapshots and convert to stable versions before the release.
- su $TEST_EXECUTION_USER -c "mvn $MAVEN_CLI_OPTS clean release:prepare -Darguments=\"-Dmaven.test.skip=true\" -DautoVersionSubmodules=true -DscmCommentPrefix=\"[maven-release-plugin] [skip_ci] \" -Dusername=$USERNAME -Dpassword=$USER_TOKEN"
- *jacoco-coverage
- VERSION_TAG=$(cat release.properties | grep "scm.tag=" | awk '{gsub("scm.tag=", ""); print}')
- git checkout $VERSION_TAG
- mvn $MAVEN_CLI_OPTS clean install -DskipTests
- *staging-image # Produces staging docker image
- *production-image # Produces production docker image
- mvn $MAVEN_CLI_OPTS clean release:perform -DscmCommentPrefix="[maven-release-plugin] [skip_ci] " -Darguments="-Dmaven.test.skip=true" -Dusername=$USERNAME -Dpassword=$USER_TOKEN
Why I’m using only/except: I build this pipelline years ago and I didn’t had the chance to upgrade every restriction to use the rules feature. The only one I updated was the code_quality since I’m trying to make it works.
I believe this is enough information so you can help me, because I spend last 48 hours trying to manage this pipeline to work but the widgets are not working 
Gitlab version: GitLab Enterprise Edition 13.8.2-ee
Gitlab plan: Starter.
Thank you!
