Gitlab Code Quality

@jheimbuck_gl hello, how are you?

Hope you are in the best health.

Sorry if this not the right thing to do, this the first time using gitlab forum and opening a new issues sound unnecessary for me.

I’m using codeclimate report in MR and I got it working.

The issue is that my CI pipeline do 2 commits in the master branch without running any pipeline (all jobs are blocked to run for the CI user that committed twice.

I was expecting the code report in MR to show for all MRs but I got it working only if my CI user didn’t commit in the master. This is expected? The baseline is based on last commit on master (with liked jobs or not) or based on last successful job that contains a quality artifact?

Thanks.

@lucasoares3 thanks for posting!

The MR widget does require that reports are available for both the source and target pipelines to display. It sounds like there maybe no report available to do the comparison in your case.

I hope this helps!

-James H, GitLab Product Manager, Verify:Testing

May you check here and see my issue then?

There is a baseline report.

Thank you very much.

@lucasoares3 Can you share the contents of your .gitlab-ci.yml file here?

-James H, GitLab Product Manager, Verify:Testing

I tried to change the prefix for maven release plugin to include [skip_ci] in the commit message hoping that GitLab status + coverage + code quality report should ignore the commit and this is what I got:

The skipped pipeline is getting the latest badge. And no feature that requires a baseline will work. You can see by my screenshot that master pipeline has all necessary baseline reports in the last successful pipeline. I even exported them as both report and paths to be able to confirm if the content is valid.

@jheimbuck_gl I will look tomorrow how to share my gitlab-ci contents because its large and has many included files that I need to revise and maybe redact. I will get the final CI generated by the gitlab lint page tomorrow.

But the issue is: my gitlab-ci file works fine. I removed the maven release plugin just to check and then coverage, pipeline status badge, everything works. As soon as I activate the deploy job that runs the maven release plugin, everything stops working.

@lucasoares3 don’t worry about the .gitlab-ci.yml for now.

I do not think I am following your use case so i’m having trouble understanding the problem.

It sounds like you have a pipeline that runs and produces artifacts so you can see the Code Quality report on the pipeline page like you can on this project. It sounds like that report and the widget on the MR page (as you can see here) stop working when you disable a release plugin.

Is this correct? Thanks!

-James H, GitLab Product Manager, Verify:Testing

1 Like

Hello @jheimbuck_gl sorry if I didn’t explained well, my English can make things difficult haha

The quality report on the pipeline page is working. The problem is with the widget on the MR page.

Quick answer to your question is: nothing that depends on the latest pipeline in the master branch to use as baseline is working:

  • code quality widget on the MR page
  • coverage difference on the MR page
  • pipeline badge used on main page of the project (it will be always as unknown after maven-release-plugin).

I will try to explain the whole context because maybe it will trigger some insights on your side.

Its a pipeline with build, test, quality and release stages.


Global Cache

cache:
  paths:
    - .m2/repository

Global Variables

variables:
  MAVEN_CLI_OPTS: "-s settings.xml --batch-mode --errors --show-version"
  MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true -Dgit-link=$CI_PROJECT_URL"

Build Stage

  • build job that runs only for MRs.
    • This is irrelevant since it doesn’t produces any artifacts. I’m performing my tests with this job commented.

Test Stage

  • test job that runs in MRs and master.
.jacoco-coverage: &jacoco-coverage |
  if [ $JACOCO_ENABLED == 'true' ]; then
    JACOCO_COVERAGE=$(awk -F"," '{ instructions += $4 + $5; covered += $5; line_condition_cov += $7 + $9; line_condition_total += $6 + $7 + $8 + $9 } END { print covered, "/", instructions, "bytecode instructions covered"; print 100*(line_condition_cov/line_condition_total), "% covered" }' $JACOCO_PATH)
    echo $JACOCO_COVERAGE > coverage.txt
    cat coverage.txt
  fi

.set-jacoco-path: &set-jacoco-path
  - finalJaCoCoPath=$JACOCO_PATH
  - finalJaCoCoPath=${finalJaCoCoPath/\.csv/.xml}
  - echo Using report file for JaCoCo located in $finalJaCoCoPath

test:
  stage: test
  needs: []
  extends:
    - .default-before-script			# This only configures maven settings.xml and basic configurations
    - .tests-tags						# This adds tags to run in the correct runner
    - .except-ci-user					# This is to skip the job from running in our GitLab CI User  (using except.variables for the User ID)
    - .mvn-cleanup						# This is to cleanup maven environment removing SNAPSHOT artifacts before being cached.
  coverage: /\d+.?\d* \% covered/
  script:
    - su $TEST_EXECUTION_USER -c "mvn $MAVEN_CLI_OPTS -U clean verify"
    - *jacoco-coverage
    - *set-jacoco-path
    - cp $finalJaCoCoPath ./ 2>/dev/null		# This will write the jacoco.xml file (will be converted in the coverage-report job).
  only:
    refs:
      - master
      - merge_requests
  artifacts:
    paths:
      - jacoco.xml
      - coverage.txt
    reports:
      junit:
        - "**/target/surefire-reports/TEST-*.xml"
  • Produces junit report, a jacoco.xml artifact and a coverage.txt artifact.

Quality Stage

  • coverage-report job that runs in MRs and master to convert jacoco.xml artifact into a valid cobertura.xml artifact:
    • I added this yesterday and its not working right now. It produces the corrrect covertura.xml but I can’t see the coverage on the MR diff page. I will show its configuration but let’s ignore this for now and focus on quality widget and coverage difference.
coverage-report:
  stage: quality
  image: haynes/jacoco2cobertura:1.0.4
  allow_failure: true
  script:
    # convert report from jacoco to cobertura
    - 'python /opt/cover2cover.py jacoco.xml src/main/java > cobertura.xml'
    # read the <source></source> tag and prepend the path to every filename attribute
    - 'python /opt/source2filename.py cobertura.xml'
  needs: ["test"]
  extends:
    - .except-ci-user
  dependencies:
    - test
  artifacts:
    reports:
      cobertura: cobertura.xml
    paths:
      - cobertura.xml
  only:
    refs:
      - master
      - merge_requests
    variables:
      - $JACOCO_ENABLED == "true"
  • code_quality job that runs only for MRs and master branch. This runs sonar scanner and creates the codeclimate file as gl-code-quality-report.json artifact:
variables:
  SONAR_QUBE_ENABLED: "true"
  SONAR_QUBE_QUERY_WAIT: 1000
  SONAR_QUBE_MAX_RETRY: 150

.load-jacoco-path: &load-jacoco-path
  - finalJaCoCoPath=$CI_PROJECT_DIR/$JACOCO_PATH
  - finalJaCoCoPath=${finalJaCoCoPath/\.csv/.xml}
  - echo Using report file for JaCoCo located in $finalJaCoCoPath

.run-for-master: &run-for-master |
  if [[ $CI_COMMIT_BRANCH == 'master' ]]; then
    su $TEST_EXECUTION_USER -c "mvn $MAVEN_CLI_OPTS verify sonar:sonar <parameters>"
  fi

.run-for-features: &run-for-features |
  if [[ $CI_COMMIT_BRANCH != 'master' ]]; then
    su $TEST_EXECUTION_USER -c "mvn $MAVEN_CLI_OPTS verify sonar:sonar <parameters>"
  fi

code_quality:
  stage: quality
  needs: []
  variables:
    GIT_DEPTH: 0
  cache:
    key: "${CI_JOB_NAME}"
    when: 'always'
    paths:
      - .sonar/cache
      - .m2/repository
  extends:
    - .default-before-script
    - .tests-tags
  allow_failure: true
  script:
    - *load-jacoco-path
    - *run-for-features
    - *run-for-master
  after_script:
    - mv codeclimate.json gl-code-quality-report.json 2>/dev/null			# The plugin we use is generating codeclimate.json instead of gitlab's file pattern.
  artifacts:
    reports:
      codequality: gl-code-quality-report.json
    paths:
      - gl-code-quality-report.json
  rules:
    - if: '$SONAR_QUBE_ENABLED != "true"'
      when: never
    - if: '$CODE_QUALITY_DISABLED'
      when: never
    - if: '$GITLAB_USER_ID == "9" || $GITLAB_USER_EMAIL == "<REDACTED>"'
      when: never
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "master"'
    - when: never

It generates the gl-code-quality-report.json file.


Release Stage

  • release job that runs only for master.

This job performs the maven-release-plugin and creates 2 commits that doesn’t runs any job. Commits are generated using the user with User Id 9. It’s ignored by rules in other stages or by the extension of .except-ci-user.

The widget only works if I comment this job and then commits on master, wait for artifact generation and then create another MR. If master runs with this job it will have 2 automatic commits without running any jobs.

This job needs to be divided into two jobs since its performing the release and create images. I create image job creation that runs only for tags but this doesn’t matter for this current issue, I think.

release:
  stage: release
  needs: ["test"]
  extends:
    - .default-before-script
    - .deploys-tags
    - .only-master-except-schedules
    - .except-ci-user
    - .mvn-cleanup
  script:
    - su $TEST_EXECUTION_USER -c "git checkout -B $CI_BUILD_REF_NAME"
    - *upgrade-versions			# Automatic bumps version based on conventional commits.
    - *update-snapshots			# Remove snapshots and convert to stable versions before the release.
    - su $TEST_EXECUTION_USER -c "mvn $MAVEN_CLI_OPTS clean release:prepare -Darguments=\"-Dmaven.test.skip=true\" -DautoVersionSubmodules=true -DscmCommentPrefix=\"[maven-release-plugin] [skip_ci] \" -Dusername=$USERNAME -Dpassword=$USER_TOKEN"
    - *jacoco-coverage
    - VERSION_TAG=$(cat release.properties | grep "scm.tag=" | awk '{gsub("scm.tag=", ""); print}')
    - git checkout $VERSION_TAG
    - mvn $MAVEN_CLI_OPTS clean install -DskipTests
    - *staging-image			# Produces staging docker image
    - *production-image			# Produces production docker image
    - mvn $MAVEN_CLI_OPTS clean release:perform -DscmCommentPrefix="[maven-release-plugin] [skip_ci] " -Darguments="-Dmaven.test.skip=true" -Dusername=$USERNAME -Dpassword=$USER_TOKEN

Why I’m using only/except: I build this pipelline years ago and I didn’t had the chance to upgrade every restriction to use the rules feature. The only one I updated was the code_quality since I’m trying to make it works.


I believe this is enough information so you can help me, because I spend last 48 hours trying to manage this pipeline to work but the widgets are not working :frowning:

Gitlab version: GitLab Enterprise Edition 13.8.2-ee
Gitlab plan: Starter.

Thank you!

@lucasoares3 thanks for the detailed explanation and data! I’m not super familiar with the rules syntax but shared this with the team who is taking a look.

In the meantime can you share if you see the code quality tab on the latest successful pipeline for the /master branch and if not is there an error in the job logs you can share?

Thanks!

-James H, GitLab Product Manager, Verify:Testing

@lucasoares3 it does look like you have this setup so commits on Master do not get the code quality job ran. What you’re describing is pretty consistent with the most common case of the base-report-missing situation.

Hopefully this helps!

-James H, GitLab Product Manager, Verify:Testing

Hello @jheimbuck_gl

Sorry but I didn’t understood what you said. What you mean abou this setup? Commits on master is running the code quality and latest successful pipeline for master branch have the code quality artifact (as shown in previous pictures).

I just tested again with a different version of this same pipeline. This version doesn’t uses detached pipelines (pipelines always runs for commits and not for merge request event). It’s also not working. Just look at the pictures.

Last master pipeline:

Last branch pipeline (commit 2 was a rebase, but doesn’t matter in this case):

MR page:

Quality report for last master pipeline:

Coverage for last master pipeline:

Quality report for last branch commit pipeline (old version of codeclimate report that doesn’t has severity, but Gitlab ignore and set it as Unknown):

Coverage for last branch commit pipeline:

Master last commits:

So I need to ask again: why aren’t the code quality widget and coverage difference showing up on the MR page?

Thank you again.

@lucasoares3 Thanks for the detailed write-up and examples! I think at this point that opening a support ticket is the best way to go since you do have a Starter license. Referencing this thread should help get things going quicker with that team and if you add my name I can help get them some context.

I’ll keep digging into why this does not seem to be working in the meantime and wanted to pass along something from the team re: where the base report needs to be

“It’s not enough to make sure your target branch has a report on the latest commit, the report has to be on the diff base commit between the source and target branches”

I’ll keep an eye out for the ticket and jump in there once I see it.

-James H, GitLab Product Manager, Verify:Testing

It turned out that the maven plugin committing to Master was what was causing the issue. @lucasoares3 figured out a workaround and was kind enough to document it in an issue until we figure out a less manual effort for users.

-James H, GitLab Product Manager, Verify:Testing

1 Like