Custom code quality tool

After trying with success gitlab’s code quality with the code climate example in a small demo project I tried to configure it for the main project of my company.

But running code climate with our codebase is too slow to have running it in a ci pipeline (tried docker in docker executor with overlay fs in our own gitlab runner, maybe its missconfigured, who knows) . Our codebase is mainly php and php code sniffer and php mess detector are faster than codeclimate (30s vs 1h).

So I came to the conclusion of writting my own “code climate reporter” which reads phpcs and phpmd reports and writes a code climate alike report (copy pasted code from… But it does not work, nothing appears in merge request (of course, the branch improves some of the issues stated in the phpcs reports downloaded from the pipeline artifacts )

I have downloaded artifacts, both from master branch, and from merge request feature branch and both are valid json files with a schema similar to the generated by code climate.

is there any other requirement to comply with what gitlab code quality report expects?

Our .gitlab-ci.yml

  - phpcs.phar
  image: php:7.0
  allow_failure: true
  - php -r "file_exists('phpcs.phar')?'':copy('', 'phpcs.phar');"
  - php phpcs.phar --standard=PSR12 --report=json app/ > phpcs_report.json 
  - php code_quality_report.php ./phpcs_report.json  > ./gl-code-quality-report.json
    paths: [gl-code-quality-report.json]

And the gl-code-quality-report.json looks like

	"type": "issue",
	"check_name": "Generic Files LineLength TooLong",
	"description": "Line exceeds 120 characters; contains 150 characters",
	"categories": ["Style"],
	"location": {
		"path": "\/builds\/my_awesome_php_project/file-with-issue.php",
		"lines": {
			"begin": 7,
			"end": 7
	"remediation_points": 60000,
	"engine_name": "phpcodesniffer",
	"fingerprint": "25483835d0658f51751af0d77bf29ef3"

I created my own php-code-sniffer reporter (with a similar output as your example) and it works, here is the interesting part of my .gitlab-ci.yml:

            - composer.json
            - composer.lock
            - .gitlab-ci.yml
            - "**/*.php"
        - ./composer validate --no-check-all --strict
        - ./composer i --prefer-dist --no-progress --no-suggest
        - ./codequality -n src --report=CodeQualityReporter > gl-code-quality-report.json
            codequality: gl-code-quality-report.json

Are you sure you run this on master and merge requests

1 Like

sorry for not updating this answer as I resolved it. I will try to summary what I learned from this.

yes @BafS , you are right. You need to have all the thing configured in master branch, so when creating a merge request the CI can create a base report to compare when you push changes.

Also very important to generate always a valid json, as I was outputting errors instead of json, gitlab ignores the report and I was getting mad trying to figure out what was wrong. Use the artifact download button to take a look to the report and validate it.


1 Like

@alvaropgl Hi Alvaro! I’ve been struggling to get this working properly too. Would you mind sharing your final working code? Would be very helpful :slight_smile:


Take a look at the example project I have created to show you my pipeline

In the first Merge Request you can see codeclimate report

the codeclimate report is missing now. which version is avaliable for Custom code quality tool. thank you