Custom code quality tool

After trying with success gitlab’s code quality with the code climate example in a small demo project I tried to configure it for the main project of my company.

But running code climate with our codebase is too slow to have running it in a ci pipeline (tried docker in docker executor with overlay fs in our own gitlab runner, maybe its missconfigured, who knows) . Our codebase is mainly php and php code sniffer and php mess detector are faster than codeclimate (30s vs 1h).

So I came to the conclusion of writting my own “code climate reporter” which reads phpcs and phpmd reports and writes a code climate alike report (copy pasted code from https://github.com/codeclimate/codeclimate-phpcodesniffer/blob/master/engine.php)… But it does not work, nothing appears in merge request (of course, the branch improves some of the issues stated in the phpcs reports downloaded from the pipeline artifacts )

I have downloaded artifacts, both from master branch, and from merge request feature branch and both are valid json files with a schema similar to the generated by code climate.

is there any other requirement to comply with what gitlab code quality report expects?

Our .gitlab-ci.yml

cache:
  paths:
  - phpcs.phar
code_quality:
  image: php:7.0
  allow_failure: true
  before_script:
  - php -r "file_exists('phpcs.phar')?'':copy('https://squizlabs.github.io/PHP_CodeSniffer/phpcs.phar', 'phpcs.phar');"
  script:
  - php phpcs.phar --standard=PSR12 --report=json app/ > phpcs_report.json 
  - php code_quality_report.php ./phpcs_report.json  > ./gl-code-quality-report.json
  artifacts:
    paths: [gl-code-quality-report.json]

And the gl-code-quality-report.json looks like

[{
	"type": "issue",
	"check_name": "Generic Files LineLength TooLong",
	"description": "Line exceeds 120 characters; contains 150 characters",
	"categories": ["Style"],
	"location": {
		"path": "\/builds\/my_awesome_php_project/file-with-issue.php",
		"lines": {
			"begin": 7,
			"end": 7
		}
	},
	"remediation_points": 60000,
	"engine_name": "phpcodesniffer",
	"fingerprint": "25483835d0658f51751af0d77bf29ef3"
}]

I created my own php-code-sniffer reporter (with a similar output as your example) and it works, here is the interesting part of my .gitlab-ci.yml:

code_quality:
    only:
        changes:
            - composer.json
            - composer.lock
            - .gitlab-ci.yml
            - "**/*.php"
    script:
        - ./composer validate --no-check-all --strict
        - ./composer i --prefer-dist --no-progress --no-suggest
        - ./codequality -n src --report=CodeQualityReporter > gl-code-quality-report.json
    artifacts:
        reports:
            codequality: gl-code-quality-report.json

Are you sure you run this on master and merge requests

sorry for not updating this answer as I resolved it. I will try to summary what I learned from this.

yes @BafS , you are right. You need to have all the thing configured in master branch, so when creating a merge request the CI can create a base report to compare when you push changes.

Also very important to generate always a valid json, as I was outputting errors instead of json, gitlab ignores the report and I was getting mad trying to figure out what was wrong. Use the artifact download button to take a look to the report and validate it.

Thanks

@alvaropgl Hi Alvaro! I’ve been struggling to get this working properly too. Would you mind sharing your final working code? Would be very helpful

Best

Take a look at the example project I have created to show you my pipeline

In the first Merge Request you can see codeclimate report

imagen991×345 38.2 KB

the codeclimate report is missing now. which version is avaliable for Custom code quality tool. thank you