GoogeTest(C/C++) -> artifacts:reports:junit -> "Test summary failed loading results"

I have been trying to solve how to get junit (test results) into my projects merge requests. I setup a special helloworld project to see if I could get it working. Originally I built a very simple object and testing its method with googletest then recording the results ("./unit-test --gtest_output=json" or “./unit-test --gtest_output=xml”) and passing those into artifacts:reports:junit. The results always seem to be the same on the merge page it recognizes that I want unit tests reported but I always get the error “Test summary failed loading results”.

So I canned all the results so I could play with the format a little better. I have a project containing the following files at the root.

FILE: test_detail.xml

<?xml version="1.0" encoding="UTF-8"?>
<testsuites tests="1" failures="1" disabled="0" errors="0" timestamp="2018-09-02T01:35:56" time="0.001" name="AllTests">
  <testsuite name="HelloMarsTest" tests="1" failures="1" disabled="0" errors="0" time="0">
    <testcase name="Basic" status="run" time="0" classname="HelloMarsTest">
      <failure message="gtest_main.C:8&#x0A;Expected equality of these values:&#x0A;  &quot;marss&quot;&#x0A;  mars.getName().c_str()&#x0A;    Which is: &quot;mars&quot;" type=""><![CDATA[gtest_main.C:8
Expected equality of these values:
  "marss"
  mars.getName().c_str()
    Which is: "mars"]]></failure>
    </testcase>
  </testsuite>
</testsuites>

FILE: test_detail.json

{
  "tests": 1,
  "failures": 1,
  "disabled": 0,
  "errors": 0,
  "timestamp": "2018-09-02T01:51:36Z",
  "time": "0.001s",
  "name": "AllTests",
  "testsuites": [
    {
      "name": "HelloMarsTest",
      "tests": 1,
      "failures": 1,
      "disabled": 0,
      "errors": 0,
      "time": "0s",
      "testsuite": [
        {
          "name": "Basic",
          "status": "RUN",
          "time": "0s",
          "classname": "HelloMarsTest",
          "failures": [
            {
              "failure": "gtest_main.C:8\nExpected equality of these values:\n  \"marss\"\n  mars.getName().c_str()\n    Which is: \"mars\"",
              "type": ""
            }
          ]
        }
      ]
    }
  ]
}

FILE: .gitlab-ci.yml

image: ubuntu:18.04

stages:
  - test

test:
  stage: test
  tags:
    - docker
  script:
    - echo hi
  artifacts:
    expire_in: 1 hour
    reports:
      junit: test_detail.xml

I have attempted this with both test_detail.xml and test_detail.json in the artifacts:reports:junit section. Any suggestions?

Thanks!

I just tested using the junit xml example off https://www.ibm.com/support/knowledgecenter/en/SSQ2R2_14.1.0/com.ibm.rsar.analysis.codereview.cobol.doc/topics/cac_useresults_junit.html and it didn’t work either.

I’m wondering if this might be related…

I need to get up to speed on Ruby so I can better investigate my issue.

Have you found a solution for getting this to work?

I am struggling with a related issue and my xml files just are NOT shown anywhere “10 tests passed. 190 tests failed“. How to setup CircleCi-style test-reports? POSSIBLE AT ALL?

No. I updated to 11.3 this morning in hope the highlighted changes might resolve it but it made no difference. I think I’m going to open a ticket on it.

I finally got things working with a very simple java project. Its here if helpful. Now I need to apply this back to my real projects.

1 Like

Hello guys,
I am starting to tru to use gitlab CI and i have the same problem than you with the xml output (junit format) of cmocka…

Did you find sometihng on your side ?
Did you open an issue ?

The link is dead, can you please post the actual solution (or abstract here), that may help a lot of other people.

Thx!