Currently I have a suite of NUnit tests running for my Unity application which are converted to JUnit xml and shown as reports in my CI configuration. However, a few tests are showing up as “skipped” in the reports with this system output.
{
"message": ""
}
The jobs are still taking up the correct amount of time, so they are being run, but I have no idea why they show up as skipped.
Furthermore, when I run the exact command the CI job is running locally, no jobs show up as skipped at all in the output xml, both in the Nunit output generated by Unity and the Junit that it gets converted into before being displayed in the GItLab report.
It’s also worth mentioning that I am not using the [ignore] or any conditional testing attributes. The failing tests are written right next to the passing ones, and look very similar. Here’s an example of a failing test
[UnityTest]
public IEnumerator Test_Integration_ImportIncompatibleModel() {
testSceneHelper.cseManager.ImportModel(testSceneHelper.testAssetsPath+"/MixamoModel.fbx");
while (testSceneHelper.cseManager.isCurrentlyImporting) {
yield return null;
}
if (testSceneHelper.cseManager._errorPopup.activeSelf == true) {
Assert.Pass();
}
Assert.Fail();
yield return null;
}
There isn’t much of a commonality between the failing tests, but I did consider that suppressing error messages like the above script could cause the skipped status.
Because everything looks good on whatever I run locally the only way I can test the problem is by making a commit and running a new pipeline, which isn’t ideal.
This has had me scratching my head, had anyone run into this type of issue? Or perhaps is there a special way that GitLab is reading test output that would be causing this behavior on the Web GUI but not anything else?