Which gitlab-runner for testing gui-applications?

I already tried different approaches in order to build a CICD environment for MagicDraw plugins. The difficulty lies in the fact that MagicDraw as a program cannot run headless, and its tests, which are an extension to junit tests, require the program to start, execute the test, and close again. I bundled the testing process in an ant script, which, when executed locally via “ant -buildfile build.xml”, starts the program, runs the test, and outputs a test-report, which I planned to upload as artifact to the CICD when automating the process.

First, I tried the Virtualbox Executor, but realized that it runs in headless mode, and therefore cannot run a gui (you can gladly correct me if I’m wrong with that assumption). Then, I tried the setup of a virtual machine running 24/7 and a shell executor on the virtual machine. But even with this setup, when running the test, it does not start the gui, but instead it seemingly tries to execute the test step of the ant buildscript, is finished in 2 seconds since it just does not start the gui (no error message, just says build sucessful, but the gui tests require 30~ secs when starting the program) and outputs failed test reports, which would be successful if the tests were actually executed. I don’t know what I can do in order to make the gitlab-runner execute the program with the gui. Maybe anyone here has experience with testing gui applications.

the test target of the ant script is as follows:

    <target name="test" depends="compile-test">
            <!-- include the JUnit platform related libraries
            required to run the tests, the magicdraw libraries and the tested plugin jars -->
            <classpath refid="junit.jupiter.classpath"/>
            <classpath refid="junit.platform.classpath"/>
            <classpath refid="lib.classpath"/>
            <classpath refid="plugin.classpath"/>
                <!-- the test classes themselves -->
                <pathelement location="${build.testclasses.dir}"/>
            <!-- the test classes themselves -->
            <testclasses outputdir="${output.dir}/test-report">
				<fileset dir="${build.testclasses.dir}"/>
				    <jvmarg value="-Xmx4000M"/>
					<jvmarg value="-Xss1024K"/>
					<jvmarg value="-Xss1024K"/>
					<jvmarg value="-Xss1024K"/>
					<jvmarg value="-Dmd.development=true"/>
					<jvmarg value="-Dmd.plugins.dir=${MagicDraw}/plugins;../../files/Plugins/TestPlugin"/>
				<listener type="legacy-xml" sendSysOut="false"/>

my gitlab-runner is on version 11.2.0
the gitlab version is GitLab Enterprise Edition [14.7.3-ee]

my .yml file is pretty simple at the moment (I’m not uploading any artifacts from the test cases until those are working correctly)

  - deploy

  stage: deploy
  - ant -buildfile build.xml
      - Test

Maybe someone here has experience with running tests that require a gui. Thanks in advance for any help.


I managed to circumvent the needed access to the ubuntu X11 component by using xvfb-run as a virtual graphics output. that solved the problem.

Hi @Orbdamu i m facing the exact issue. I m using azure cloud windows as my virtual machine and using shell executor i m trying to run the gui tests but as you mentions the job ia able to communicate but does not launch the gui/application.

Could you please elaborate the solution?