Having problems getting Quarkus to build native images

I can’t seem to get a Quarkus project to build in GitLab. GL is running on a Docker stack, and it’s runners on the same Docker runtime.

I added a runner with the recommended configuration.

According to TestContainers:

This applies if you have your own GitlabCI runner installed, use the Docker executor and you have /var/run/docker.sock mounted in the runner configuration.

My runner config:

[[runners]]
  name = "Docker Container"
  url = "http://git.mydomain.com"
  id = 16
  token = "glrt-ASDFASDFADSFASD-Z"
  token_obtained_at = 2024-05-24T22:41:53Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker"
  [runners.cache]
    MaxUploadedArchiveSize = 0
  [runners.docker]
    tls_verify = false
    image = "docker:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0
    network_mtu = 0

And added TESTCONTAINERS_HOST_OVERRIDE to my .gitlab-ci.yml:

variables:
  TESTCONTAINERS_HOST_OVERRIDE: "host.docker.internal"

Here is my .gitlab-ci.yml, where I use a maven image for the sage and manually install quarkus before running the build:

variables:
  MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode --errors --update-snapshots"
  MAVEN_OPTS: "-Daether.connector.https.securityMode=insecure"
  TESTCONTAINERS_HOST_OVERRIDE: "host.docker.internal"

test:
  stage: test
  image:
    name: maven:3.9.6-eclipse-temurin-21
  services:
    - name: redis
      alias: redis
    - name: docker:dind
      alias: docker
      command: ["--tls=false"]
  variables:
    DOCKER_HOST: "tcp://docker:2375"
    DOCKER_TLS_CERTDIR: ""
    DOCKER_DRIVER: overlay2
  before_script:
    - curl -Ls https://sh.jbang.dev | bash -s - trust add https://repo1.maven.org/maven2/io/quarkus/quarkus-cli/
    - curl -Ls https://sh.jbang.dev | bash -s - app install --fresh --force quarkus@quarkusio
  script:
    - PATH=~/.jbang/bin:$PATH ./mvnw verify $MAVEN_CLI_OPTS -Dquarkus.redis.hosts=redis://redis -Dnative -Dquarkus.profile=dev -Dquarkus.container-image.build=true -Dquarkus.native.container-runtime=docker -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21
  tags:
    - quarkus
  artifacts:
    paths:
      - target
    expire_in: 1 week
  only:
    - master
    - dev
    - /^dev-[0-9]+(\.[0-9]+)+/
    - /^master-[0-9]+(\.[0-9]+)+/
    - tags

Error:

No container runtime was found. Make sure you have either Docker or Podman installed in your environment.

Full Error:

[INFO] --- quarkus:3.10.0:build (default) @ data-service ---
[WARNING] [io.quarkus.config] Unrecognized configuration key "quarkus.container-image.build" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
[WARNING] [io.quarkus.deployment.util.ContainerRuntimeUtil] quarkus.native.container-runtime config property must be set to either podman or docker and the executable must be available. Ignoring it.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  22.406 s
[INFO] Finished at: 2024-05-28T13:57:47Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.10.0:build (default) on project data-service: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR] 	[error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#resolveNativeImageBuildRunner threw an exception: java.lang.IllegalStateException: No container runtime was found. Make sure you have either Docker or Podman installed in your environment.
[ERROR] 	at io.quarkus.deployment.util.ContainerRuntimeUtil.detectContainerRuntime(ContainerRuntimeUtil.java:58)
[ERROR] 	at io.quarkus.deployment.util.ContainerRuntimeUtil.detectContainerRuntime(ContainerRuntimeUtil.java:44)
[ERROR] 	at io.quarkus.deployment.pkg.steps.NativeImageBuildContainerRunner.<init>(NativeImageBuildContainerRunner.java:31)
[ERROR] 	at io.quarkus.deployment.pkg.steps.NativeImageBuildLocalContainerRunner.<init>(NativeImageBuildLocalContainerRunner.java:19)
[ERROR] 	at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.resolveNativeImageBuildRunner(NativeImageBuildStep.java:347)
[ERROR] 	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[ERROR] 	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
[ERROR] 	at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
[ERROR] 	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
[ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
[ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
[ERROR] 	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1495)
[ERROR] 	at java.base/java.lang.Thread.run(Thread.java:1583)
[ERROR] 	at org.jboss.threads.JBossThread.run(JBossThread.java:483)
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.

I make sure that docker is available as a service, and override DOCKER_HOST. None of that works, though. I even moved the service variable declarations to the root of the config, to no avail:

services:
  - name: docker:dind
    alias: docker
    command: ["--tls=false"]
    
variables:
  TESTCONTAINERS_HOST_OVERRIDE: "host.docker.internal"
  DOCKER_HOST: "tcp://docker:2375"
  DOCKER_TLS_CERTDIR: ""
  DOCKER_DRIVER: overlay2
  MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode --errors --update-snapshots -U"
  MAVEN_OPTS: "-Daether.connector.https.securityMode=insecure"

test:
  stage: test
  image:
    name: maven:3.9.6-eclipse-temurin-21
  services:
    - name: redis
      alias: redis
  before_script:
    - curl -Ls https://sh.jbang.dev | bash -s - trust add https://repo1.maven.org/maven2/io/quarkus/quarkus-cli/
    - curl -Ls https://sh.jbang.dev | bash -s - app install --fresh --force quarkus@quarkusio
  script:
    - PATH=~/.jbang/bin:$PATH ./mvnw verify -Dquarkus.redis.hosts=redis://redis -Dnative -Dquarkus.profile=dev -Dquarkus.container-image.build=true -Dquarkus.native.container-runtime=docker -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21
#-DskipTests -DskipIT     
  tags:
    - quarkus
  artifacts:
    paths:
      - target
    expire_in: 1 week
  only:
    - master
    - dev
    - /^dev-[0-9]+(\.[0-9]+)+/
    - /^master-[0-9]+(\.[0-9]+)+/
    - tags

Same error.

I have a question posted at the Quarkus GitHub Discussion group regarding this, and there was a suggestion on how to change my job config to get the builds working. I do get past the error above, but now getting a permission error.

Here is the new .gitlab-ci.yml config:

variables:
  MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode --errors --update-snapshots"
  MAVEN_OPTS: "-Daether.connector.https.securityMode=insecure"

cache:
  paths:
    - .m2/repository/
    - target
    - public
    
test:
  stage: test
  image:
    name: quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21 
    entrypoint:
      - ""
  services:
    - name: redis
      alias: redis
  script:
    - ./mvnw quarkus:remove-extension -Dextensions='quarkus-container-image-docker'
    - ./mvnw verify -X -Dquarkus.redis.hosts=redis://redis -Pnative -X
  tags:
    - quarkus
  artifacts:
    name: artifacts-build-native
    expire_in: 1 days
    paths:
      - $CI_PROJECT_DIR/target/    
  only:
    - master
    - dev
    - /^dev-[0-9]+(\.[0-9]+)+/
    - /^master-[0-9]+(\.[0-9]+)+/
    - tags

The new error:

[INFO] Copying 3 resources from src/main/resources to target/classes
[DEBUG] Copying file application.properties
[DEBUG] file application.properties has a filtered file extension
[DEBUG] Using 'UTF-8' encoding to copy filtered resource 'application.properties'.
[DEBUG] copy /builds/products/services/data-service/src/main/resources/application.properties to /builds/products/services/data-service/target/classes/application.properties
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13.502 s
[INFO] Finished at: 2024-05-28T15:12:00Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.3.1:resources (default-resources) on project data-service: copying /builds/products/services/data-service/src/main/resources/application.properties to /builds/products/services/data-service/target/classes/application.properties failed with FileSystemException: /builds/products/services/data-service/target/classes/application.properties: Operation not permitted -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.3.1:resources (default-resources) on project data-service: copying /builds/products/services/data-service/src/main/resources/application.properties to /builds/products/services/data-service/target/classes/application.properties failed with FileSystemException
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:333)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:52)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:161)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:73)
Caused by: org.apache.maven.plugin.MojoExecutionException: copying /builds/products/services/data-service/src/main/resources/application.properties to /builds/products/services/data-service/target/classes/application.properties failed with FileSystemException
    at org.apache.maven.plugins.resources.ResourcesMojo.execute (ResourcesMojo.java:347)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:52)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:161)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:73)
Caused by: org.apache.maven.shared.filtering.MavenFilteringException: copying /builds/products/services/data-service/src/main/resources/application.properties to /builds/products/services/data-service/target/classes/application.properties failed with FileSystemException
    at org.apache.maven.shared.filtering.DefaultMavenFileFilter.copyFile (DefaultMavenFileFilter.java:116)
    at org.apache.maven.shared.filtering.DefaultMavenResourcesFiltering.filterResources (DefaultMavenResourcesFiltering.java:268)
    at org.apache.maven.plugins.resources.ResourcesMojo.execute (ResourcesMojo.java:343)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:52)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:161)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:73)
Caused by: java.nio.file.FileSystemException: /builds/products/services/data-service/target/classes/application.properties: Operation not permitted
    at sun.nio.fs.UnixException.translateToIOException (UnixException.java:100)
    at sun.nio.fs.UnixException.rethrowAsIOException (UnixException.java:106)
    at sun.nio.fs.UnixException.rethrowAsIOException (UnixException.java:111)
    at sun.nio.fs.UnixFileAttributeViews$Posix.setMode (UnixFileAttributeViews.java:277)
    at sun.nio.fs.UnixFileAttributeViews$Posix.setPermissions (UnixFileAttributeViews.java:299)
    at java.nio.file.Files.setPosixFilePermissions (Files.java:2169)
    at org.apache.maven.shared.filtering.FilteringUtils.copyFilePermissions (FilteringUtils.java:410)
    at org.apache.maven.shared.filtering.FilteringUtils.copyFile (FilteringUtils.java:394)
    at org.apache.maven.shared.filtering.DefaultMavenFileFilter.copyFile (DefaultMavenFileFilter.java:109)
    at org.apache.maven.shared.filtering.DefaultMavenResourcesFiltering.filterResources (DefaultMavenResourcesFiltering.java:268)
    at org.apache.maven.plugins.resources.ResourcesMojo.execute (ResourcesMojo.java:343)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:906)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:283)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:206)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:283)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:226)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:407)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke (DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke (Method.java:580)
    at org.apache.maven.wrapper.BootstrapMainStarter.start (BootstrapMainStarter.java:52)
    at org.apache.maven.wrapper.WrapperExecutor.execute (WrapperExecutor.java:161)
    at org.apache.maven.wrapper.MavenWrapperMain.main (MavenWrapperMain.java:73)
[ERROR] 

It looks like a file permission issue:

Caused by: java.nio.file.FileSystemException: /builds/products/services/data-service/target/classes/application.properties: Operation not permitted
    at sun.nio.fs.UnixException.translateToIOException (UnixException.java:100)
    at sun.nio.fs.UnixException.rethrowAsIOException (UnixException.java:106)
    at sun.nio.fs.UnixException.rethrowAsIOException (UnixException.java:111)
    at sun.nio.fs.UnixFileAttributeViews$Posix.setMode (UnixFileAttributeViews.java:277)
    at sun.nio.fs.UnixFileAttributeViews$Posix.setPermissions (UnixFileAttributeViews.java:299)
    at java.nio.file.Files.setPosixFilePermissions (Files.java:2169)
    at org.apache.maven.shared.filtering.FilteringUtils.copyFilePermissions (FilteringUtils.java:410)
    at org.apache.maven.shared.filtering.FilteringUtils.copyFile (FilteringUtils.java:394)
    at org.apache.maven.shared.filtering.DefaultMavenFileFilter.copyFile (DefaultMavenFileFilter.java:109)

I know that quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21 is running as user 1001:

Would that make a difference? I’m not sure what user the runners run as.

In solving the permissions issue, I jumped into the container while it was running (added a sleep 5m to the start of scripts list`), and poked around:

  script:      
    - sleep 5m
    - MAVEN_CONFIG="" ./mvnw clean package -Daether.connector.https.securityMode=insecure -s .m2/settings.xml --errors --update-snapshots -Dquarkus.redis.hosts="redis://redis" -Pnative 

Taking a look at the ownership of the files (root) and the container’s running user (quarkus):

[quarkus@runner-kfkk5xzrr-project-118-concurrent-0 data-service]$ ls -al
total 88
drwxrwxrwx  9 root root  4096 May 29 13:24 .
drwxrwxrwx  4 root root  4096 May 24 22:49 ..
-rw-rw-rw-  1 root root    75 May 24 22:49 .dockerignore
drwxrwxrwx  2 root root  4096 May 24 22:49 docs
drwxrwxrwx  6 root root  4096 May 29 13:24 .git
-rw-rw-rw-  1 root root   461 May 28 13:48 .gitignore
-rw-rw-rw-  1 root root  7626 May 29 13:16 .gitlab-ci.yml
-rw-rw-rw-  1 root root     0 May 29 13:24 LICENSE
drwxrwxrwx  2 root root  4096 May 24 22:49 .m2
drwxrwxrwx  3 root root  4096 May 24 22:49 .mvn
-rwxrwxrwx  1 root root 11289 May 24 22:49 mvnw
-rw-rw-rw-  1 root root  7591 May 24 22:49 mvnw.cmd
-rw-rw-rw-  1 root root 11110 May 28 20:44 pom.xml
-rw-rw-rw-  1 root root  2718 May 27 04:43 README.md
drwxrwxrwx  2 root root  4096 May 28 13:48 redoc
drwxrwxrwx  4 root root  4096 May 24 22:49 src
drwxr-xr-x 10 root root  4096 May 29 13:16 target
[quarkus@runner-kfkk5xzrr-project-118-concurrent-0 data-service]$ ls -al target
total 60
drwxr-xr-x 10 root root  4096 May 29 13:16 .
drwxrwxrwx  9 root root  4096 May 29 13:24 ..
drwxr-xr-x  4 root root  4096 May 29 13:16 classes
drwxr-xr-x  3 root root  4096 May 29 13:16 generated-sources
drwxr-xr-x  3 root root  4096 May 29 13:16 generated-test-sources
drwxr-xr-x  2 root root  4096 May 29 13:16 maven-archiver
drwxr-xr-x  3 root root  4096 May 29 13:16 maven-status
-rw-r--r--  1 root root 15661 May 29 13:16 data-service-1.0.0-SNAPSHOT.jar
drwxr-xr-x  5 root root  4096 May 29 13:16 quarkus-app
-rw-r--r--  1 root root   206 May 29 13:16 quarkus-artifact.properties
drwxr-xr-x  2 root root  4096 May 29 13:16 smallrye
drwxr-xr-x  3 root root  4096 May 29 13:16 test-classes
[quarkus@runner-kfkk5xzrr-project-118-concurrent-0 data-service]$ whoami
quarkus

I therefore updated the user id of the container in the yaml:

  image:
    name: quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21
    entrypoint:
      - ""    
    docker:
      user: root

Here is the latest (notice -DskipITs):

test:
  stage: test
  image:
    name: quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21
    entrypoint:
      - ""    
    docker:
      user: root
  services:
    - name: redis
      alias: redis
    - name: docker:dind
      alias: docker
      command: ["--tls=false"]
  variables:
    DOCKER_HOST: "tcp://docker:2375"
    DOCKER_TLS_CERTDIR: ""
    DOCKER_DRIVER: overlay2
  script:      
    - MAVEN_CONFIG="" ./mvnw verify -Daether.connector.https.securityMode=insecure -DskipITs -s .m2/settings.xml --errors --update-snapshots -Dquarkus.redis.hosts="redis://redis" -Dnative 
  tags:
    - quarkus
  artifacts:
    name: artifacts-build-native
    expire_in: 1 days
    paths:
      - target
  cache:
    paths:
      - target
      - public
  only:
    - master
    - dev
    - /^dev-[0-9]+(\.[0-9]+)+/
    - /^master-[0-9]+(\.[0-9]+)+/
    - tags

The result is that I’m able to build and unit test, but building the native image fails:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] |
[INFO] +-- Data REST Resource
[INFO] | +-- [OK] Endpoint: GET /data - 0.763 ss
[INFO] | +-- [OK] Endpoint: GET /data/{key} - 0.034 ss
[INFO] | +-- [OK] Endpoint: POST /data - 0.052 ss
[INFO] | +-- [OK] Endpoint: DELETE /data/{key} (EXISTING) - 0.012 ss
[INFO] | +-- [OK] Endpoint: DELETE /data/{key} (NON-EXISTING) - 0.008 ss
[INFO] |
[INFO] +-- Status REST Resource
[INFO] | +-- [OK] Endpoint: GET /status/ping - 0.023 ss
[INFO] | +-- [OK] Endpoint: GET /status - 0.012 ss
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- jar:3.3.0:jar (default-jar) @ data-service ---
[INFO] Building jar: /builds/products/services/data-service/target/data-service-1.0.0-SNAPSHOT.jar
[INFO] 
[INFO] --- git-commit-id:4.9.10:validateRevision (validate-the-git-infos) @ data-service ---
[INFO] 
[INFO] --- quarkus:3.10.0:build (default) @ data-service ---
[WARNING] [io.quarkus.config] Unrecognized configuration key "quarkus.container-image.build" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
[WARNING] [io.quarkus.config] Unrecognized configuration key "quarkus.container-image.group" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
[WARNING] [io.quarkus.config] Unrecognized configuration key "quarkus.container-image.name" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
[WARNING] [io.quarkus.config] Unrecognized configuration key "quarkus.container-image.registry" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  27.561 s
[INFO] Finished at: 2024-05-29T14:32:33Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.10.0:build (default) on project data-service: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR] 	[error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#resolveNativeImageBuildRunner threw an exception: java.lang.IllegalStateException: No container runtime was found. Make sure you have either Docker or Podman installed in your environment.

I even removed the docker service and environment variables above to use the alternative TestContainers suggestion of direct runner configuration (volume mounting /var/run/docker.sock):

Runner config:

ca3143707d21:/etc/gitlab-runner# cat /etc/gitlab-runner/config.toml

[[runners]]
  name = "Docker Container"
  url = "http://git.mydomain.com"
  id = 16
  token = "glrt-asdfasdfasdfadsf-Z"
  token_obtained_at = 2024-05-24T22:41:53Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker"
  [runners.cache]
    MaxUploadedArchiveSize = 0
  [runners.docker]
    tls_verify = false
    image = "docker:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0
    network_mtu = 0

Changes to .gitlab-ci.yaml:

  services:
    - name: redis
      alias: redis
  variables:
    TESTCONTAINERS_HOST_OVERRIDE: "host.docker.internal"

Neither of those methods for making docker available worked.