What are you seeing, and how does that differ from what you expect to see?
I am using GitLab CI on a self-hosted instance: 12.10.3, with runners using docker executor running in a self-hosted Docker environment (see details below). The pipeline fails “every once in a while” with this error message
Pulling docker image gitlab/gitlab-runner-helper:x86_64-c5874a4b ... ERROR: Job failed: execution took longer than 1h0m0s seconds
meaning a failed job could succeed when I execute it a second or third time.
I already found these threads
- GitLab.com CI stuck on docker build
- Pulling docker image gitlab/gitlab-runner-helper:x86_64-003fe500 times out
but they build Docker images in docker-in-docker, but in my case I am running a Maven and Node.js build).
These are the last messages in the job’s log. The runner’s log level is set to debug:
[...] Downloading artifacts Running before_script and script Authenticating with credentials from $DOCKER_AUTH_CONFIG $ mvn $MAVEN_CLI_OPTS verify --projects gui [INFO] Error stacktraces are turned on. [INFO] Scanning for projects... [INFO] [INFO] ------------------------< fxmi:gui >------------------------ [INFO] Building gui 1.18.0-SNAPSHOT [INFO] --------------------------------[ pom ]--------------------------------- [INFO] [INFO] --- frontend-maven-plugin:1.3:install-node-and-npm (install node and npm) @ gui --- [INFO] Node v8.16.2 is already installed. [INFO] NPM 6.4.1 is already installed. [INFO] [INFO] --- frontend-maven-plugin:1.3:npm (npm install) @ gui --- [INFO] Running 'npm ci --cache ../.npm' in /builds/gitlab/im/fxmi/gui [INFO] [INFO] > email@example.com install /builds/gitlab/im/fxmi/gui/node_modules/uws [INFO] > node-gyp rebuild > build_log.txt 2>&1 || exit 0 [INFO] Running after_script WARNING: Failed to inspect build container 79a6a08400ad12696541620e90709a8b1c037dfa469de16a1958d4a3ad7be1c2 context deadline exceeded (docker_command.go:77:0s) Authenticating with credentials from $DOCKER_AUTH_CONFIG Pulling docker image my-registry/maven-nvm:3.6-jdk-8 ... Uploading artifacts for failed job Pulling docker image gitlab/gitlab-runner-helper:x86_64-c5874a4b ... ERROR: Job failed: execution took longer than 1h0m0s seconds
Docker image used by this job
I use a custom Docker image for running this job. This is the Dockerfile that creates the image:
FROM maven:3.6-jdk-8 RUN mkdir /builds \ && useradd --no-log-init -mr -g users -N non-root \ && mkdir -p /home/non-root/.m2 \ && chown -R non-root:users /builds /home/non-root USER non-root ENV MAVEN_CONFIG="/home/non-root/.m2" ENV NVM_DIR="/home/non-root/.nvm" ENV YVM_DIR="/home/non-root/.yvm" COPY settings.xml /home/non-root/.m2/ COPY install-scripts / RUN ./install-node-yarn-version-manager.sh
image: my-registry/maven-nvm:3.6-jdk-8 variables: MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository" MAVEN_CLI_OPTS: "--batch-mode --errors" stages: - build-gui - build-backend build-gui: stage: build-gui cache: key: "gui" paths: - .m2/repository/ - .npm/ script: - mvn $MAVEN_CLI_OPTS verify --projects gui artifacts: paths: - gui/target [...]
This Docker hosts the GitLab runners and build containers
|OS Information||linux x86_64 Debian GNU/Linux 10 (buster)|
|Version||19.03.9 (API: 1.40)|
|Network Plugins||bridge, host, ipvlan, macvlan, null, overlay|
Any ideas? I was hoping, that this issue https://github.com/moby/moby/issues/40514 has been fixed and according to its milestone label “19.03.7” I should be safe running 19.03.9 here