Problem to solve
We are using a gitlab-runner with docker executor.
Very often, when two or more jobs run in parallel in our pipeline, the two jobs “step on each other”, one succeeds and the other fails with this:
The documentation says that Docker Executor “Maintain the same build environment for each job.”. I’m not sure if that’s related but it seems to me that if “each job” have the “same build environment”, they share it.
What we want is a complete isolation between jobs, no matter what they are doing. We do not share files or cache or anything using a build dir.
We also tried to use a custom build dir, using various values that we thought would be unique for each job, but they turn to have the same value so no hope with this.
What should we do here? Thanks
Steps to reproduce
Setup a machine with Docker.
Create a runner group, get a token and register a new runner that way:
docker run -d \
--name "gitlab-runner" \
--volume /var/run/docker.sock:/var/run/docker.sock \
--restart always \
"gitlab/gitlab-runner:alpine3.18-v16.6.1"
docker exec "gitlab-runner" gitlab-runner register \
--non-interactive \
--url "https://gitlab.com" \
--token "${gitlab-registration-token}" \
--executor "docker" \
--docker-image "alpine:3.18" \
--docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
--description "${environment} (docker)"
docker restart "gitlab-runner"
docker exec "gitlab-runner" gitlab-runner run
This launches a runner in docker, and it will use the host’s docker daemon to run jobs with the defined image.
Configuration
In your project, have a CI such as this one (adapt your tags to the runner you’ve registered…):
stages:
- run-parallel
run-one:
stage: run-parallel
environment:
name: staging
image:
name: debian:latest
script:
- hostname
- ls -li
run-two:
stage: run-parallel
environment:
name: staging
image:
name: debian:latest
script:
- hostname
- ls -li
Versions
- GitLab.com SaaS
- Self-hosted Runners
Versions
- GitLab GitLab Enterprise Edition 17.4.0-pre de710e6b02e
- GitLab Runner 16.6.1