We’re using a self-hosted GitLab with self-hosted GitLab CI runners on on-premise VMs. We have a variety of jobs / build steps with greatly differing resource usage characteristics.
We currently have a bit of a problem balancing job concurrency. For most job types (let’s call them
build-step-, our VMs can perfectly well handle a runner with two concurrent jobs. Actually, they could probably handle even more. However, there are other job types (let’s call one
build-step-5) that are rather long-running and resource-intensive. If two of these are scheduled on the same runner, they compete for resources and can each take up to three times their typical execution time and run into timeouts. Running one of these jobs alongside one for any other build step is totally fine.
Now my question is: Is it possible to limit per-runner concurrency for a certain job type / build step?
A few directions I’ve thought about:
- Use a tool like
resource_groups in the
.gitlab-ci.yml. However, as far as I understand
resource_groups, they can only be used to limit a job to one concurrent instance globally. I’m looking for a similar way, only per runner.
- Set up a fleet of VMs and runners with custom configuration for
build-step-5and a separate group for all other build steps. While this is possible, I’d like to have all-purpose runners, especially since we are running only a few VMs. If nothing else works, I guess I’ll have to go for this approach.
Maybe there’s already a way to do so that I’m just missing?
Example excerpt from a runner configuration toml, in case that helps:
concurrent = 2 check_interval = 0 [session_server] session_timeout = 1800 [[runners]] name = "gitlab-runner-linux3" url = "..." token = "..." executor = "docker" [runners.custom_build_dir] ... [runners.docker] image = "ubuntu:20.04" privileged = false disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false volumes = ["/cache"] shm_size = 0