One gitlab-runner instance blocks another

I have an odd issue with gitlab-runner. It’s something only happens once in a while, and has been occurring over the past few years.

We have a self hosted GitlLab instance. I have two separate projects on that instance that use gitlab-runner to perform tasks on the same vm. One runner is for deploying code, the other runs a docker container with my backup software.

Both runners are use the shell executor.

What happens is that when the backup job is running, sometimes the code deploy will not run. I have not pinned down if the backup job always blocks the deploy job, or not.

I thought that gitlab-runner was supposed to be able to run each instance at the same time. Is that not the case?

Any feedback will be appreciated.

Current version of gitlab-runner is 12.10.1.

Can you provide your gitlab runner config file? It might be that you have it limited to run 1 concurrent job at a time.

Look at
limit and request_concurrency.

1 Like
concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "root_at_ws-test-blogs-02_blogs_shell"
  url = "https://gitlabhost/"
  token = ""
  executor = "shell"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

[[runners]]
  name = "root_at_ws-test-blogs_duplicati-backup_or_repair_shell"
  url = "https://gitlabhost/"
  token = ""
  executor = "shell"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

[[runners]]
  name = "root_at_ws-test-blogs-02_duplicati-startstop-webui_shell"
  url = "https://gitlabhost/"
  token = ""
  executor = "shell"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

concurrent looks promising. The docs confirm that’s the setting I should update.

Makes me wonder why the default is 1. That kinda defeats the whole point of registering multiple runners. I’d think the default should be equal to how many runners you have registered. Ah well, hopefully there are good reasons behind it. At least now I know about it.

Thanks for getting me to look at the config file, @tmos22 .

Increasing the value of concurrent fixed it.