Warning request concurrency larger than global concurrent limit

Describe your question in as much detail as possible:
I’m setting concurrency request to “4” and global concurrent to “8”. Despite this, when I’m starting the gitlab-runner pod, I get the following error:

WARNING: The specified runner request concurrency (4) is larger than the current global concurrent limit (1). The global concurrent limit will not be increased and takes precedence.

• What version are you on? Are you using self-managed or GitLab.com?

  • GitLab (Hint: /help): 16.3.4
  • Runner (Hint: /admin/runners): 16.5.0

• Add the CI configuration from .gitlab-ci.yml and other configuration if relevant (e.g. docker-compose.yml)

This is is the content of /home/gitlab-runner/.gitlab-runner/config.toml:

concurrent = 4
check_interval = 0
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "gitlab-ci-runner-d8c589f68-2t8bq"
  request_concurrency = 4
  url = "https://git.example.com"
  id = 33
  token = "<token>"
  token_obtained_at = 2023-10-24T12:47:56Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "kubernetes"
    MaxUploadedArchiveSize = 0
  [runners.kubernetes]
    host = ""
    bearer_token_overwrite_allowed = false
    image = "registry.example.com/open/docker-ci:6534e148-62d3c4ae"
    namespace = "kube-gitlab-runners"
    namespace_overwrite_allowed = ""
    privileged = true
    pull_policy = ["if-not-present"]
    node_selector_overwrite_allowed = ""
    image_pull_secrets = ["regcred"]
    helper_image = "gitlab/gitlab-runner-helper:x86_64-v16.5.0"
    terminationGracePeriodSeconds = 10
    poll_interval = 5
    poll_timeout = 360
    pod_labels_overwrite_allowed = ""
    service_account_overwrite_allowed = ""
    pod_annotations_overwrite_allowed = ""
    [runners.kubernetes.pod_security_context]
    [runners.kubernetes.init_permissions_container_security_context]
    [runners.kubernetes.build_container_security_context]
    [runners.kubernetes.helper_container_security_context]
    [runners.kubernetes.service_container_security_context]
    [runners.kubernetes.volumes]
    [runners.kubernetes.dns_config]
  [runners.kubernetes.node_selector]
    "environment" = "staging"
  [runners.cache]
    Type = "s3"
    ServerAddress = "minio.live.svc.i.example.com"
    AccessKey = "<user>"
    SecretKey = "<key>"
    BucketName = "runners-cache"
    Insecure = true
    Path = "kube-runners"
    Shared = true

• What troubleshooting steps have you already taken? Can you link to any docs or other resources so we know where you have been?
  I’ve added a configmap for a file that mounts over /etc/gitlab-runner/config.toml in order to change the global concurrency to 8, but this doesn’t seem have any affect:

  config.toml: |
    concurrent = 8
    check_interval = 0
    shutdown_timeout = 0

    [session_server]
      session_timeout = 1800

Deployment method is Helm? Just use the value in Helm Chart to set global concurrency.

It’s helm, but it’s not the official chart. It’s a very much simplified version. I really don’t need the official chart, it’s overly complicated and it doesn’t have many advantages for my setup, I prefer to understand a little bit what’s going on. Besides the fact that I’d be dependent on yet another layer (waiting for the chart updates and what not) of complexity.

So what I need to know is simply why the gitlab-runner ignores this particular directive while the others work without any issues.

If you haven’t made any modifications to the container, it’s using config.toml in the /home/gitlab-runner/.gitlab-runner/config.toml. Any other location is ignored.

Well, I’ve already shown the contents of config.toml in that exact path. The warning is still there. So that’s that.
I’ve also changed /etc/gitlab-runner/config.toml, just to make sure. Of course, it makes no difference.

And I know it uses /home/gitlab-runner/.gitlab-runner/config.toml because other settings such as request_concurrency or image or image_pull_secrets (which is passed on through the command line) do work.

I’m getting the warning only when registering the runner. When it starts running it seems that concurrent does affect max_build, given the output of gitlab-runner run:

Starting multi-runner from /home/gitlab-runner/.gitlab-runner/config.toml...  builds=0 max_builds=0
WARNING: Running in user-mode.
WARNING: Use sudo for system-mode:
WARNING: $ sudo gitlab-runner...

Configuration loaded                                builds=0 max_builds=4
listen_address not defined, metrics & debug endpoints disabled  builds=0 max_builds=4
[session_server].listen_address not defined, session endpoints disabled  builds=0 max_builds=4
Initializing executor providers                     builds=0 max_builds=4

So maybe it’s just a really, really misleading warning?

If you want to limit number of running jobs you need to use limit instead of request_concurrency. If you really want to change number of simultanious HTTP requests that GitLab Runner does to GitLab API you might have found a bug. I assume that the limit is hardcoded to 1 during registering, which kind of make sense.

So if I set the global limit through concurrency I could just use that as such, right? Without adding any other per-runner limits, if I understand correctly.
I misinterpreted request_concurrency, indeed, I thought it was referring to the number of concomitant jobs that the runner could execute, but I don’t mind having just a generic global limit, it doesn’t have to be very strict.

simply put, yes.

Global limit on concurrently running jobs and limit per runner makes sense in non-k8s runners, where you can have multiple runners on a single server. (And limit max jobs between them)
In k8s environment (using the official container) where you can have single runner per pod, global limit practically means limit for that single runner and there is no need to set any runner specific limits.

Thanks, this has cleared things up for me. It makes sense now.