GitLab upgrades register a new runner - runner count increases indefinitely

We’ve been running GitLab in a Kubernetes cluster for a few months now, and I’ve noticed something peculiar about our CI/CD runners. When I go into the UI to list the runners I see about 19 pages (!!) of configured runners. But my cluster is only 6 nodes large!

I think what might be happening is when I upgrade GitLab via helm it’s destroying the deployment for the old version and creating the deployment of the new one, but the old runner never had a chance to un-register. Do we need to ensure we delete all old runners on every GitLab upgrade?

  • What are you seeing, and how does that differ from what you expect to see?
    19 pages of GitLab runners, but I expect to see only 1 page with ~6 runners since I have 6 nodes in the cluster.

  • Consider including screenshots, error messages, and/or other helpful visuals

  • What version are you on? Are you using self-managed or GitLab.com?

    • GitLab (Hint: /help): 14.0.5-ee
    • Runner (Hint: /admin/runners): ranges from 13.7.0 to 14.0.0
  • Add the CI configuration from .gitlab-ci.yml and other configuration if relevant (e.g. docker-compose.yml)
    N/A

  • What troubleshooting steps have you already taken? Can you link to any docs or other resources so we know where you have been?

I have read the documentation to see if this question could be answered. I have also searched this forum but could not find the answer.

Docs: The Kubernetes executor | GitLab

Thanks for taking the time to be thorough in your request, it really helps! :blush: