All Gitlab runners on pending after update (gitlab-ee)

Hello,

I’ve updated our own Gitlab-EE and Gitlab-Runner instance, and now all our jobs are on pending.
gitlab-ee: 17.9.2 > 17.10.3
gitlab-runner: 17.9.1-1 > 17.10.1-1

According to gitlab-runner status everything is running. And all the runners are up in the interface.

Could someone help me out?

Thank you

1 Like

Hi, do you see the new runners on your GitLab projects?

I’ve added a new runner and this runner is picking up the job. What could it be?

I don’t know :sweat_smile: But we have to get more information to understand the problem.
I think you are administrator of your GitLab instance, can you see the runner on the runner page?
When you go to a project, you see the new runner on the runner page?

Haha I understand.

Yes correct, I’m the admin. I can see the old AND new runner on the project page. Both runners are ‘Green’ and Online.

#10 is the old one.

If I’m verifying the runners via CLI, this is the output:

I’ve recreated new runners and re-registered them via CLI. They are now working properly.
Still don’t know why I had this issue after the gitlab-ee / gitlab-runner update.

If they are working, it’s good :flexed_biceps:

But I don’t know why they didn’t work after update. Do you see the migration page of this version? maybe something is present

Can you share a screenshot of such a pending pipeline/job?

What does the CI/CD jobs configuration look like, which job tags are used and do they match the tags that the runners provide?

I have the same issue and logs contain the same message /api/v4/jobs/request: 404 Not Found (404 Not found)

Any updates on how to solve this?

Thanks

Did you find a solution to this? We updated GitLab a couple of days ago. We had to upgrade our Windows runners — no big deal, they’re working now — but we’re seeing the same 404 from our Linux runners. We re-installed, reset the API key, etc.

Hey all,
Not sure what caused this, but I ‘fixed’ it with new runners. Deleted all old runners afterwards.

Please see Runner not picking up jobs after upgrade to 17.11 (#543351) · Issues · GitLab.org / GitLab · GitLab for details.

1 Like

GitLab 17.10.7 and 17.11.3 were released today to fix this issue: GitLab Patch Release: 18.0.1, 17.11.3, 17.10.7 | GitLab.

GitLab 18.0.1 was also released, but upgrading to GitLab 18.0.0 should have also fixed the issue.

3 Likes