Which setting affects the issue list api rate limit? (gitlab-triage runs into rate limit)

Hi everyone, I am running into a rate-limit issue and just seem to cannot figure out what I missed to configure.

Problem to solve

I am running gitlab-triage on a cronjob to annotate our issue, change assignees, etc. This has all been working fine for a long time. Earlier this year, we upgraded to GitLab EE with a Premium plan. I don’t want to blame the problem on that but I think the issue exists roughly since that time.

Every time I run gitlab-triage now, it runs into rate limit responses. Not only that but it doesn’t seem to be able to fetch a retry-after time, so it just doesnt work properly. I couldn’t really find similar reports on their repo, so I figured it’s probably not a bug but I just missed something in my rate limit settings.

This is how my gitlab-triage --debug output looks:

----------------------------------------------------------------------------------------------
Gathering resources for rule: **If a closed issue still has an assignee, remove the assignee**
----------------------------------------------------------------------------------------------
.[DEBUG] query_api: https://sgit/api/v4/groups/development/issues?per_page=100&state=closed
[DEBUG] rate_limit_infos: Rate limit remaining: 0 (reset at 1970-01-01 00:00:00 +0000)
[DEBUG] Rate limit almost exceeded, sleeping for -1711446293.7551877 seconds
.[DEBUG] query_api: https://sgit/api/v4/groups/development/issues?per_page=100&state=closed&page=2
[DEBUG] rate_limit_infos: Rate limit remaining: 0 (reset at 1970-01-01 00:00:00 +0000)
[DEBUG] Rate limit almost exceeded, sleeping for -1711446294.314207 seconds
.[DEBUG] query_api: https://sgit/api/v4/groups/development/issues?per_page=100&state=closed&page=3
...

However, GitLab returns a 200, not a 429, which I would expect for a rate limit:

==> /var/log/gitlab/nginx/gitlab_access.log <==
172.17.0.1 - - [26/Mar/2024:09:32:25 +0000] "GET /api/v4/groups/development/issues?per_page=100&state=closed&page=2 HTTP/1.1" 200 46640 "" "GitLab Triage 1.42.2" 5.82
==> /var/log/gitlab/nginx/gitlab_access.log <==
172.17.0.1 - - [26/Mar/2024:09:32:26 +0000] "GET /api/v4/groups/development/issues?per_page=100&state=closed&page=3 HTTP/1.1" 200 52023 "" "GitLab Triage 1.42.2" 5.55

The visible effect of this is that most jobs of gitlab-triage are simply not working anymore. I have tried setting issue API rate limits, user rate limits, excluded the specific user from rate limits, etc, pretty much everything on admin/application_settings/network but the issue persists.

So my question is, how do I increase the rate limit of the /api/v4/groups/*/issues endpoint?

Steps to reproduce

  • Updated gitlab-triage to the latest version (1.42.2)
  • Worked through this whole docs page and attempted to tweak every documented setting.

Configuration




Versions

Please select whether options apply, and add the version information.

Versions

Thanks in advance for any replies! :slight_smile: