Hi everyone, I am running into a rate-limit issue and just seem to cannot figure out what I missed to configure.
Problem to solve
I am running gitlab-triage on a cronjob to annotate our issue, change assignees, etc. This has all been working fine for a long time. Earlier this year, we upgraded to GitLab EE with a Premium plan. I don’t want to blame the problem on that but I think the issue exists roughly since that time.
Every time I run gitlab-triage now, it runs into rate limit responses. Not only that but it doesn’t seem to be able to fetch a retry-after time, so it just doesnt work properly. I couldn’t really find similar reports on their repo, so I figured it’s probably not a bug but I just missed something in my rate limit settings.
This is how my gitlab-triage --debug output looks:
----------------------------------------------------------------------------------------------
Gathering resources for rule: **If a closed issue still has an assignee, remove the assignee**
----------------------------------------------------------------------------------------------
.[DEBUG] query_api: https://sgit/api/v4/groups/development/issues?per_page=100&state=closed
[DEBUG] rate_limit_infos: Rate limit remaining: 0 (reset at 1970-01-01 00:00:00 +0000)
[DEBUG] Rate limit almost exceeded, sleeping for -1711446293.7551877 seconds
.[DEBUG] query_api: https://sgit/api/v4/groups/development/issues?per_page=100&state=closed&page=2
[DEBUG] rate_limit_infos: Rate limit remaining: 0 (reset at 1970-01-01 00:00:00 +0000)
[DEBUG] Rate limit almost exceeded, sleeping for -1711446294.314207 seconds
.[DEBUG] query_api: https://sgit/api/v4/groups/development/issues?per_page=100&state=closed&page=3
...
However, GitLab returns a 200, not a 429, which I would expect for a rate limit:
==> /var/log/gitlab/nginx/gitlab_access.log <==
172.17.0.1 - - [26/Mar/2024:09:32:25 +0000] "GET /api/v4/groups/development/issues?per_page=100&state=closed&page=2 HTTP/1.1" 200 46640 "" "GitLab Triage 1.42.2" 5.82
==> /var/log/gitlab/nginx/gitlab_access.log <==
172.17.0.1 - - [26/Mar/2024:09:32:26 +0000] "GET /api/v4/groups/development/issues?per_page=100&state=closed&page=3 HTTP/1.1" 200 52023 "" "GitLab Triage 1.42.2" 5.55
The visible effect of this is that most jobs of gitlab-triage are simply not working anymore. I have tried setting issue API rate limits, user rate limits, excluded the specific user from rate limits, etc, pretty much everything on admin/application_settings/network but the issue persists.
So my question is, how do I increase the rate limit of the /api/v4/groups/*/issues endpoint?
Steps to reproduce
- Updated
gitlab-triageto the latest version (1.42.2) - Worked through this whole docs page and attempted to tweak every documented setting.
Configuration
Versions
Please select whether options apply, and add the version information.
- Self-managed
- GitLab.com SaaS
Versions
- GitLab (Web:
/helpor self-managed system information):v16.10.0-ee
Thanks in advance for any replies! 