We are seeing incredibly slow load times on Merge Requests that have a lot of comments (400+). This may be related to (gitlab:11.4.5/11.4.7 discussions.json response is very slow ！ (#25380) · Issues · GitLab.org / GitLab · GitLab over 5 years old but still open) (and a few more here Issues · GitLab.org / GitLab · GitLab). Our comments have a fair number of diffs as well which may help diagnose the issue.
Wondering if anyone has found a solution or work-around for this? Here is a profiled sample, showing how long each segment of comments takes to load for a total of almost 1 minute to load the page entirely.
| 6912ms | /group/project/-/merge_requests/4729/discussions.json?notes_filter=0&persist_filter=false&per_page=100&cursor=eyJjcmVhdGVkX2F0IjoiMjAyNC0wMS0xNSAxMDowMzo1Ny41OTMxMzAwMDAgKzAwMDAiLCJpZCI6IjQ0OTQwMCIsIl9rZCI6Im4ifQ%3D%3D 1.19 MiB |
| 6654ms | /group/project/-/merge_requests/4729/discussions.json?notes_filter=0&persist_filter=false&per_page=100&cursor=eyJjcmVhdGVkX2F0IjoiMjAyNC0wMi0yMSAxMjo1MTo0MS40MjgwMzEwMDAgKzAwMDAiLCJpZCI6IjQ1NTgzOSIsIl9rZCI6Im4ifQ%3D%3D 638.43 KiB |
| 6586ms | /group/project/-/merge_requests/4729/discussions.json?notes_filter=0&persist_filter=false&per_page=68&cursor=eyJjcmVhdGVkX2F0IjoiMjAyMi0wNi0yOSAxMDo1MzozNi4yODAwNDYwMDAgKzAwMDAiLCJpZCI6IjM2NTYxMiIsIl9rZCI6Im4ifQ%3D%3D 498.55 KiB |
| 6203ms | /group/project/-/merge_requests/4729/discussions.json?notes_filter=0&persist_filter=false&per_page=100&cursor=eyJjcmVhdGVkX2F0IjoiMjAyMi0xMi0wNiAxMzoxNDowOS41NTkyNTAwMDAgKzAwMDAiLCJpZCI6IjM4ODY4MCIsIl9rZCI6Im4ifQ%3D%3D 829.83 KiB |
| 5688ms | /group/project/-/merge_requests/4729/discussions.json?per_page=20 199.79 KiB |
| 5411ms | /group/project/-/merge_requests/4729/discussions.json?notes_filter=0&persist_filter=false&per_page=30&cursor=eyJjcmVhdGVkX2F0IjoiMjAyMS0wOS0wNyAxODo0NDowMS43NTkzMzQwMDAgKzAwMDAiLCJpZCI6IjMzMDk2MCIsIl9rZCI6Im4ifQ%3D%3D 326.58 KiB |
| 5159ms | /group/project/-/merge_requests/4729/discussions.json?notes_filter=0&persist_filter=false&per_page=45&cursor=eyJjcmVhdGVkX2F0IjoiMjAyMi0wMS0wNSAxMzo0MDo1NC43MzMyMzkwMDAgKzAwMDAiLCJpZCI6IjM0NTMxNSIsIl9rZCI6Im4ifQ%3D%3D 445.63 KiB |
| 4611ms | /group/project/-/merge_requests/4729/discussions.json?notes_filter=0&persist_filter=false&per_page=20 199.79 KiB |
| 1062ms | /api/graphql 7.44 KiB |
| 729ms | /api/graphql 486 B |
| 502ms | /group/project/-/merge_requests/4729/widget.json 2.13 KiB |
| 468ms | /group/project/-/merge_requests/4729/diffs_metadata.json?diff_head=true&view=inline&w=0 85.60 KiB |
| 434ms | /group/project/-/merge_requests/4729/cached_widget.json 20.45 KiB |
| 401ms | /api/graphql 940 B |
| 378ms | /api/graphql 1.12 KiB |
| 358ms | /api/graphql 32.02 KiB |
| 319ms | /group/project/-/merge_requests/4729/cached_widget.json 20.45 KiB |
| 313ms | /api/graphql 638 B |
| 256ms | /group/project/-/merge_requests/4729/widget.json 2.13 KiB |
| 230ms | /api/graphql 612 B |
Hardware
RAM: 64GB
vCPU: 16
System information
System: Ubuntu 22.04
Current User: git
Using RVM: no
Ruby Version: 3.1.4p223
Gem Version: 3.3.26
Bundler Version:2.5.4
Rake Version: 13.0.6
Redis Version: 6.2.9
Sidekiq Version:7.1.6
Go Version: go1.20.8 linux/amd64
GitLab information
Version: 16.8.6
Revision: eeba2d09540
Directory: /home/git/gitlab
DB Adapter: PostgreSQL
DB Version: 14.11
URL: https://gitlab.example.com
HTTP Clone URL: https://gitlab.example.com/some-group/some-project.git
SSH Clone URL: git@gitlab.example.com:some-group/some-project.git
Using LDAP: no
Using Omniauth: yes
Omniauth Providers: example
GitLab Shell
Version: 14.33.0
Repository storages:
- default: unix:/home/git/gitlab/tmp/sockets/private/gitaly.socket
GitLab Shell path: /home/git/gitlab-shell
Gitaly
- default Address: unix:/home/git/gitlab/tmp/sockets/private/gitaly.socket
- default Version: 14.8.0-rc1-7222-g6cd77cc07e
- default Git Version: 2.42.0
Our dev team is struggling greatly to use these pages effectively when discussing and working through merge requests on calls.