Upgrade from 15.5.3 to 15.6.1 resulted in 500s, timeouts,

Recently upgraded from 15.5.3 to 15.6.1 and ran into problems the following day with CI/CD pipeline failures, slow git commands, 500s, … By looking at the output of gitlab-ctl tail specifically in production.log I could see activerecord connection failures. When I connected to the database I found over 190+ connections peaking at 202 when I saw failures. I haven’t made too many changes to gitlab.rb and happen to be running on a 4 socket machine with 14 cores/socket and 2 threads/core so I had 112 puma workers. Explicitly setting the number of puma workers reduced the number of connections to the database and the problem was resolved. The limits on connections is 200. It seems that something changed between 15.5.3 and 15.6.1 leading to connection exhaustion. GitLab does not seem to select a reasonable number of workers and I think what is shown in gitlab.rb is a bit confusing. It is also a bit tedious to have to go between gitlab.rb the the gitlab.rb template and keep up with all of the changes. It would be much easier if we have gitlab.rb with the default and an overrides file.