Problem Upgrade from 12.6 to 12.10

Have run the upgrade of GitLab EE from 12.6 to 12.10

Have installed GitLab EE on SUSE Linux Enterprise Server 12 SP3
gitlab-ee-12.6.2-ee.0.sles12.x86_64
Upgraded to gitlab-ee-gitlab-ee-12.10.14-ee.0.sles12

Our GitLab did not work after the upgrade
502 Whoops, GitLab is taking too much time to respond.

I even tried after the upgrade to run
gitlab-ctl restart
gitlab-ctl reconfigure

No errors from either of these

Found this in /var/log/gitlab/gitlab-workhorse/current

{"correlation_id":"17uPNoDlPY8","duration_ms":0,"host":"gitlab.company.com","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":502,"system":"http","time":"2020-09-17T14:43:25+02:00","uri":"/users/sign_in","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.69 Safari/537.36","written_bytes":2940}
{"correlation_id":"GRU2XS8rKX5","duration_ms":0,"error":"badgateway: failed to receive response: dial unix /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: connect: connection refused","level":"error","method":"GET","msg":"error","time":"2020-09-17T14:43:25+02:00","uri":"/favicon.ico"}

From /var/log/gitlab/gitlab-rails/production.log

Cached record for ApplicationSetting couldn't be loaded, falling back to uncached record: FATAL:  the database system is starting up
...
Cached record for ApplicationSetting couldn't be loaded, falling back to uncached record: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?

Edit:
Not sure how, but GitLab is now running, and I am able to open the UI.
Strange though.

It could be that after the restart, it was doing some stuff in the background and taking some time to start up.

I’ve had some upgrades where they take a while to start and have to wait a few minutes, and sometimes it refreshes much quicker.

Also once or twice when I’ve not had success with:

gitlab-ctl restart

I’ve used:

systemctl restart gitlab-runsvdir.service

as well, and then my system was running fine again. You can also check if background migrations are happening, as you’ll need these to finish before upgrading further:

gitlab-rails runner -e production ‘puts Gitlab::BackgroundMigration.remaining’

1 Like