Gitlab-ce/buster upgrade 12.10.2 to 13.0.5-ce - brings error 502

hi,

i upgraded my gitlab today, and now i does not become it back working.

># cat /etc/apt/sources.list.d/gitlab_gitlab-ce.list
# this file was generated by packages.gitlab.com for
# the repository at https://packages.gitlab.com/gitlab/gitlab-ce

deb https://packages.gitlab.com/gitlab/gitlab-ce/debian/ buster main
deb-src https://packages.gitlab.com/gitlab/gitlab-ce/debian/ buster main


># apt update
...
gitlab-ce/buster 13.0.5-ce.0 amd64 [upgradable from: 12.10.2-ce.0]
...


># apt upgrade gitlab-ce
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  libicu64 linux-image-4.19.0-5-amd64
Use 'apt autoremove' to remove them.
  * runit_service[gitlab-workhorse] action restart (up to date)
Recipe: monitoring::gitlab-exporter
  * runit_service[gitlab-exporter] action restart (up to date)
Recipe: monitoring::prometheus
  * execute[reload prometheus] action run (skipped due to only_if)

Running handlers:
Running handlers complete
Chef Client finished, 77/772 resources updated in 45 seconds
gitlab Reconfigured!
Restarting previously running GitLab services
ok: run: alertmanager: (pid 12044) 1s
ok: run: gitaly: (pid 12061) 1s
ok: run: gitlab-exporter: (pid 12025) 1s
ok: run: gitlab-workhorse: (pid 11983) 2s
ok: run: grafana: (pid 12079) 0s
ok: run: logrotate: (pid 12172) 1s
ok: run: node-exporter: (pid 12178) 0s
ok: run: postgres-exporter: (pid 12185) 0s
ok: run: postgresql: (pid 11679) 3050185s
ok: run: prometheus: (pid 12194) 1s
ok: run: redis: (pid 11737) 43s
ok: run: redis-exporter: (pid 12213) 0s
ok: run: sidekiq: (pid 12221) 1s

     _______ __  __          __
    / ____(_) /_/ /   ____ _/ /_
   / / __/ / __/ /   / __ `/ __ \
  / /_/ / / /_/ /___/ /_/ / /_/ /
  \____/_/\__/_____/\__,_/_.___/


Upgrade complete! If your GitLab server is misbehaving try running
  sudo gitlab-ctl restart
before anything else.
If you need to roll back to the previous version you can use the database
backup made during the upgrade (scroll up for the filename).




># gitlab-ctl restart
ok: run: alertmanager: (pid 12763) 1s
ok: run: gitaly: (pid 12781) 0s
ok: run: gitlab-exporter: (pid 12811) 0s
ok: run: gitlab-workhorse: (pid 12833) 1s
ok: run: grafana: (pid 12847) 0s
ok: run: logrotate: (pid 12943) 1s
ok: run: node-exporter: (pid 12949) 0s
ok: run: postgres-exporter: (pid 12955) 1s
ok: run: postgresql: (pid 12981) 0s
ok: run: prometheus: (pid 12992) 1s
ok: run: puma: (pid 13011) 0s
ok: run: redis: (pid 13018) 1s
ok: run: redis-exporter: (pid 13024) 0s
ok: run: sidekiq: (pid 13035) 0s

Browser:
502 - Whoops, GitLab is taking too much time to respond.

i searched the logs, but can not find a error.

i removed the “unicorn” entries in config and added puma:

puma['enable']
puma['worker_timeout'] = 60
puma['worker_processes'] = 2
puma['listen'] = '127.0.0.1'
puma['port'] = 8090
puma['socket'] = '/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket'
puma['pidfile'] = '/opt/gitlab/var/puma/puma.pid'
puma['log_directory'] = "/var/log/gitlab/puma"
puma['per_worker_max_memory_mb'] = 500

i proxy gitlab on a own https nginx.

i use gitlab-ce since years, moved it to new debian buster, always found the update problems in the good docs.
But this time i have no idea where is the problem and where i can search.

what can i check to find the problem?

best regards,
daniel

i re-read update path.

looks i need first go to 12.10.6.

will test this…

now i wanted to go back to old version and recover update.

i first updated to 13.1.1 - and this already fixed my problem.

its back running :slight_smile: