Yes, yet another one of these posts.
I am using gitlab-ce on debian buster using the gitlab repository/omnibus. While using apt my installation was upgraded from 14.0.x to the latest version (14.2.3). As a result I only got HTTP/502 on the WebUI. I rolled back and manuall< installed all incremental upgrades which narrowed the problem down to the upgrade 14.1.4 → 14.2.0.
After the Upgrade I’ve got the following envigonment:
Ruby: ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux] GitLab: 14.2.0 (d678b7c987f) FOSS GitLab Shell: 13.19.1 PostgreSQL: 13.3
(Postgres 12.x was manually updated in this test, 12.x showed the same error)
gitlab-ctl upgrade and
gitlab-rake db:migrate all finish without errors,
gitlab-ctl status also doesn’t show any errors.
gitlab-rake db:migrate:status confirms all migrations to be “up”. As said WebUI is responding but stuck at 502. I see puma and sidekiq both running at 100% CPU and changing PIDs, so probably crashing and respawning. I tail -f’d all *log-files in /var/log/gitlab and did only get some startup messages, but no errors.
Looking for background tasks using
gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' (or any other gitlab-rails-command like
gitlab-rails console) yields:
[...]file /opt/gitlab/embedded/service/gitlab-rails/lib/migrate.rb to define constant Migrate, but didn't (Zeitwerk::NameError).
I’m not quite sure where to go from here. As far as I can tell all migrations should be completed, so why exactly is rake crashing at the migration stage? Any ideas or tips where I can look next?`