Migration gitlab version 14.6.1 from Ubuntu-18.04 to Ubuntu-22.04

I’m migrating GitLab version 14.6.1 from Ubuntu 18.04 to a new server running Ubuntu 22.04. It’s crucial to install the same GitLab version for the restoration of the backup. However, the earliest available package for Ubuntu 22.04 is GitLab CE 15.5, which doesn’t match the required version. As a result, I couldn’t perform the omnibus package installation. Instead, I manually downloaded the package and installed it from the link @iwalker comment before; (ubuntu/focal/gitlab-ce_14.6.1-ce.0_amd64.deb - gitlab/gitlab-ce · packages.gitlab.com)

During the restoration process using the command “gitlab-rake gitlab:backup:restore BACKUP=17059273532_2024_01_22_14.6.1”, I encountered an error stating

“/usr/bin/gitlab-rake error: could not load /opt/gitlab/etc/gitlab-rails-rc
Either you are not allowed to read the file, or it does not exist yet.
You can generate it with: sudo gitlab-ctl reconfigure”

However, even after copying “gitlab-rails-rc” from the old GitLab instance, I encountered another error:

chpst: fatal: unable to switch to directory: /opt/gitlab/etc/gitlab-rails/env: file does not exist

I’m uncertain if manually copying the missing file is the correct approach, or if there’s an alternative method to migrate the old GitLab version to the latest Ubuntu server. Any guidance on resolving these errors or alternative migration methods would be appreciated.

Like @iwalker says, you should have run sudo gitlab-ctl reconfigure, and the message from gitlab-rake even suggested that to generate /opt/gitlab/etc/gitlab-rails-rc instead of just copying it from an existing installation. The reconfigure might fix more issues, so you generally have to be very sure that you know what you’re doing to ignore advice like that and “fix” it by doing something else.
Looking at (one of) our GitLab servers here, that file doesn’t contain that much, but it might depend on your setup, and if something is even slightly different between you old and your new server, copying that file from the old to the new might have caused the restore process to do something silly (or worse: harmful) on the new. The safe way forward is probably to start over.

With the time you’ve already spend and the uncertainty if the packages for 20.04 will ever work on 22.04, I think it would be wiser to follow a more standard approach and back up the existing server (in case of problems) and upgrade that to something that can more easily be migrated to a newer Ubuntu.

1 Like

Before you do a restore, you should run gitlab-ctl reconfigure at least once. It perhaps looks like this wasn’t done. You cannot just copy files from the old one to the new one when it doesn’t exist. It doesn’t exist for a reason, because reconfigure most likely wasn’t run first.

Restore procedure in summary are:

  1. Copy /etc/gitlab/gitlab.rb and /etc/gitlab/gitlab-secrets.json to new server in /etc/gitlab.
  2. Run gitlab-ctl reconfigure.
  3. Copy backup file to new server under /var/opt/gitlab/backups.
  4. Ensure correct permissions on backup file.
  5. Run gitlab restore process as per the docs ensuring the required services are stopped first (not all of them, since the database needs to be active for restore to take place).

Generally, whilst packages for Ubuntu 20.04 may work on Ubuntu 22.04 it is not guaranteed since dependencies are different. If the problem has occurred because of this, then you would need to upgrade your old Gitlab instance to the first version of Gitlab available for Ubuntu 22.04 which means 15.5. At this point, make a backup and copy it to the new server and run the restore.

Hard to tell without knowing the exact steps you followed when restoring. Since /etc/passwd in the output above shows that the git user doesn’t exist, it sounds like you didn’t do the procedures correctly and you need to start from clean. What you should have done:

  1. Install gitlab package
  2. Copy /etc/gitlab/gitlab.rb and /etc/gitlab/gitlab-secrets.json to new server under /etc/gitlab.
  3. Run gitlab-ctl reconfigure to get a basic system ready and configured with all the users under /etc/passwd, as well as a working postgres database.
  4. Start the restore procedure as per the Gitlab docs.

If you did it like this, then /etc/passwd as well as /etc/group would have been populated with all the required users/groups for Gitlab.