Seeking Upgrade Advise

Seeking Advise for upgrading our current, and very out of date, version of Gitlab to latest stable

Get current Gitlab info

sudo gitlab-rake gitlab:env:info

System information
System: Ubuntu 20.04
Current User: git
Using RVM: no
Ruby Version: 2.6.3p62
Gem Version: 2.7.9
Bundler Version:1.17.3
Rake Version: 12.3.3
Redis Version: 3.2.12
Git Version: 2.24.1
Sidekiq Version:5.2.7
Go Version: unknown

GitLab information
Version: 12.6.1
Revision: a0094541f87
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: PostgreSQL
DB Version: 10.9
URL: http://gitlabhost
HTTP Clone URL: http://gitlabhost/some-group/some-project.git
SSH Clone URL: git@gitlabhost:some-group/some-project.git
Using LDAP: no
Using Omniauth: yes
Omniauth Providers:

GitLab Shell
Version: 10.3.0
Repository storage paths:

  • default: /var/opt/gitlab/git-data/repositories
  • GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
  • Git: /opt/gitlab/embedded/bin/git

Anything that you could advise about the upgrade plan? Would you perform this in any other way giving the old version of the program?

Upgrading

  1. Backup
  2. Stop Server
  3. Update Ruby
  4. Update Node.js
  5. Update Go
  6. Update Git
  7. Update PostgreSQL (
  8. including extensions:
    1. pg_trgm - Gitlab version 8.6 → Done
    2. btree_gist → Gitlab version 13.1 → Needed
    3. plpgsql → Gitlab 11.7 → Done
  9. Config files review
  10. Install Libraries
  11. Update Gitlab Shell
  12. Update Gitlab Workhorse
  13. Update Gitaly
  14. Update Gitlab Pages
  15. Start Application
  16. Check Application status
  17. Optional Step if Product Documentation was installed with the package
  18. Review version specific upgrading instructions
    Ref: Upgrade GitLab by using the GitLab package | GitLab

Are you 100% sure this is a source installation, and not the Gitlab omnibus package? Can you do:

dpkg -l | grep -i gitlab

just to verify if a package is installed or not, as I would normally expect source installs to be in a different location based on the info you provided so far.

Hi iwalker,
I am positive it is a omnibus CE install, sorry if I made you think otherwise, I have updated the inital post, there was misleading info that I found in Gitlab for self-compiled instances no related to ours, my bad

dpkg -l | grep -i gitlab
ii gitlab-ce 12.6.1-ce.0 amd64 GitLab Community Edition (including NGINX, Postgres, Redis)
ii gitlab-runner 14.2.0 amd64 GitLab Runner
ii node-hosted-git-info 2.8.5-1 all Provides metadata from Github, Bitbucket and Gitlab

I am not sure if I need to proceed with the whole upgrade one by one, or the install will take care of it:

  1. Update Ruby
  2. Update Node.js
  3. Update Go
  4. Update Git
  5. Update PostgreSQL …

I am not sure about PostgreSQL, I thought would be manual upgrade first

What I am sure is that I will need to make intermediate upgrades between major versions:
E.g Version: 12.6.1 → latest 12.. / 13 to latest etc

Ah, your initial post had links to source upgrades, hence why I asked. Also you mention updating ruby, etc, which is simply not needed with an omnibus installation, since Gitlab takes care of updating it’s components as and when it’s needed.

Suggest you start here, this link has a ton of resources to look at including the upgrade path - since you will need to follow the upgrade path to ensure a successful upgrade: GitLab Upgrade Path Resources

You do not need to stop Gitlab, if you do this, services required for the upgrade process will not be running, and things like database migrations will not occur. Gitlab will stop any services it needs to stop during the upgrade process. Postgresql is also bundled in the omnibus package, and Gitlab itself will take care of upgrading this when it’s needed.

You also need to make sure background migrations have finished before starting the next upgrade on the upgrade path. If you don’t, and upgrade before they have finished, you will break your Gitlab installation.

2 Likes

Thanks for your replies, even that will need few jumps the process now does not look so painful :slight_smile:

1 Like

Nope, not really painful. A bit time consuming to go through each upgrade. The package update itself takes no more than 5 mins, but background migrations depending on size and amount of data on your Gitlab install can take hours - I’ve had one install I had to wait 8 hours before I could start the next upgrade.

You are basically just using apt commands to upgrade the gitlab-ce package that is installed. Nothing else needs to be done other than this, since this package contains all the updates things like ruby, postgres, etc as and when required. To get to latest 17.x, you are prob looking at a few days at best. I doubt you’ll work your way through all upgrades on the upgrade path any quicker than that, again depends how many repositories and data you have.

Obviously, make sure to have backups, check the Gitlab backup documentation for that.

3 Likes

Thank you, very much appreciated

1 Like

Great tool here, even gave us the upgrade path

12.6.8 => 12.7.9 => 12.8.10 => 12.9.10 => 12.10.14 => 13.0.14 => 13.1.11 => 13.2.10 => 13.3.9 => 13.4.7 => 13.5.7 => 13.6.7 => 13.7.9 => 13.8.8 => 13.9.7 => 13.10.5 => 13.11.7 => 13.12.15 => 14.0.12 => 14.1.8 => 14.2.7 => 14.3.6 => 14.4.5 => 14.5.4 => 14.6.7 => 14.7.7 => 14.8.6 => 14.9.5 => 14.10.5 => 15.0.5 => 15.1.6 => 15.2.5 => 15.3.5 => 15.4.6 => 15.5.9 => 15.6.8 => 15.7.9 => 15.8.6 => 15.9.8 => 15.10.8 => 15.11.13 => 16.0.8 => 16.1.6 => 16.2.9 => 16.3.7 => 16.4.5 => 16.5.8 => 16.6.8 => 16.7.8 => 16.8.8 => 16.9.9 => 16.10.8 => 16.11.5

2 Likes

Do not forget to refresh your gitlab repository.

Care to elaborate? I thought would be part of the package upgrade

Other than the standard apt-get update to refresh your repo data, he could well mean something I came across when I was on Gitlab 12.9.3 that I didn’t see any new packages appearing in the repo data. Also recently similar situation in as much that the SSL certificates would not update on previously configured repositories. In both of these situations, it’s enough to do this from the install docs:

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash

the above command is for gitlab-ce so safe for you to run. Otherwise, change gitlab-ce to gitlab-ee in the URL if using Enterprise Edition. This will refresh your repo data, similar to what I experienced when 12.9.3 was the latest I saw available. After running the command, newer versions then showed up and could then continue the upgrade path.

For the SSL error I mentioned when certificates had expired, that command will also help out.

For security reasons, it’s probably best to download the script first and then read it to see what it does before running it like the command above. It just depends on whether you trust the source it comes from. For Gitlab I run it as it (my personal preference, others may disagree with me and that’s fine), for other sites I may not trust, I would probably download and read it first. If you wish to do that, being more security conscious then just do:

wget https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh

after checking the script to make sure it’s not doing anything nefarious:

sudo bash script.deb.sh

or:

chmod +x script.deb.sh
sudo ./script.deb.sh
1 Like

Exactly. Recently I tried to upgrade my Gitlab I think to 16.11.2 version, but I wasn’t allowed to see the new available versions and I had to run the process @iwalker mention above.

1 Like

I just want to thank everyone for all the advise, trying to get scheduled this job, it is our production bread and butter environment for one of the products, so good luck to us

1 Like

Thanks, first hurdle passed

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash

wget --content-disposition https://packages.gitlab.com/gitlab/gitlab-ce/packages/ubuntu/xenial/gitlab-ce_12.6.8-ce.0_amd64.deb/download.deb

sudo dpkg -i gitlab-ce_12.6.8-ce.0_amd64.deb

image
strong text

1 Like

Cool, once background migrations have finished (see Gitlab docs) you can then continue the next upgrade on the upgrade path :+1:

Hello iwalker ,

I have GitLab Community Edition [15.11.0] installed in my system on RHEL 8 and i wanted it to upgrade .Could you Please advise on which version we should upgrade ?appreciate if you can suggest or provide upgrade steps .
reference doc we received for security patch GitLab Critical Patch Release: 17.1.2, 17.0.4, 16.11.6 | GitLab

From a .0 patch release you should first go to the latest release in that minor version, i.e. 15.11.13.

Otherwise you should look at what the Upgrade path tool mentioned earlier says.

Found the first hurdle, even with the omnibus CE package I am having issues with 13.6 and 13.7 upgrades and postgresql, the packages expect version 12.5 but looks like 11.5 still installed

DB location default folder: /var/opt/gitlab/postgresql
Followed instructions here: Database settings | GitLab

Launched:

  • sudo gitlab-ctl reconfigure
  • sudo gitlab-ctl pg-upgrade
    No errors found but the dashboard still showing version 11.7!

Any thoughts?? At the moment stopped the upgrades

Sorted, panic mode silenced

I just run sudo gitlab-ctl restart and the version got upgraded in the dashboard! Simples (Phew!)

Anyone have seen these warnings before?

The version of the running redis service is different than what is installed.
Please restart redis to start the new version.

sudo gitlab-ctl restart redis

[2024-08-02T15:43:11+02:00] WARN: This release of Cinc Client became end of life (EOL) on May 1st 2023. Please update to a supported release to receive new features, bug fixes, and security updates.

For redis I am launching sudo gitlab-ctl restart redis but the warning stays, the other one I don’t really know what to do with it