Upgrading GitLab from 6.2 to 8.5

We are currently on GitLab 6.2.1-0 version and we would like to upgrade to 8.5 to get the newest features. Is it ok to upgrade straight to 8.5 or should we take small steps? We are not using GitLab CI FYI.

Appreciate your inputs

What is you installation type Omnibus or Source method?

Plz give O/P of
(omnibus)
sudo gitlab-rake gitlab:env:info --trace
(Source method)
sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production

Note: u can erase ur server hostname from the o/p for security reasons.

Looks like the installation was done by Bitnami method by an ex-employee
I am looking for an easier installation method for future upgrades. Is it easier to do by Omnibus installer?

System information
System:         CentOS 5.8
Current User:   git
Using RVM:      no
Ruby Version:   1.9.3p448
Gem Version:    1.8.24
Bundler Version:1.3.5
Rake Version:   0.9.2.2

GitLab information
Version:        6.2.1
Revision:       f2a77a8
Directory:      /opt/gitlab-6.2.1-0/apps/gitlab/htdocs
DB Adapter:     mysql2
URL:            http://xx.xxx.com
HTTP Clone URL: http://xx.xxxx.com/some-project.git
SSH Clone URL:  git@xxxx.xxxx.com:some-project.git
Using LDAP:     no
Using Omniauth: no

GitLab Shell
Version:        1.7.1
Repositories:   /opt/gitlab-6.2.1-0/apps/gitlab/repositories/
Hooks:          /opt/gitlab-6.2.1-0/apps/gitlab/gitlab-shell/hooks/
Git:            /opt/gitlab-6.2.1-0/git/bin/git

Hi

Sorry for the delayed response. Thanks for providing the info. Few things before embarking on the Upgrade.

  • The upgrade is a giant leap from 6.2 to 8.5. There are a lot of changes between 6.2 to 8.5. U can see the Change Log
  • Multiple steps are need to upgrade to latest, direct upgrade is not possible.
  • Gitlab backup can be restored on the same version of Gitlab version. Eg: 6.9 backup will go in to 6.9 only.
  • In Source code method of installation there was an option to use Mysql db, in Omnibus method installation Postgresql is used.
  • I recommend you to change to Omnibus method because it is very simple, easy to maintain, easy to manage upgrades.
  • In source code method of installation if you have used mysql DB, during conversion to Omnibus method u might need to convert it using Mysql to Postgresql converter.

Since your installation is on source method Gitlab 6.2. Please try to do the following.

  1. Upgrade your Gitlab (source method) from 6.2 to 6.6 in source method.
  2. Take a backup from the Gitlab 6.6, now you have Gitlab backup (source method)
  3. I assume your installation has Mysql database as back end. (check it in /home/git/gitlab/config/database.yml file)
  4. Now if you extract the backup file you will find db/, repositories/, uploads/ and backup_information.yml
  5. You need to remove the database.sql file from db/ folder and replace it with a postgresql database sql file.
  6. Please use this link to convert your Mysql db file to Postgress sql , carefully read and follow the steps in this link and convert your DB file. Link: Mysql2PostgresqlConvertion
  7. Check the ownership and permissions of the db file,
  8. From step 6 you got your Postgre compatible db file, just replace it in the db directory.
  9. Change the permission and ownership of the file to git , (chmod 644/700 database.sql, chow git:git database.sql)
  10. Compress the all the files once again with same file name. (tar -cvf 1234XX_gitlab_backup.tar db uploads repos backup_information.yml)
  11. Download Gitlab 6.6 omnibus rpm from Gitlab Archives
  12. Restore the backup on Gitlab 6.6. Now your installation is Omnibus Gitlab.
  13. Take small steps upgrade your Gitlab in following steps. Upgrade Instructions
  14. Gitlab 6.6 to Gitlab 6.9

14 A. Till 6.9 GitLab creates git user as Normal user , ie., userid < 500 (RHEL/CentOS), after 7.1 we need to change git user type to system user, other wise git clone problem would arise.
14 B. run following commands to modify user ;
groupmod -g 496 git (chose available user id if 496 is not available)
usermod -u 496 -g 496 -d /var/opt/gitlab -s /bin/sh git
chown 496:496 gitlab-shell.log
chown git:git application.log
chown git:git production.log
chown git:git satellites.log
chmod 0666 production.log

  1. Gitlab 6.9 to Gitlab 7.1
  2. Gitlab 7.1 to Gitlab 7.4
  3. Gitlab 7.4 to Gitlab 7.6
  4. Gitlab 7.6 to Gitlab 8.0
  5. Gitlab 8.0 to Gitlab 8.3
  6. Gitlab 8.3 to Gitlab 8.5
  7. The reason for multiple steps is, there are multiple db changes between the versions, during upgrade db migration will fail, due different db errors.
  8. Run Rake tests in between every steps to check the health of the gitlab installation.
    Rake tests:
    sudo gitlab-rake gitlab:env:info --trace
    sudo gitlab-rake gitlab:gitlab_shell:check --trace
    sudo gitlab-rake gitlab:sidekiq:check --trace
    sudo gitlab-rake gitlab:app:check --trace
    sudo gitlab-rake db:migrate:status --trace
    sudo gitlab-rake gitlab:check SANITIZE=true --trace
    (try to solve all issues before progressing to next step of upgrade, u can ignore this error which comes every time )
  9. Configure your HTTPS/LDAP etc if needed.
  • During all these steps you will encounter many errors and issues which you need to solve by yourself and take help from Internet/Gitlab Forum.
  • By the end of this process you will get a lot of information about gitlab.
  • I suggest you to try this on a test server and then do this upgrade on the actual server.
  • All the steps are based on my experience of upgrading our Gitlab installation from Gitlab 6.9 source method to 8.5 Omnibus method with over 1200 users and 50GB user data for our client,
    Good Luck !!

Edited: Added Steps 14 A & B

3 Likes

Hi Ershad,

Apologies for the delayed reply. I highly appreciate you providing detailed steps based on your experience. The upgrade process definitely looks not that easy. We do not have a whole lot of user base, hardly 20 active users and around 10GB of data. If we do not care of merge requests history, but of course the commit history is very important, is it a safer option to spin up gitlab 8.5 version on a different box and port the git repositories over to the new server. I am sure if we were to port the entire old data to the newer version, system will go mad. Another reason of asking this is to implement a cleaner install using chef / ansible provisioning system, so it can make the future upgrades manageable.

Hi

For making the further Upgrades smoother and easily manageable the system has to be converted to Omnibus method of Installation.
Yes, Indeed it is a difficult task to convert it to Omnibus and upgrade it to Latest version.
Yes it is indeed safe to convert and upgrade to Latest, all the data will be restored with out any loss.
I have already told you try this on a separate system and upon success you can do it on the actual server.
10 GB data is not too large, the system could handle it during the backup and restore you could do it without problem.

@sandeepa8387

  1. Follow this guide to upgrade from 6.2 to 7.14: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/update/6.x-or-7.x-to-7.14.md#L16
  2. On 7.14, convert to OmniBus by following this guide: http://doc.gitlab.com/omnibus/update/README.html#upgrading-from-a-non-omnibus-installation-to-an-omnibus-installation
  3. Once you are on 7.14 OmniBus, you can cleanly update to latest without any hickups using your OS package manager in this case, yum by running an update and upgrade

Thanks Ershad and Kelvin, I will try out your recommendations and keep posted how that goes, thanks much…

Hi Ahmad,
I needed help on gitlab CE upgrade. My installation is source method with MySQL. I upgraded from 6.2 to 7.14. The issue is I don’t see these two tables - user_projects and user_groups in upgraded db. I was looking for online documentation to see if that’s expected or my db migration was not clean.

Karan

Hi Karan

Please run this following command to find out if your db migration was successful.
(omnibus installation)
$sudo gitlab-rake db:migrate:status --trace

Thanks Ahmad.

Below is the trace op.

# sudo -u git -H bundle exec rake db:migrate:status RAILS_ENV=production --trace
** Invoke db:migrate:status (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke db:load_config (first_time)
** Execute db:load_config
** Execute db:migrate:status

database: gitlabhq_production

 Status   Migration ID    Migration Name
--------------------------------------------------
   up     20110913200833  ********** NO FILE **********
   up     20110913204141  ********** NO FILE **********
   up     20110914221600  ********** NO FILE **********
   up     20110915205627  ********** NO FILE **********
   up     20110915213352  ********** NO FILE **********
   up     20110916123731  ********** NO FILE **********
   up     20110916162511  ********** NO FILE **********
   up     20110917212932  ********** NO FILE **********
   up     20110921192501  ********** NO FILE **********
   up     20110922110156  ********** NO FILE **********
   up     20110923211333  ********** NO FILE **********
   up     20110924214549  ********** NO FILE **********
   up     20110924215658  ********** NO FILE **********
   up     20110926082616  ********** NO FILE **********
   up     20110927130352  ********** NO FILE **********
   up     20110928140106  ********** NO FILE **********
   up     20110928142747  ********** NO FILE **********
   up     20110928161328  ********** NO FILE **********
   up     20111005193700  ********** NO FILE **********
   up     20111009101738  ********** NO FILE **********
   up     20111009110913  ********** NO FILE **********
   up     20111009111204  ********** NO FILE **********
   up     20111015154310  ********** NO FILE **********
   up     20111016183422  ********** NO FILE **********
   up     20111016193417  ********** NO FILE **********
   up     20111016195506  ********** NO FILE **********
   up     20111019212429  ********** NO FILE **********
   up     20111021101550  ********** NO FILE **********
   up     20111025134235  ********** NO FILE **********
   up     20111027051828  ********** NO FILE **********
   up     20111027142641  ********** NO FILE **********
   up     20111027152724  ********** NO FILE **********
   up     20111101222453  ********** NO FILE **********
   up     20111111093150  ********** NO FILE **********
   up     20111115063954  ********** NO FILE **********
   up     20111124115339  ********** NO FILE **********
   up     20111127155345  ********** NO FILE **********
   up     20111206213842  ********** NO FILE **********
   up     20111206222316  ********** NO FILE **********
   up     20111207211728  ********** NO FILE **********
   up     20111214091851  ********** NO FILE **********
   up     20111220190817  ********** NO FILE **********
   up     20111231111825  ********** NO FILE **********
   up     20120110180749  ********** NO FILE **********
   up     20120119202326  ********** NO FILE **********
   up     20120121122616  ********** NO FILE **********
   up     20120206170141  ********** NO FILE **********
   up     20120215182305  ********** NO FILE **********
   up     20120216085842  ********** NO FILE **********
   up     20120216215008  ********** NO FILE **********
   up     20120219130957  ********** NO FILE **********
   up     20120219140810  ********** NO FILE **********
   up     20120219193300  ********** NO FILE **********
   up     20120228130210  ********** NO FILE **********
   up     20120228134252  ********** NO FILE **********
   up     20120301185805  ********** NO FILE **********
   up     20120307095918  ********** NO FILE **********
   up     20120315111711  ********** NO FILE **********
   up     20120315132931  ********** NO FILE **********
   up     20120317095543  ********** NO FILE **********
   up     20120323221339  ********** NO FILE **********
   up     20120329170745  ********** NO FILE **********
   up     20120405211750  ********** NO FILE **********
   up     20120408180246  ********** NO FILE **********
   up     20120408181910  ********** NO FILE **********
   up     20120413135904  ********** NO FILE **********
   up     20120627145613  ********** NO FILE **********
   up     20120706065612  ********** NO FILE **********
   up     20120712080407  ********** NO FILE **********
   up     20120729131232  ********** NO FILE **********
   up     20120905043334  ********** NO FILE **********
   up     20121002150926  ********** NO FILE **********
   up     20121002151033  ********** NO FILE **********
   up     20121009205010  ********** NO FILE **********
   up     20121026114600  ********** NO FILE **********
   up     20121119170638  ********** NO FILE **********
   up     20121120051432  ********** NO FILE **********
   up     20121120103700  ********** NO FILE **********
   up     20121120113838  ********** NO FILE **********
   up     20121122145155  ********** NO FILE **********
   up     20121122150932  ********** NO FILE **********
   up     20121123104937  ********** NO FILE **********
   up     20121123164910  ********** NO FILE **********
   up     20121203154450  ********** NO FILE **********
   up     20121203160507  ********** NO FILE **********
   up     20121205201726  ********** NO FILE **********
   up     20121218164840  ********** NO FILE **********
   up     20121219095402  ********** NO FILE **********
   up     20121219183753  ********** NO FILE **********
   up     20121220064104  ********** NO FILE **********
   up     20121220064453  Init schema
   up     20130102143055  Rename owner to creator for project
   up     20130110172407  Add public to project
   up     20130123114545  Add issues tracker to project
   up     20130125090214  Add user permissions
   up     20130131070232  Remove private flag from project
   up     20130206084024  Add description to namsespace
   up     20130207104426  Add description to teams
   up     20130211085435  Add issues tracker id to project
   up     20130214154045  Rename state to merge status in milestone
   up     20130218140952  Add state to issue
   up     20130218141038  Add state to merge request
   up     20130218141117  Add state to milestone
   up     20130218141258  Convert closed to state in issue
   up     20130218141327  Convert closed to state in merge request
   up     20130218141344  Convert closed to state in milestone
   up     20130218141444  Remove merged from merge request
   up     20130218141507  Remove closed from issue
   up     20130218141536  Remove closed from merge request
   up     20130218141554  Remove closed from milestone
   up     20130220124204  Add new merge status to merge request
   up     20130220125544  Convert merge status in merge request
   up     20130220125545  Remove merge status from merge request
   up     20130220133245  Rename new merge status to merge status in milestone
   up     20130304104623  Add state to user
   up     20130304104740  Convert blocked to state
   up     20130304105317  Remove blocked from user
   up     20130315124931  User color scheme
   up     20130318212250  Add snippets to features
   up     20130319214458  Create forked project links
   up     20130323174317  Add private to snippets
   up     20130324151736  Add type to snippets
   up     20130324172327  Change project id to null in snipepts
   up     20130324203535  Add type value for snippets
   up     20130325173941  Add notification level to user
   up     20130326142630  Add index to users authentication token
   up     20130403003950  Add last activity column into project
   up     20130404164628  Add notification level to user project
   up     20130410175022  Remove wiki table
   up     20130419190306  Allow merges for forks
   up     20130506085413  Add type to key
   up     20130506090604  Create deploy keys projects
   up     20130506095501  Remove project id from key
   up     20130522141856  Add more fields to service
   up     20130528184641  Add system to notes
   up     20130611210815  Increase snippet text column size
   up     20130613165816  Add password expires at to users
   up     20130613173246  Add created by id to user
   up     20130614132337  Add improted to project
   up     20130617095603  Create users groups
   up     20130621195223  Add notification level to user group
   up     20130622115340  Add more db index
   up     20130624162710  Add fingerprint to key
   up     20130804151314  Add st diff to note
   up     20130809124851  Add permission check to user
   up     20130812143708  Add import url to project
   up     20130819182730  Add internal ids to issues and mr
   up     20130821090530  Remove deprecated tables
   up     20130821090531  Add internal ids to milestones
   up     20130909132950  Add description to merge request
   up     20130926081215  Change owner id for group
   up     20131005191208  Add avatar to users
   up     20131009115346  Add confirmable to users
   up     20131106151520  Remove default branch
   up     20131112114325  Create broadcast messages
   up     20131112220935  Add visibility level to projects
   up     20131129154016  Add archived to projects
   up     20131130165425  Add color and font to broadcast messages
   up     20131202192556  Add event fields for web hook
   up     20131214224427  Add hide no ssh key to users
   up     20131217102743  Add recipients to service
   up     20140116231608  Add website url to users
   up     20140122112253  Create merge request diffs
   up     20140122114406  Migrate mr diffs
   up     20140122122549  Remove m rdiff fields
   up     20140125162722  Add avatar to projects
   up     20140127170938  Add group avatars
   up     20140209025651  Create emails
   up     20140214102325  Add api key to services
   up     20140304005354  Add index merge request diffs on merge request
   up     20140305193308  Add tag push hooks to project hook
   up     20140312145357  Add import status to project
   up     20140313092127  Migrate already imported projects
   up     20140407135544  Fix namespaces
   up     20140414131055  Change state to allow empty merge request diffs
   up     20140415124820  Limits to mysql
   up     20140416074002  Add index on iid
   up     20140416185734  Index on current sign in at
   up     20140428105831  Add notes index updated at
   up     20140502115131  Add repo size to db
   up     20140502125220  Migrate repo size
   up     20140611135229  Add position to merge request
   up     20140625115202  Create users star projects
   up     20140729134820  Create labels
   up     20140729140420  Create label links
   up     20140729145339  Migrate project tags
   up     20140729152420  Migrate taggable labels
   up     20140730111702  Add index to labels
   up     20140903115954  Migrate to new shell
   up     20140907220153  Serialize service properties
   up     20140914113604  Add members table
   up     20140914145549  Migrate to new members model
   up     20140914173417  Remove old member tables
   up     20141006143943  Move slack service to webhook
   up     20141007100818  Add visibility level to snippet
   up     20141118150935  Add audit event
   up     20141121133009  Add timestamps to members
   up     20141121161704  Add identity table
   up     20141205134006  Add locked at to merge request
   up     20141216155758  Create doorkeeper tables
   up     20141217125223  Add owner to application
   up     20141223135007  Add import data to project table
   up     20141226080412  Add developers can push to protected branches
   up     20150108073740  Create application settings
   up     20150116234544  Add home page url for application settings
   up     20150116234545  Add gitlab access token to user
   up     20150125163100  Add default branch protection setting
   up     20150205211843  Add timestamps to identities
   up     20150206181414  Add index to created at
   up     20150206222854  Add notification email to user
   up     20150209222013  Add missing index
   up     20150211172122  Add template to service
   up     20150211174341  Allow null in services project
   up     20150213104043  Add twitter sharing enabled to application settings
   up     20150213114800  Add hide no password to user
   up     20150213121042  Add password automatically set to user
   up     20150217123345  Add bitbucket access token and secret to user
   up     20150219004514  Add events to services
   up     20150223022001  Set missing last activity at
   up     20150225065047  Add note events to services
   up     20150301014758  Add restricted visibility levels to application settings
   up     20150306023106  Fix namespace duplication
   up     20150306023112  Add unique index to namespace
   up     20150310194358  Add version check to application settings
   up     20150313012111  Create subscriptions table
   up     20150320234437  Add location to user
   up     20150324155957  Set incorrect assignee id to null
   up     20150327122227  Add public to key
   up     20150327150017  Add import data to project
   up     20150327223628  Add devise two factor to users
   up     20150328132231  Add max attachment size to application settings
   up     20150331183602  Add devise two factor backupable to users
   up     20150406133311  Add invite data to member
   up     20150411000035  Fix identities
   up     20150411180045  Rename buildbox service
   up     20150413192223  Add public email to users
   up     20150417121913  Create project import data
   up     20150417122318  Remove import data from project
   up     20150421120000  Remove periods at ends of usernames
   up     20150423033240  Add default project visibililty to application settings
   up     20150425164646  Gitlab change collation for tag names.acts as taggable on engine
   up     20150425164647  Remove duplicate tags
   up     20150425164648  Add missing unique indices.acts as taggable on engine
   up     20150425164649  Add taggings counter cache to tags.acts as taggable on engine
   up     20150425164650  Add missing taggable index.acts as taggable on engine
   up     20150425164651  Change collation for tag names.acts as taggable on engine
   up     20150425173433  Add default snippet visibility to app settings
   up     20150429002313  Remove abandoned group members records
   up     20150502064022  Add restricted signup domains to application settings
   up     20150509180749  Convert legacy reference notes
   up     20150516060434  Add note events to web hooks
   up     20150529111607  Add user oauth applications to application settings
   up     20150529150354  Add after sign out path for application settings
   up     20150609141121  Add session expire delay for application settings
   up     20150610065936  Add dashboard to users
   up     20150620233230  Add default otp required for login value
   up     20150713160110  Add project view to users
   up     20150717130904  Add commits count to project
   up     20150730122406  Add updated by to issuables and notes
   up     20150806104937  Create abuse reports

Hello

I’m on 7.14 Source install and I’d like to do exactly as you say to convert to omnibus. But in the link to the packages I cannot find a 7.14 omnibus RPM (for el7).

Is one available or is a new update method now necessary?

thanks

r.

Hi

Please find it here

Thank you. None of those are named ‘omnibus’, unlike some other releases are specifically named. Are all of them omnibus releases even though they are not explicitly named that way?

Yes all of them are omnibus artifacts, you can download and install on a test machine to verify, later on proceed with actual server.

1 Like