Gitlab Migrate old server(Version 14.0.3-ee) to new server(Version 14.0.3-ee)

ducen · September 28, 2021, 12:28pm

We need to Migrate Gitlab old server(Version 14.0.3-ee) to new server(Version 14.0.3-ee) so kindly let us know to how to proceed this one or share any YouTube link

iwalker · September 28, 2021, 12:43pm

Hi,

Pretty much everything you need for backup/restore you can find here: Back up and restore GitLab | GitLab

In summary:

Install Gitlab-ee of the same version on your new server.
Copy /etc/gitlab/gitlab.rb and /etc/gitlab/gitlab-secrets.json to the new server.
Run gitlab-ctl reconfigure to get the system running with the configuration just copied across. This will activate the postgres database and you will have an empty system ready for restore.
Follow the restore instructions here: Back up and restore GitLab | GitLab
After restore, run gitlab-ctl reconfigure again, restart with gitlab-ctl restart, and then check sanity gitlab-rake gitlab:check SANITIZE=true.

If you are not running those commands as the root user, then you will need to add sudo at the beginning of each of the lines, similar to what you will see in the restore documentation.

The backup on your old server will be under /var/opt/gitlab/backups. You might want to create a new backup as per the backup documentation so that you ensure you have all the data before moving to the new server, since people might have made commits during the day since the last backup was automatically ran during the night.

ducen · September 29, 2021, 7:00am

Hi Team,

We have done for you mentioned instruction but we are facing some issue so kindly find the attached file and kindly help to us

(Attachment Gitlab Restore issue.txt is missing)

ducen · September 29, 2021, 7:09am

Hi Iwalker,

We have done for you mentioned instruction but we are facing below issue so kindly help to us

root@ducen:~# gitlab-backup restore BACKUP=1632410448_2021_09_23_14.0.3-ee
Unpacking backup … done
2021-09-28 19:44:35 +0530 – Restoring repositories …
rake aborted!
ActiveRecord::ConnectionNotEstablished: could not connect to server: No such fil e or directory
Is the server running locally and accepting
connections on Unix domain socket “/var/opt/gitlab/postgresql/.s.PGSQL.5 432”?
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:187:in eac h' /opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:187:in cle anup_snippets_without_repositories’
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:50:in ensu re in restore' /opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:51:in rest ore’
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:122:in b lock (4 levels) in <top (required)>' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:80:in bl ock (3 levels) in <top (required)>’
/opt/gitlab/embedded/bin/bundle:23:in load' /opt/gitlab/embedded/bin/bundle:23:in ’

Caused by:
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket “/var/opt/gitlab/postgresql/.s.PGSQL.5 432”?
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:187:in eac h' /opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:187:in cle anup_snippets_without_repositories’
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:50:in ensu re in restore' /opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:51:in rest ore’
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:122:in b lock (4 levels) in <top (required)>' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:80:in bl ock (3 levels) in <top (required)>’
/opt/gitlab/embedded/bin/bundle:23:in load' /opt/gitlab/embedded/bin/bundle:23:in ’

Caused by:
ActiveRecord::ConnectionNotEstablished: could not connect to server: No such fil e or directory
Is the server running locally and accepting
connections on Unix domain socket “/var/opt/gitlab/postgresql/.s.PGSQL.5 432”?
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/backup/repositories.rb:16:in group_relation' /opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/backup/repositories.rb:20:in find_groups_in_batches’
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/backup/repositories.rb:46:in enqueue_consecutive_groups' /opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/backup/repositories.rb:40:in enqueue_consecutive’
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:45:in rest ore' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:122:in b lock (4 levels) in <top (required)>’
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:80:in bl ock (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in load’
/opt/gitlab/embedded/bin/bundle:23:in `’

Caused by:
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket “/var/opt/gitlab/postgresql/.s.PGSQL.5 432”?
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/backup/repositories.rb:16:in group_relation' /opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/backup/repositories.rb:20:in find_groups_in_batches’
/opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/backup/repositories.rb:46:in enqueue_consecutive_groups' /opt/gitlab/embedded/service/gitlab-rails/ee/lib/ee/backup/repositories.rb:40:in enqueue_consecutive’
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:45:in rest ore' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:122:in b lock (4 levels) in <top (required)>’
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:80:in bl ock (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in load’
/opt/gitlab/embedded/bin/bundle:23:in `’
Tasks: TOP => gitlab:backup:repo:restore
(See full trace by running task with --trace)
root@ducen:~#

iwalker · September 29, 2021, 7:21am

Once you have configured the empty Gitlab with the configuration copied across, you need to have all gitlab services running.

gitlab-ctl restart

gitlab-ctl status

make sure they are running else you cannot restore since the database is not running.

ducen · September 29, 2021, 10:24am

I have run showing below status

root@ducen:~# gitlab-ctl restart
ok: run: alertmanager: (pid 31788) 0s
ok: run: gitaly: (pid 31798) 0s
ok: run: gitlab-exporter: (pid 31827) 1s
ok: run: gitlab-workhorse: (pid 31833) 0s
ok: run: grafana: (pid 31843) 1s
ok: run: logrotate: (pid 31936) 0s
ok: run: nginx: (pid 31942) 1s
ok: run: node-exporter: (pid 31950) 0s
ok: run: postgres-exporter: (pid 31956) 0s
ok: run: postgresql: (pid 31965) 1s
ok: run: prometheus: (pid 31978) 0s
ok: run: puma: (pid 31991) 0s
ok: run: redis: (pid 31996) 0s
ok: run: redis-exporter: (pid 32002) 1s
ok: run: sidekiq: (pid 32011) 0s
root@ducen:~# gitlab-ctl status
run: alertmanager: (pid 31788) 20s; run: log: (pid 18460) 166567s
run: gitaly: (pid 31798) 19s; run: log: (pid 17865) 166657s
run: gitlab-exporter: (pid 31827) 19s; run: log: (pid 18248) 166585s
run: gitlab-workhorse: (pid 31833) 18s; run: log: (pid 18150) 166603s
run: grafana: (pid 31843) 18s; run: log: (pid 18597) 166525s
run: logrotate: (pid 31936) 17s; run: log: (pid 17696) 166671s
run: nginx: (pid 31942) 17s; run: log: (pid 18198) 166595s
run: node-exporter: (pid 31950) 16s; run: log: (pid 18237) 166590s
run: postgres-exporter: (pid 31956) 16s; run: log: (pid 18501) 166560s
run: postgresql: (pid 31965) 16s; run: log: (pid 17931) 166651s
run: prometheus: (pid 31978) 15s; run: log: (pid 18434) 166572s
run: puma: (pid 31991) 14s; run: log: (pid 18096) 166614s
run: redis: (pid 31996) 14s; run: log: (pid 17735) 166663s
run: redis-exporter: (pid 32002) 14s; run: log: (pid 18393) 166579s
run: sidekiq: (pid 32011) 11s; run: log: (pid 18116) 166607s
root@ducen:~#

How to fix this restoration issue

iwalker · September 29, 2021, 10:36am

Wait until services have been running at least 5 minutes, and then try restore. Also, how many cpu and memory does your new server have? It should have at least 4cpu, and 8gb ram.

ducen · September 29, 2021, 10:50am

we have fixed 4cpu & 16GB RAM & while we restore GitLab that time services is running or not

(You have mentioned at below points

sudo gitlab-ctl stop puma
sudo gitlab-ctl stop sidekiq
# Verify
sudo gitlab-ctl status

Next, restore the backup, specifying the timestamp of the backup you wish to restore:

This command will overwrite the contents of your GitLab database!

sudo gitlab-backup restore BACKUP=11493107454_2018_04_25_10.6.4-ce)

Is it possible to connect remote session??

iwalker · September 29, 2021, 11:03am

Why are you trying to restore a really old (2018-04-25) backup for version 10.6.4-ce? You need to be restoring the EXACT same version of backup to the same version of Gitlab installed. So if you have 14.0.3-ee then it must be a 14.0.3-ee backup. If it is 14.0.3-ce then it must be backup from 14.0.3-ce. If you want to restore 10.6.4-ce then you must install Gitlab-CE 10.6.4 on your server.

Attempting to restore a backup that doesn’t match the version of Gitlab installed, will not work.

ducen · September 29, 2021, 12:11pm

No I am restoring this version(14.0.3-ee) only file only and I am saying to you for example of that cmd

sudo gitlab-backup restore BACKUP=1632410448_2021_09_23_14.0.3-ee

I have attached first error text file so kindly find that file
(root@ducen:~# gitlab-backup restore BACKUP=1632410448_2021_09_23_14.0.3-ee
Unpacking backup … done)

Hi Iwalker

Restore is done but data is not there in new instance so kindly help to us

root@ducen:~# gitlab-backup restore BACKUP=1632410448_2021_09_23_14.0.3-ee
Unpacking backup … done
2021-09-29 18:38:56 +0530 – Restoring repositories …

gitlab-instance-a16e787d.wiki (@groups/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.wiki) …
gitlab-instance-a16e787d.wiki (@groups/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.wiki) … [DONE]
gitlab-instance-a16e787d/Monitoring (@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b) …
gitlab-instance-a16e787d/Monitoring (@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b) … [DONE]
gitlab-instance-a16e787d/Monitoring.wiki (@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki) …
gitlab-instance-a16e787d/Monitoring.wiki (@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki) … [DONE]
gitlab-instance-a16e787d/Monitoring.design (@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.design) …
gitlab-instance-a16e787d/Monitoring.design (@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.design) … [DONE]
2021-09-29 18:38:57 +0530 – done
2021-09-29 18:38:57 +0530 – Restoring builds …
2021-09-29 18:38:57 +0530 – done
2021-09-29 18:38:57 +0530 – Restoring artifacts …
2021-09-29 18:39:08 +0530 – done
2021-09-29 18:39:08 +0530 – Restoring pages …
2021-09-29 18:39:08 +0530 – done
2021-09-29 18:39:08 +0530 – Restoring lfs objects …
2021-09-29 18:39:08 +0530 – done
This task will now rebuild the authorized_keys file.
You will lose any data stored in the authorized_keys file.
Do you want to continue (yes/no)? yes

Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data
and are not included in this backup. You will need to restore these files manually.
Restore task is done.
root@ducen:~# sudo gitlab-ctl reconfigure

ducen · September 30, 2021, 4:50am

Hi Team,

   waiting for your response

iwalker · September 30, 2021, 6:51am

Well from your restore above, it only restored the monitoring repo. It seems your backup doesn’t have any repos in it. You will need to search on your old server on /var/opt/gitlab/backups to make sure there is a full backup, or like I posted before - go to the link related to backup/restore, and manually run the backup command to make a backup of your server to make sure all repos are in it. If the backup doesn’t contain the repos, how can it show/restore them? This needs to be resolved first.

ducen · October 5, 2021, 7:31am

Hi Walkar,

As you instructed we have followed but this single repository is only restored so kindly help to us to restore our all repository to new server

ducen · October 5, 2021, 12:57pm

Waiting for your response

ducen · October 8, 2021, 8:40am

Waiting for your response

dnsmichi · October 8, 2021, 6:58pm

Hi,

@iwalker was so kind to guide you on our community forum this far, he is not a GitLab team member, albeit one could believe so from his amazing responses here.

@ducen I can see that you are using the Enterprise version, mind opening a support ticket so that GitLab support teams can help you and investigate in detail what’s wrong with your restore/migration?

Cheers,
Michael