Copying an GitLab Instance

Dear all,

I want to copy a GitLab instance to a new server with another operating system (Ubuntu instead of Centos).

As the instance is quite big, I did a fresh install on the new server and then copied over all data files, the /etc/gitlab.rb and last not least the Postgres database.

Everything seems tp work fine now, only thing which is missing are the GitLab Runners. When opening the “Runners” entry in the admin interface, only a “500” error appears. As there are many runners configured in our instance, I can’t recreate all of them manually.

I was searching through the old installation, but I didn’t find the place where GitLab is storing the information about the configured runners - it seems to be not in the Postgres database as well not in any config files.

So my question is, where can I find the configured runners? And what is the best way of copying them over to a new instance?

Did you also copy /etc/gitlab/gitlab-secrets.json to the new server? (just in case this was missed).

Yes, but that gave other problems with the pages configuration, so I went back to the original one …

Pages is probably easier to fix than re-registering all your runners. When moving a server, or restoring it, gitlab-secrets.json should be restored. A lot of things rely on this. This is explained in the Gitlab documentation.

I believe the list of configured runners are included in backups, so if you had done a backup on the old server and then a restore on the new server you wouldn’t have this issue. Also everything else - i.e. the stuff you haven’t yet discovered to be missing - should be included.
(This of course involves restoring gitlab-secrets.json)

If you don’t want to start over, using a proper method[1], I think the only real option is to re-register all of your runners (you say that can’t be done, so I’ll leave it to you to draw the conclusion).

[1] A proper method is one based on backup+restore. For a big instance it might be faster to skip the repositories from the backup and synchronise those in another way (it was for us), but that requires that you know what you’re doing, and probably also a few more tests of your method.

Dear all,

thanks for your help: indeed, changes in gitlab-secrets.json brought me some steps forward and the runner problem seems to be solved.

But you are right, maybe other problems will pop up which I can’t see currently. Our instance ist very big (several TB), so I thought a “classical” backup is not the way to go.

Or what do you think, until which size the GitLabs included backup/restore procedures make sense?

That is not simple. The purpose and the reason why the instance has reached a certain size matters a lot when determining which size is the limit.

For certain disasters a backup taken with the internal tools are very good to have. As a lot of the reason for our instance taking up a couple of hundred GiB is projects that belong to ex-employees (Personal forks of projects) and thus data that doesn’t change, we can save a lot of time when migrating by not copying that into a backup and restoring it, but using e.g. rsync on the repositories (that is not so realistic after we have put data into a gitaly setup, but hopefully we can just let that live and avoid big migrations in the future).
For migrations the build-in backup+restore might not be as effective as other ways, but in most cases it will have to be a part of it, possibly with certain things skipped, that you then need to find other ways of getting over, but you really need to know what you’re doing before going that way.

So you have to look at your instance, find out what it uses all that space on (moving it to some kind of object storage is also an option).