Migrating Projects between Omnibus Gitlab Servers

How to Use GitLab
1

Hey guys,

I have a 7.1.1 Omnibus installation on 12.04 LTS Ubuntu which is functional but has a few systemic problems that can’t be solved. I’ve built a new 9.1.3 Omnibus server on 16.04 LTS and I’m now looking to migrate projects from the old server to the new server.

7.1.1 doesn’t have the export feature which was released sometime after 8.X.X so I have tried the following strategy:

  1. Use rake to take a full backup on the 7.1.1 server
  2. Copied the backup to the 9.1.3 server and unpacked the tar
  3. Unpack a .bundle file for a specific project and place the project.git folder into the appropriate git-data repo folder
  4. Use rake to import repos via gitlab-shell

Issues/variations I had with the above:

  1. The .bundle files are not tarballs like many people say they are. The contents of the files are plain text
  2. I was able to use git verify and git clone on the .bundle files successfully
  3. I copied the .git folder contents from the output of git clone on the .bundle file to the appropriate git-data repo folder
  4. gitlab-shell creates the group successfully but the import of the repository fails almost instantly
  5. Even with DEBUG logging level for gitlab-shell, there are no failure details in the log file for the rake import.

The error is get is:

Processing my-repo/my-project.git

  • Failed trying to create my-project (my-repo/my-project.git)
    Errors: {:base=>[“Failed to create repository via gitlab-shell”]}

I’ve also tried doing a straight copy of my-project.git from the 7.1.1 server to the 9.1.3 server and get the same error. Am i missing anything here? Better strategy? Are the errors logged elsewhere?

2

Looks like the issue was purely permissions. I needed to make the owner of all the folders/files git:git (which nobody makes clear in their guides).

For anyone interested in a full solution for importing repos from a Gitlab tar backup file:

  • Extract the contents of the full backup into a suitable folder

  • Navigate into the repositories folder in the extracted backup

  • All the repo groups are in here. Create a shell script file (call it whatever e.g. extract.sh) and paste the following text:

      #!/bin/bash
  gitDataRoot="/var/opt/gitlab/git-data/repositories"
  for group in ./*; do
      gitGroup=$(basename $group)
      echo $gitGroup
      [ "$gitGroup" == "extract.sh" ] && continue
      
      mkdir "$gitDataRoot/$gitGroup"
      cd $gitGroup
      for filename in ./*.bundle; do
              mkdir ./extract
              git clone --mirror $filename ./extract
              repository=$(basename $filename .bundle)
              cp -a ./extract/ "$gitDataRoot/$gitGroup/$repository.git/"
              chown -R git:git "$gitDataRoot/$gitGroup/$repository.git/"
              rm -rf ./extract
      done
      cd ..
  done

  • Give extract.sh permissions to execute (chmod u+x ./extract.sh)

  • Run extract.sh as root (sudo or under sudo bash)

  • Run gitlab-rake gitlab:import:repos to import the bare repositories

Notes:

  • The above only applies if your .bundle files are not tarballs (this was the case for me with v7.1.1 backups).
  • –mirror is required when cloning the .bundle files so that all branches are extracted. Without it, only the master branch is extracted.
  • I have an omnibus installation so your gitDataRoot may be in a different place
  • This script is only really suitable for fresh installations (i.e. no existing Gitlab groups with the same names as any groups in the backup)