Issue with git clone after migration into the new server

How to Use GitLab Migrations
1

After migration into the new server (self-hosted instance) I have an issue connected with clone repo from the new server.

after

$ git clon example-repository
Cloning into 'example-repository'...
time="2021-11-26T12:32:05Z" level=info msg="SSL_CERT_DIR is configured" ssl_cert_dir=/opt/gitlab/embedded/ssl/certs/
remote: Enumerating objects: 1305, done.
remote: Total 1305 (delta 0), reused 0 (delta 0), pack-reused 1305
Receiving objects: 100% (1305/1305), 165.21 KiB | 929.00 KiB/s, done.
Resolving deltas: 100% (912/912), done.
warning: remote HEAD refers to nonexistent ref, unable to checkout.

$ cd example-repository

is visible only .git directory

cat .git/config
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = git@gitlab.example.com:group/gitlab-repository.git
	fetch = +refs/heads/*:refs/remotes/origin/*

ls .git/refs/heads/ is empty

to see cloned repo I need run git checkout master
output:

Branch 'master' set up to track remote branch 'master' from 'origin'.
Already on 'master'
Your branch is up-to-date with 'origin/master'.

then when I run

cat .git/config
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = git@gitlab.example.com:group/gitlab-repository.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master


ls .git/refs/heads/
master

Configuration after git checkout is expected immediately after git clone (I have them after cloning from the old server)

How I made migration:
Install 14.5.0-ce on the new server
Upgrade old server to latest version 14.5.0-ce
Copy gitlab.rb oraz gitlab-secrets_org.json and gilab-ctl reconfigure

gitlab-backup create SKIP=uploads,builds,registry,artifacts,pages
gitlab-backup create SKIP=db,repositories,registry

On the new server

gitlab-ctl stop puma
gitlab-ctl stop sidekiq
gitlab-backup restore backup1
gitlab-backup restore backup2
gilab-ctl reconfigure
gilab-ctl restart

Everything working ok on new server instead git cloning

The old server is on Debian 9, the new one on Debian 10

root@gitlab:~# gitlab-rake gitlab:env:info

System information
System:		Debian 10
Current User:	git
Using RVM:	no
Ruby Version:	2.7.4p191
Gem Version:	3.1.4
Bundler Version:2.1.4
Rake Version:	13.0.6
Redis Version:	6.0.16
Git Version:	2.33.1.
Sidekiq Version:6.2.2
Go Version:	unknown

GitLab information
Version:	14.5.0
Revision:	f8796c0836e
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	12.7
URL:		https://gitlab.example.com
HTTP Clone URL:	https://gitlab.example.com/some-group/some-project.git
SSH Clone URL:	git@gitlab.example.com:some-group/some-project.git
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers: 

GitLab Shell
Version:	13.22.0
Repository storage paths:
- default: 	/home/gitlab/git-data/repositories
GitLab Shell path:		/opt/gitlab/embedded/service/gitlab-shell
Git:		/opt/gitlab/embedded/bin/git

Any idea how to solve this.

2

Hi,

when you clone the repository, can you run git branch -r to list all remote branches? (or git branch -vv for more details). I’m curious how this looks on the GitLab server, when you navigate into the UI into the project > branches, can you share a screenshot?

Googling for warning: remote HEAD refers to nonexistent ref, unable to checkout. unveils a few threads saying that the warning only says that the default local checkout did not happen. git - warning: remote HEAD refers to nonexistent ref, unable to checkout - Stack Overflow

One possibility is that the project uses the main branch as default head branch, while your local git client config expects that master - and then says it cannot find it. Run git branch -a to see what’s there, as noted above.

Cheers,
Michael

1 Like
3

Hi.
Very thanks for reply

❯ git branch -r -vv
Alias tip: gb -r -vv
  origin/1.2.35                                           a0c18f3 New jenkins configuration
  origin/1.2.36                                           b75cd15 Bumped version refs #4578
  origin/1.2.38                                           78876e8 Merge branch 'master' into 1.2.38
  origin/1.2.39                                           c79cd6a Improved _loadfixtures load_sample_data logic
  origin/10799_node_npm_version                           6d617ed Add npm_version env setting nodeenv_install now can install specific npm version by npm_version setting
  origin/11712_nodeenv_install_in_install_script          63d4145 Add installing nodeenv requirements in install script
  origin/11896_set_managed_property_for_legacy_models     8e78204 Add settings managed property in models based on settings
  origin/2-support-for-fixtures-in-format-other-than-yaml c5d8753 Update setup.py
  origin/6579_dumpdata_all                                5c980c2 Added dumpdata_to_directory command
  origin/celery_multi_queues                              0879629 Add celery multi queues to base_fabfile
  origin/config_based_patch_commands                      d1b427b Virtualenv_patch() is using now command list from config.
  origin/deploy_mf_branch                                 05f5454 deploy mf_branch parameter added
  origin/git_hooks_fix                                    72b2706 Add git_hooks dir fix in manifest.in
  origin/manifest_fix                                     4f81b6e include fix
  origin/master                                           f801525 Merge branch 'VPR-1322_add_worn_only_for_flower_method' into 'master'
  origin/revert-5f7fdadf                                  aadcde1 Revert "Merge branch '18931_improve_LB_traffic_redirect_during_deploy' into 'master'"
  origin/virtualenv_improvement                           bda9bbb Update CHANGELOG.md

gitlab_branches
gitlab_branches904×822 106 KB

I tried to migrate db and repositories with both default initial branch names (master, main) and the result is the same.

In migrated projects I didn’t find any project without this issue. New projects created on the new server works fine.

With best regards
Marcin

4

Hi,

hmmm, tough one, never seen this before. The screenshot shows that the remote server has the master branch as default branch. This should tell a client the head ref on the clone then.

Maybe the remote refs head isn’t updated. Can you try changing the default branch in the project settings to ‘main’, save, clone again, change to ‘master’, save, clone again? I’d like to rule out that something is cached. Please share the clone output of all steps.

Cheers,
M8chael

5

Hi
After changing the default branch in a project on another one, and then back to the master branch, clone works ok.

Do you have any tips, how to do this globally for all projects at once?

With best regards
Marcin

6

I found a few projects where the default branch was main and for that projects, everything works good (no special actions required)

7

Same her after migrating an gitlab 14.5 installation from a debian 10 to a debian 11 machine

8

If we change the default branch back and forth in the web gui the problem seems to be solved, but if we backup the new machine and restore it, the problem reoccurs.

9

Hi,

interesting, thanks for sharing. So the remote ref seems to think it is main while the imported project only provides master. I don’t think that this is an expected behaviour, can you create an issue and link this topic please?

For automating the process, the GitLab API provides the project endpoint, where you can set the default_branch setting. You’ll need to fetch all projects from the API and then iterate over them in a loop. I’m using python-gitlab because I feel confident in Python, as an example: Projects — python-gitlab 2.10.1 documentation

Cheers,
Michael

10

Issue link

11

Hi Michael,
thanks for hinting to project endpoit, because this bug reoccurs after manually fixing it, once a a “fiexed” backup is restored.

12

Hi,

we had the same issue after migrating our gitlab to different host. For me this workaround helped to get it the repos working again. Don’t know if it has any side effects tho at the moment, so be careful.

Repo: git.example.com/Example/Project1

$ gitlab-rails c
> p = Project.find_by_full_path('Example/Project1')
> p.change_head(p.default_branch)