Internal API unreachable during git push --mirror

I’m using the git repository duplication guide linked here to mirror from one location to GitLab:

https://help.github.com/articles/duplicating-a-repository/#mirroring-a-repository-in-another-location

Essentially the following commands (except I’m using SSH instead of HTTPS):

$ git clone --mirror https://github.com/exampleuser/repository-to-mirror.git
$ cd repository-to-mirror.git
$ git remote set-url --push origin https://github.com/exampleuser/mirrored
$ git fetch -p origin
$ git push --mirror

Summary:

  • GitLab is currently running version 10.2.2 CE
  • I have 1800+ branches to mirror (approximately 1.5GB of data bare).
  • The new repository I’m pushing to in GitLab is empty/new.
  • Pushing from non-mirror git clones for individual branches succeeds.
  • SSH keys are working for other repositories and pushing individual branches.
  • I’m an Owner at the group level (also tried adding explicitly to the project level).

Steps already tried:

  • gitlab-rake gitlab:check Succeeds with API check.
  • Increased unicorn timeout/workers (unicorn['worker_timeout'] = 3600).
  • Increased git timeout (gitlab_rails['git_timeout'] = 1200).
  • The push fails after some time has passed (always less than the timeouts above).
  • As above - pushing individual branches from non-mirror repository succeeds.

Verbose output from git when the mirror push fails:

Pushing to git@gitlab:development/my-repo.git
remote: GitLab: Failed to authorize your Git request: internal API unreachable
To git@gitlab:development/my-repo.git
 ! [remote rejected] master -> master (pre-receive hook declined)
<repeated message for all branches/tags>
error: failed to push some refs to 'git@gitlab:development/my-repo.git'

Log from gitlab-shell when this fails:

[2017-12-14T14:06:22.458931 #185783]  INFO -- : gitlab-shell: executing git command <git-receive-pack /var/opt/gitlab/git-data/repositories/development/my-repo.git> for user with key key-33.
[2017-12-14T14:08:57.208911 #185988]  WARN -- : Failed to connect to internal API <POST http://localhost:8080/api/v4/internal/allowed>: #<EOFError: end of file reached>

I see the sam issue

Confirmed with more testing, this seems to only affect “large” repositories. I succeeded in mirroring a small repository using the above method.

This issue looks like the same thing also: Push --mirror large repos fail

I’m seeing the same issue, on 10.2.4-EE.

I can push everything if I do every branch / tag one at a time, so this appears to be related to the number of refs rather than the size of the repository. The repository I’m trying to mirror has ~300 branches and ~8800 tags, and is ~400 MB in size.

Can you please also add your comments to this official GitLab issue? Looks like the same thing.