[SOLVED] Git pull / push behind reverse proxy not working

Hey people,

I currently installed gitlab 8.4.0.pre from source and I’m running it behind a reverse proxy as mydomain.tld/gitlab (sadly, using a subdomain is no option in my setup). I’m running it on a current Debian version in a docker container and with an apache2 server to serve it and another apache2 for the reverse proxy.

After some fiddling I’ve got the web frontend working and everything seems nice and well, except that I can’t pull / push from / to git repos via http. Cloning (of empty repos…) is no problem, but it seems like when ever I try to push / pull the following error occurs:

git push origin master
Username for 'http://mydomain.tld': <user>
Password for 'http://<user>@mydomain.tld': 
error: Cannot access URL http://mydomain.tld/gitlab/<user>/test_project.git/, return code 22
fatal: git-http-push failed
error: failed to push some refs to 'http://mydomain.tld/gitlab/<user>/test_project.git'

After some digging in the log files I found that there must be a problem while resolving the http-Request, as the GET Request for /gitlab//test_project.git/info/refs?service=git-receive-pack completes with a 200, but the following requests break.

My apache2 config can be found here, I also included my apache2 log and my gitlab log.


Edit: I also tried cloning from localhost, using localhost instead of my domain, however here I experience the same problems - so I guess I can rule out any problems with the reverse Proxy.


About a weekend of digging in gitlab-workhorse code and fiddling around I finally managed to solve my problem :slightly_smiling: .

The problem was (partially) within the startup parameters of gitlab-workhorse. Gitlab-workhorse expects the “authBackend” URL to contain the whole URL path, including its subdirectory (IMHO not the cleanest solution, since for authentification reasons the rails app is also accessible at localhost:8080, not only at localhost:8080/gitlab). Having this part solved, a local git clone http://localhost:8181/gitlab/… worked.
This must be set up in /etc/default/gitlab
gitlab_workhorse_options="-listenUmask 0 -listenNetwork tcp -listenAddr 127.0.0.1:8181 -authBackend http://127.0.0.1:8080/gitlab"

The second part was to adjust the RewriteRules in the apache2 configuration, since the rewrite rules for git repos never were matched successfuly, thus:

RewriteCond %{REQUEST_URI} ^/[\w\.-]+/[\w\.-]+/gitlab-lfs/objects.* [OR]
RewriteCond %{REQUEST_URI} ^/[\w\.-]+/[\w\.-]+/builds/download.* [OR]
RewriteCond %{REQUEST_URI} ^/[\w\.-]+/[\w\.-]+/repository/archive.* [OR]
RewriteCond %{REQUEST_URI} ^/api/v3/projects/.*/repository/archive.* [OR]
RewriteCond %{REQUEST_URI} ^/ci/api/v1/builds/[0-9]+/artifacts.* [OR]
RewriteCond %{REQUEST_URI} ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$
RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA,NE]

became:

RewriteCond %{REQUEST_URI} ^/gitlab/[\w\.-]+/[\w\.-]+/gitlab-lfs/objects.* [OR]
RewriteCond %{REQUEST_URI} ^/gitlab/[\w\.-]+/[\w\.-]+/builds/download.* [OR]
RewriteCond %{REQUEST_URI} ^/gitlab/[\w\.-]+/[\w\.-]+/repository/archive.* [OR]
RewriteCond %{REQUEST_URI} ^/gitlab/api/v3/projects/.*/repository/archive.* [OR]
RewriteCond %{REQUEST_URI} ^/gitlab/ci/api/v1/builds/[0-9]+/artifacts.* [OR]
RewriteCond %{REQUEST_URI} ^/gitlab/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$
RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA,NE]

I hope this helps some folks :slight_smile:

Thank you so much for this answer. I’m having the same problem (except for the fact that I use nginx and ubuntu). I don’t quite understand what you did with the gitlab-workhorse to solve the problem. I didn’t find any files in /etc/default/gitlab

Hey,

from your last setence I take that you have not (yet) solved the problem?

What version of GitLab are you running? The Omnibus package? For the source version, the /etc/default/gitlab file contains the command line options with which the gitlab-workhorse service is started. This might differ depending on your GitLab version (since I’m debian based I guess it’s not a linux distro difference). What I did was telling GitLab workhorse to use localhost:8080/gitlab for authentification reasons, instead of localhost:8080. This is important since gitlab-workhorse uses this option to determine the subdirectory for which it should listen on 8181 (or the port its configured to listen on).

A guess into the wild would be: Take a look into your /etc/gitlab/gitlab.{rb yml}, followed by a gitlabctl reconfigure. If this does not help, or if the commandline for gitlab-workhorse is not to be found there, I suggest searching for it with “grep -rni gitlab-workhorse 2> /dev/null” or something like that.

1 Like

No, I have not yet sovled the problem unfortunately. I’m running Version 8.4.0-rc2 installed from source using the official guide. /etc/default/gitlab doesn’t exist in my filestructure. Most gitlab files are unter /home/git/. I found the file main.go under /home/git/gitlab-workhorse where I added /gitlab to authBackend (see row 36). This unfortunately did not help :unamused:
My gitlab.yml file has no settings regarding gitlab-workhorse as far as I can tell.