Git clone via https, receiving 500 Internal error

How to Use GitLab
1

I am trying to troubleshoot the 500 error when trying to clone a repo via https.

fatal: unable to access ‘https://gitsrv.domain.local/user/testproject.git/’: The requested URL returned error: 500

when I try to perform the git clone from a user within the gitlab server, I get the following error.

fatal: unable to access ‘https://gitsrv.domain.local/user/testproject.git/’: SSL certificate problem: unable to get local issuer certificate

No errors in logs that I can find.

Server Specifics
10.2.2 omnibus Community Editioinstalledn upgraded from 9.5.10
Linux RHEL-7 .4 - hardened to DISA STIG RHEL7 version 1 release 2
SELinux in permissive mode
LDAP over SSL configured to AD
Local internal CA authority

Client setups - included sslVerify = false and true at times.

I have looked over many of the related entries and listings I could find in the forums

By the instructions listed here
https://docs.gitlab.com/omnibus/settings/ssl.html

A symbolic link should be created in the
/opt/gitlab/embedded/ssl/certs/

I have checked and verified my file is a valid PEM file

When running the gitlab-ctl reconfigure command I noticed the following

Recipe: gitlab::add_trusted_certs

  • directory[/etc/gitlab/trusted-certs] action create (up to date)

  • directory[/opt/gitlab/embedded/ssl/certs] action create (up to date)

  • file[/opt/gitlab/embedded/ssl/certs/README] action create (up to date)

  • ruby_block[Move existing certs and link to /opt/gitlab/embedded/ssl/certs] action run

  • Moving existing certificates found in /opt/gitlab/embedded/ssl/certs

  • Symlinking existing certificates found in /etc/gitlab/trusted-certs

Skipping /etc/gitlab/trusted-certs/customcacert.pem.

Investigating “Skipping /etc/gitlab/trusted-certs/…”

I came across this issue


Missing perl, installed perl.
I now see

Recipe: gitlab::add_trusted_certs

  • directory[/etc/gitlab/trusted-certs] action create (up to date)
  • directory[/opt/gitlab/embedded/ssl/certs] action create (up to date)
  • file[/opt/gitlab/embedded/ssl/certs/README] action create (up to date)
  • ruby_block[Move existing certs and link to /opt/gitlab/embedded/ssl/certs] action run (skipped due to only_if)

Still no symbolic link in the /opt/gitlab/embedded/ssl/certs

I have tried manually creating the symbolic link using the subject hash of my cert.

The web interface ssl is working correctly.
LDAP over ssl setup is working correctly.
Git operations via ssh work as expected with no issues.

Is this even a cert issue?

I am at a loss and possibly in a very mixed up world after trying many different setups on the server and clients to try and get this working.

Can happily provide any additional information as necessary?

Thanks in advance for any assitance provided.

2

Ongoing troubleshooting efforts.

Made a clone of the VM
Uninstalled gitlab
1.sudo gitlab-ctl uninstall
2. sudo gitlab-ctl cleanse
3. sudo gitlab-ctl remove-accounts
4. removed the package using
rpm --erase gitlab-ce-10.2.2-ce.0.el7
removed remaining files
rm -rf /data/git-data/* /opt/gitlab /var/log/gitlab
rm -rf /etc/sysctl.d/90-omnibus-gitlab* /usr/lib/systemd/system/gitlab-runsvdir.service /etc/systemd/system/basic.target.wants/gitlab-runsvdir.service
rm -rf /etc/selinux/targeted/active/modules/100/git /etc/selinux/targeted/active/modules/100/gitosis /etc/selinux/targeted/active/modules/400/gitlab-7.2.0-ssh-keygen

performed a fresh installation from rpm of 10.2.2 CE
configured http only access with an external url

Still get the internal 500 error.
Only entry I can find in files showing any sort of error
From the ./gitlab/gitlab-workhorse/current file

2017-12-12_15:15:18.86240 2017/12/12 10:15:18 error: GET “/username/httpdec12_1.git/info/refs?service=git-upload-pack”: handleGetInfoRefs: GetInfoRefsHandler: rpc error: code = Unavailable desc = grpc: the connection is unavailable

Searches to this point all seem to point to docker related issues, this is not a docker issue

SSL and certs being the issue is a red herring and am ignoring at this point. .

Will continue the struggle…

3

This has been resolved.
Nothing to do with SSL certs or symbolic links.

A bad parameter in the gitlab.rb file.
gitaly_address in the gitlab.rb file was pointing to an invalid directory structure.
This was not being caught in the gitlab-ctl reconfigure nor being seen in the error logs.

‘gitaly_address’ => ‘unix:/data/gitaly/gitaly.socket’ }

Thanks