SSL problems for internal network behind VPN with no access to Let'sEncrypt SSL method for v14.7ee

I’ve searched the Reddit group and found very similar help given in the past (thank you, /u/KaelumForever for your post) that I haven’t been able to fully leverage to get my internal-subnet Gitlab-ee fully secured.

Sorry in advance if this is obvious - I am by no means an expert in Linux and/or Gitlab.

I’ve got a CentOS 7 standalone physical PC sitting on a 10.1.10.x subnet behind a Fortigate VPN running Gitlab-ee v14.7. Any use of this machine will be restricted to the 10.1.10.x subnet and access from outside the building will require authentication to the VPN. To me, that rules out the automatic LE SSL path.

So, instead, I’ve had a couple of different results. All methods have allowed connection via https:, but each with (different) warnings.

I first followed the suggested routes for installing non-LE certs, but keep ending up with “This CA Root certificate is not trusted because it is not in the Trusted Root Certification Authorities store.”

image
image
image
image

Thinking maybe I could get a root CA in the chain, I tried this method:

but didn’t really do any better there.

I am a bit lost here, I’m afraid. I feel like I’m failing at a base level to understand SSL implementation and where exactly my problem lies. I’ve got no trouble generating the SSL certs via openssl, but getting their root trust conveyed to other machines on the subnet is eluding me. When inspecting my gitlab.x.x.x internal site’s site via the remote browser, I do see that only the base-level cert is reported, vs. the multi-step chain reported by, say, Google or my VPN or whatever other properly-secured site.

I’ve struggled for several days on this now and just know I’m overlooking something that one of you very smart people could correct in five minutes. I sincerely appreciate any help you could offer.

Thank you very much in advance.

Generally, you have two options. Purchase a certificate to use with your domain name, or create your own internal CA. This is how you would create the internal CA.

openssl genrsa -des3 -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -sha256 -days 3650 -out ca.pem

that will generate a CA certificate for 10 years you can change the days value if you want it to be longer. It will also need a password to be set, so make sure you don’t forget it, as this needs to be entered every time you generate a certificate later. This CA certficate will then be used in conjunction with certificates that you create. Now, you want to generate a CSR for your domain - this is used to create the key for use with generating the certificate:

openssl genrsa -out gitlab.key 2048
openssl req -new -key gitlab.key -out gitlab.csr

you then obviously answer all the questions filling in the CN properly which would be the full domain, eg: git.ad.cuaerospace.com

now generate the certificate:

openssl x509 -req -days 3650 -sha256 -in gitlab.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out gitlab.crt

I generated a certificate for 10 years, maybe you want for less. It’s up to you.

You import the ca.pem into your browser, and when importing you tick the boxes to allow it to trust certificates, etc, etc. For example, in firefox when importing under the authorities tab:

Screenshot from 2022-02-18 18-53-41

as you can see this is me telling my web browser to trust the certificates my CA generates.

Now, what you need to do is get the gitlab.crt and ca.pem and put these together in one single file. So you can do it like this:

cat gitlab.crt ca.pem > gitlab_combined.crt

copy the gitlab_combined.crt and the gitlab.key to your gitlab server. Put under /etc/gitlab/ssl and rename the certificate and key as git.ad.cuaerospace.com.crt and git.ad.cuaerospace.com.key. Reconfigure gitlab.

Now when you go to your gitlab server in your web browser with the imported CA certificate, you will now be trusted.

You are now running your own internal CA.

Now, if you want to import it so that other apps will work, commands from console etc or other browser:

For RHEL based distros:

mv ca.pem ca.mydomain.com
cp ca.mydomain.com /etc/pki/ca-trust/source/anchors
update-ca-trust

For Debian/Ubuntu:

mkdir /usr/share/ca-certificates/extra
mv ca.pem ca.mydomain.com
cp ca.mydomain.com /usr/share/ca-certificates/extra
update-ca-certificates

Thank you for helping. I have followed your instructions, except for the browser-side ca.pem installation. I’m hoping to roll this out for a half-dozen Windows users to use and am hoping to get something handled at the AD level if possible. If not, of course, I’ll make the rounds and/or teach people to import the CA.

That said, without the ca.pem import, I’ve followed your guidance and now Firefox from a candidate Win machine sees:
image

Edge sees:
image

(I used an 8-year exp this time to make sure it was the correct one in use)

I’m missing something.

Yes, if you don’t import the CA, then your certificate won’t be trusted.

If you have Active Directory, you can install on a Windows server a CA and do it all from a GUI environment. I was assuming you just wanted on Linux. But instead of doing all the openssl commands, you can do it from installing the CA functionality on a Windows Server.

https://support.n4l.co.nz/s/article/How-to-install-Certificate-Authority-CA-server-and-create-certificates

1 Like

Outstanding. I’m going to finish my lunch and give that a shot.
Thanks again, iwalker. I’ll reply back in this thread with results.

1 Like

I get about halfway down this path and it warns that it may need to reboot among other scariness. I’ve got a dozen folks using this machine’s file share at the moment and don’t want to ruin anyone’s Friday. I believe I’ll pick this back up this weekend sometime and see if I can get there without doing too much the AD machine. Heck, I’d buy a cert from GoDaddy or whoever, but they wouldn’t be able to see the machine it was going on and wouldn’t issue it, right?

Providing that the certificate you purchase is for a domain that you own, then it isn’t a problem. The server can be private and inaccessible to the internet.

I just tried that path via ssls.com, but am stuck at validation since they can’t see past the VPN.


Ian, is there any chance I can invite you to a Zoom or Webex or something and promise you a nice gift basket in return after a few minutes of your time? :slight_smile:
I promise I’ll post a full summary of the correct path to successful implementation for the community, should that be achieved.

It would be better to do domain validation via email, normally when you purchase a certificate (or at least when I have), they usually send to the email address that is purchasing the SSL cert. You click a link in the email, and then that verifies it.

The one you have chosen seems to want to do it with a server that is available to the internet, and this is the problem. Unfortunately a zoom or webex won’t resolve that issue. I don’t know if they will allow you to choose the type of verification though, you would have to ask them, or see if there is a choice.

From this link: https://www.ssls.com/knowledgebase/how-can-i-complete-the-domain-control-validation-for-my-ssl-certificate/

it seems they do give a choice. You need email validation.

Ugh, I must be getting tired. Yes, I see that option now and have the bundle delivered to my inbox. I’ll upload that to /etc/gitlab/ssl and restart gitlab.

Good and bad news. I’ve downloaded the bundle and uploaded it to /etc/gitlab/ssl and issued a gitlab-ctl restart, but nginx timed out. I think it’s looking for the wrong cert in gitlab.rb if I had to guess.

They were delivered as “git_ad_cuaerospace_com.ca-bundle” and “git_ad_cuaerospace_com.crt”.
Do you imagine the underscores need to be renamed?

You will need the key as well that was generated during the CSR. The crt and the bundle need to be combined:

cat git_ad_cuaerospace_com.crt git_ad_cuaerospace_com.ca-bundle > git.ad.cuaerospace.com.crt

and rename the key git git.ad.cuaerospace.com.key and both those files then go in /etc/gitlab/ssl.

Also when combining the two files, take a look in the combined files, to make sure that the they are separated properly, like this:

-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----

if you see any like that on the same line, that will not work, for example, if it looks like this:

-----END CERTIFICATE----------BEGIN CERTIFICATE-----

then that is wrong.

1 Like

Let’s say I didn’t have the key handy, but I do have the .csr, is there a path forward?

No, without the key you cannot use the certificate. If you generated the CSR yourself with openssl, you should have the key. If you did it through their web interface, it should have been available for you to download and save. It’s important that you have the key.

Lol. Nice - how in the world do I have every other file but not the key? Argh. I was clicking through their instructions pretty fast and cleaning up my previous failed openssl attempts at the same time and appear to have gotten sloppy. This is unfortunate.

You might be lucky that they let you regenerate everything for free. It just depends on the SSL issuer, sometimes they do, sometimes not and you have to buy another.

The key is always provided when you generate the CSR on their website, or when you generate it yourself using openssl at the Linux console.

Ok, I look like an idiot here. The “.pem” was the key. Just not labeled “.key”.
lol
I checked the issued .crt vs. my .pem here:

and they match. Now I just need to get them in the /etc/gitlab/ssl folder and point to them in gitlab.rb, right?

1 Like