GitLab LetsEncrypt issue

happy new year Gitlab community

I hope I am in the correct section and i apologies if I have made a mistake…

We are having a bit of problem with our GitLab instance with LetsEncrypt issue this might be common issue already and also we tried solutions but didn’t work out well

Please see screenshot: letsencrypt_certificate-issue.png for our latest issue with LetsEncrypt when we run the command gitlab-ctl reconfigure

We tried looking at the file /opt/gitlab/embedded/cookbooks/letsencrypt/resources/certificate.rb if there any issues in here, but somehow it doesn’t show any issues

We also tried disabling the LetsEncrypt in the file /etc/gitlab/gitlab.rb by setting this value to false: letsencrypt['enable'] = false

However, it didn’t help at all

We’re kinda stuck on this issue, hope someone can help us

letsencrypt_certificate-issue1911×127 16.7 KB

Thanks

Richard

Hi,

If you don’t want to use letsencrypt, you need to configure a valid certificate, so:

nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.example.com.crt"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.example.com.key"

that is assuming that external_url has been configured with https. If the /etc/gitlab/ssl directory doesn’t exist, create it. Then make sure your certificate with intermediate and CA is in the gitlab.example.com.crt file, and the certificate key in gitlab.example.com.key.

The name of the two certificate files is the same as your external_url, so if it is git.openb.net then our certificate would be git.openb.net.crt and the key git.openb.net.key. The certificate and key are always the FQDN of the server, so the URL you use to access it.

Hi, I managed to google a temporar fix by commenting out lines 27-54 in /opt/gitlab/embedded/cookbooks/letsencrypt/resources/certificate.rb

But I believe the file will be overwritten during next update.

I get into this issue while migrating from an old instance to a new one. I migrated the instance via backup as recommended, but I was unable to resolve this issue.

For those interested, see the file in source code: files/gitlab-cookbooks/letsencrypt/resources/certificate.rb · master · GitLab.org / omnibus-gitlab · GitLab

The article pointing me into right direction: Unable to renew letsencrypt certificate anymore (#4900) · Issues · GitLab.org / omnibus-gitlab · GitLab

I tried deleting /etc/gitlab/ssl/* but without success. Only commenting out these lines fixed it.

Hello,

many thanks for your replies., it is most appreciated

We are definitely stuck on the issuing a certificate on our GitLab instance

If you visit https://git.openb.net/, you’ll end up having an error with the certificate being self signed but we enabled the letsencrypt['enable'] = true under our gitlab.rb

I’m guessing that GitLab isn’t mounting correctly with nginx?

We tried generating a certificate through OpenSSL and apply to nginx please see “self-signed-snippets.png”, the conf file tells to use the certificate under /etc/gitlab/ssl/git.openb.net.crt and certificate key in /etc/gitlab/ssl/git.openb.net.key

This might be the reason why we are having the “Self sign certificate” issue

Then next one is we generated a ssl-params.conf file under /etc/nginx/snippets/ with the following snippet, please see screenshot: “ssl-params-snippet.png”

Then we created a server block under /etc/nginx/sites-available/[git.openb.net](http://git.openb.net) in this file we added the two server blocks, please see screenshot: “sites-available-server-blocks.png”

Then we linked the sites-available to sites-enabled

Please take note that we are still having this error:

"There was an error running gitlab-ctl reconfigure:

letsencrypt_certificate[git.openb.net] (letsencrypt::http_authorization line 6) had an error: Acme::Client::Error::AccountDoesNotExist: acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 41) had an error: Acme::Client::Error::AccountDoesNotExist: No account exists with the provided key"

Even if we commented the said code block in /opt/gitlab/embedded/cookbooks/letsencrypt/resources/certificate.rb

I am not sure if we’re on the right path on this, hope you can shed some light on how we mount GitLab with nginx correctly with SSL please

Really appreciate your help on this

Thanks,

ssl-params-snippet.png1370×312 48.1 KB

sites-available-server-blocks1446×432 32.1 KB

self-signed-snippets1920×130 14.3 KB

regards

Richard

If you did create the self-signed one, and have uncommented lines similar to this:

nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.example.com.crt"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.example.com.key"

put a comment (#) at the beginning of the line and try re-running reconfigure to see if it will use letsencrypt. Maybe it has a mix of config items, which is causing the problem.

For successful letsencrypt, see the docs here: Configure SSL for a Linux package installation | GitLab

Mostly it should only need you to configure these items only:

letsencrypt['enable'] = true                      # GitLab 10.5 and 10.6 require this option
external_url "https://gitlab.example.com"         # Must use https protocol
letsencrypt['contact_emails'] = ['foo@email.com'] # Optional

that should be enough for you to then run reconfigure and get letsencrypt working. For automatic renewal of letsencrypt, check these options:

# This example renews every 7th day at 12:30
letsencrypt['auto_renew_hour'] = "12"
letsencrypt['auto_renew_minute'] = "30"
letsencrypt['auto_renew_day_of_month'] = "*/7"
letsencrypt['auto_renew'] = true

many thanks for your reply, I tried the latest suggested fix, it somehow gave a new issue in which I think we’re about to solve it…-

Hi @iwalker

Thank you so much for your suggestion

We commented the lines that contain the nginx['ssl_certificate'] and nginx['ssl_certificate_key'] please see screenshot: nginx-gitlab-rb.png

Then when we visit git.openb.net the certificate error is now about “NET::ERR_CERT_AUTHORITY_INVALID” in which I think points to having the .pem file (this file is already in place in /etc/gitlab/ssl/dhparam.pem )

It also led to have this error when we ran gitlab-ctl reconfigure, please see screenshot: gitlab-ctl-reconfigure-acme-error.png

It seems that LetsEncrypt is now kicking in, but having some validation errors

I think we are getting somewhere but we are still having some issues with the acme and validation of the certificates

Thanks all for your help

gitlab-ctl-reconfigure-acme-error1909×209 61.9 KB

nginx-gitlab-rb1136×876 114 KB

Comment that line as well:

# nginx['ssl_dhparam'] = nil # Path to dhparams.pem, eg. /etc/gitlab/ssl/dhparams.pem

I don’t have this line uncommented, so I guess you were using this when experimenting. Then try again.

Hi @iwalker

Thanks for your advice, however, it doesn’t seem to do anything even if we commented that line in

We are still getting the same error please see screenshot: “gitlab-ctl-reconfigure-acme-error-2.png”

This might be on LetsEncrypt’s end or something is blocking the issuance of the new certs and validating them?

I am not sure how to debug this now, hope you have any suggestions

Really appreciate your help on this

gitlab-ctl-reconfigure-acme-error-21907×222 62.5 KB

regards

Richard

LetsEncrypt requires access to ports 80 and 443 - so it looks like you have a firewall blocking it. It says so in the error message.

Timeout during connect (likely firewall problem).

Hi iwalker,

Many thanks for all your help on this

We managed to fix the firewall issue, however, we are still getting an issue with the validation of LetsEncrypt into our Gitlab instance, please see attached screenshot: letsencrypt-validation-failing.png

It looks like there is something else that we can’t figure on this, please let me know if you need more info

On the other hand, is there any guideline you can give us to use a custom certificate and just ditch LetsEncrypt? Like using ZeroSSL?

Hope you can help us out

letsencrypt-validation-failing1919×154 36.5 KB

regards

Richard

Hi Richard,

Never used ZeroSSL but if it works the same as purchasing an SSL certificate, then the only thing you need to do is:

1. Combine the SSL certificate + CA bundle into a single file. Make sure it is formatted correctly, with the BEGIN and END lines for each part of the certificate on a new line.
2. Copy the combined certificate and key to the gitlab server. These should be placed under /etc/gitlab/ssl (create the directory if it doesn’t exist).
3. The files should be named as per your FQDN so the external_url that you use. Therefore git.openb.net.crt and git.openb.net.key
4. Make sure letsencrypt is disabled in gitlab.rb and run reconfigure.

That should be enough. I find it weird you are having these problems though. All those errors shouldn’t be appearing, and I have no idea why it is doing that unless something in gitlab.rb is really messed up.

Hi, is there a way to pass in letsencrypt[enable] = false during installation?