Letsencrypt failing when I reconfigure

GitLab CI/CD
1

I installed the omnibus CE package, on Ubuntu 18.04. The installation went fine and I was able to login, and change the root password, using HTTP. I now want to enable support for auto generation/renewal of letsencrypt ssh certificate.

I followed the instruction, by making the following modifications to gitlab.rb:

letsencrypt['enable'] = true                      # GitLab 10.5 and 10.6 require this option
external_url "https://gitlab-ci.mycompany.com"   # Must use https protocol
letsencrypt['contact_emails'] = ['foo@email.com'] # Optional
letsencrypt['auto_renew'] = true
letsencrypt['auto_renew_hour'] = "12"
letsencrypt['auto_renew_minute'] = "30"
letsencrypt['auto_renew_day_of_month'] = "*/7"

When I run:

sudo gitlab-ctl reconfigure

I get the following:

================================================================================
Error executing action `create` on resource 'letsencrypt_certificate[gitlab-ci.mycompany.com]'
================================================================================

RuntimeError
------------
acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 20) had an error: RuntimeError: [gitlab-ci.mycompany.com] Validation failed for domain gitlab-ci.mycompany.com

Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/providers/certificate.rb:93:in `block (2 levels) in class_from_file'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/providers/certificate.rb:68:in `map'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/providers/certificate.rb:68:in `block in class_from_file'

Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/recipes/http_authorization.rb

  3: letsencrypt_certificate site do
  4:   fullchain node['gitlab']['nginx']['ssl_certificate']
  5:   key node['gitlab']['nginx']['ssl_certificate_key']
  6:   notifies :run, "execute[reload nginx]", :immediate
  7:   notifies :run, 'ruby_block[display_le_message]'
  8: end

Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/recipes/http_authorization.rb:3:in `from_file'

letsencrypt_certificate("gitlab-ci.mycompany.com") do
  action [:create]
  updated true
  updated_by_last_action true
  default_guard_interpreter :default
  declared_type :letsencrypt_certificate
  cookbook_name "letsencrypt"
  recipe_name "http_authorization"
  fullchain "/etc/gitlab/ssl/gitlab-ci.mycompany.com.crt"
  key "/etc/gitlab/ssl/gitlab-ci.mycompany.com.key"
  alt_names []
  cn "gitlab-ci.mycompany.com"
end

System Info:
------------
chef_version=13.6.4
platform=ubuntu
platform_version=18.04
ruby=ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client

Recipe: gitlab::unicorn
  * service[unicorn] action restart
    - restart service service[unicorn]
Recipe: gitlab::sidekiq
  * service[sidekiq] action restart
    - restart service service[sidekiq]
Recipe: gitlab::gitlab-rails
  * execute[clear the gitlab-rails cache] action run
    - execute /opt/gitlab/bin/gitlab-rake cache:clear
Recipe: nginx::enable
  * service[nginx] action restart
    - restart service service[nginx]

Running handlers:
There was an error running gitlab-ctl reconfigure:

letsencrypt_certificate[gitlab-ci.mycompany.com] (letsencrypt::http_authorization line 3) had an error: RuntimeError: acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 20) had an error: RuntimeError: [gitlab-ci.mycompany.com] Validation failed for domain gitlab-ci.mycompany.com

Running handlers complete
Chef Client failed. 15 resources updated in 33 seconds

When I check for the certificates, I see they are there:

root@gitlab-ci:/etc/gitlab/ssl# ls -l

 -rw-r--r-- 1 root root 1131 Jan  6 01:36 gitlab-ci.mycompany.com.crt
 -r-------- 1 root root 1679 Jan  6 01:36 gitlab-ci.mycompany.com.key
 -r-------- 1 root root 1675 Jan  6 01:36 gitlab-ci.mycompany.com.key-staging

If I try to connect using HTTPS, I get a message that my connection is not safe and I should not proceed. Any ideas?

2 Likes
2

I had this issue and solved it by deleting the files in /etc/gitlab/ssl and re-running gitlab-ctl reconfigure

3 Likes
3

Deleting files in /etc/gitlab/ssl worked for me, too.

Being a Linux novice, I was searching for the location of gitlab-ctl just to find out it seems to be in the search path, so I can call it from any location.

My complete command line was:

sudo gitlab-ctl reconfigure
4

There are references about another similar issue here: LetsEncrypt certificates fail in domain validation
It seems the same error:

  • ā€œThere was an error running gitlab-ctl reconfigureā€
  • ā€œValidation failed for domainā€

Restarting from the initial steps(if not dangerous) is a good way when trying to understand/solve the error. The certificate can also be verified with the s_client command (-servername and -connect are useful options): /docs/manmaster/man1/s_client.html

NOTES

s_client can be used to debug SSL servers. To connect to an SSL HTTP server the command:

 openssl s_client -connect servername:443

If -servername is not provided, the TLS SNI extension will be populated with the name given to -connect if it follows a DNS name format.

   s_client
       This implements a generic SSL/TLS client which can establish a transparent connection to a remote server speaking SSL/TLS. It's intended for testing purposes only and provides only
       rudimentary interface functionality but internally uses mostly all functionality of the OpenSSL ssl library.
1 Like
5

I faced a similar problem. Detailed steps mentioned here: https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4900#note_292031191