SSL certificate generation fails on ubuntu 18LTS

Good Day :slight_smile:

I have tried to setup GitLab CE “12.8.0-ce.0” on a “clean” Ubuntu 18.04 server using the following commands:

curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
sudo EXTERNAL_URL="https://subdomain.mydomain.com" apt-get install gitlab-ce

The installation runs fine until it hits:

      * ruby_block[create certificate for subdomain.mydomain.com] action run
        
        ================================================================================
        Error executing action `run` on resource 'ruby_block[create certificate for subdomain.mydomain.com]'
        ================================================================================
        
        RuntimeError
        ------------
        [subdomain.mydomain.com] Validation failed, unable to request certificate
        
        Cookbook Trace:
        ---------------
        /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb:111:in `block (3 levels) in class_from_file'
        
        Resource Declaration:
        ---------------------
        # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb
        
        108:     ruby_block "create certificate for #{new_resource.cn}" do # ~FC014
        109:       block do
        110:         unless (all_validations.map { |authz| authz.status == 'valid' }).all?
        111:           fail "[#{new_resource.cn}] Validation failed, unable to request certificate"
        112:         end
        113: 
        114:         begin
        115:           newcert = acme_cert(order, new_resource.cn, mykey, new_resource.alt_names)
        116:         rescue Acme::Client::Error => e
        117:           fail "[#{new_resource.cn}] Certificate request failed: #{e.message}"
        118:         else
        119:           Chef::Resource::File.new("#{new_resource.cn} SSL new crt", run_context).tap do |f|
        120:             f.path    new_resource.crt
        121:             f.owner   new_resource.owner
        122:             f.group   new_resource.group
        123:             f.content newcert
        124:             f.mode    00644
        125:           end.run_action :create
        126:         end
        127:       end
        128:     end
        
        Compiled Resource:
        ------------------
        # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb:108:in `block in class_from_file'
        
        ruby_block("create certificate for subdomain.mydomain.com") do
          action [:run]
          default_guard_interpreter :default
          declared_type :ruby_block
          cookbook_name "letsencrypt"
          block #<Proc:0x00005622956f7698@/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb:109>
          block_name "create certificate for subdomain.mydomain.com"
        end
        
        System Info:
        ------------
        chef_version=14.14.29
        platform=ubuntu
        platform_version=18.04
        ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
        program_name=/opt/gitlab/embedded/bin/chef-client
        executable=/opt/gitlab/embedded/bin/chef-client
        
      
      ================================================================================
      Error executing action `create` on resource 'acme_certificate[staging]'
      ================================================================================
      
      RuntimeError
      ------------
      ruby_block[create certificate for subdomain.mydomain.com] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb line 108) had an error: RuntimeError: [subdomain.mydomain.com] Validation failed, unable to request certificate
      
      Cookbook Trace:
      ---------------
      /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb:111:in `block (3 levels) in class_from_file'
      
      Resource Declaration:
      ---------------------
      suppressed sensitive resource output
      
      Compiled Resource:
      ------------------
      suppressed sensitive resource output
      
      System Info:
      ------------
      chef_version=14.14.29
      platform=ubuntu
      platform_version=18.04
      ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
      program_name=/opt/gitlab/embedded/bin/chef-client
      executable=/opt/gitlab/embedded/bin/chef-client
      
    
    ================================================================================
    Error executing action `create` on resource 'letsencrypt_certificate[subdomain.mydomain.com]'
    ================================================================================
    
    RuntimeError
    ------------
    acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 25) had an error: RuntimeError: ruby_block[create certificate for subdomain.mydomain.com] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb line 108) had an error: RuntimeError: [subdomain.mydomain.com] Validation failed, unable to request certificate
    
    Cookbook Trace:
    ---------------
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb:111:in `block (3 levels) in class_from_file'
    
    Resource Declaration:
    ---------------------
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/recipes/http_authorization.rb
    
      5: letsencrypt_certificate site do
      6:   crt node['gitlab']['nginx']['ssl_certificate']
      7:   key node['gitlab']['nginx']['ssl_certificate_key']
      8:   notifies :run, "execute[reload nginx]", :immediate
      9:   notifies :run, 'ruby_block[display_le_message]'
     10:   only_if { omnibus_helper.service_up?('nginx') }
     11: end
    
    Compiled Resource:
    ------------------
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/recipes/http_authorization.rb:5:in `from_file'
    
    letsencrypt_certificate("subdomain.mydomain.com") do
      action [:create]
      updated true
      updated_by_last_action true
      default_guard_interpreter :default
      declared_type :letsencrypt_certificate
      cookbook_name "letsencrypt"
      recipe_name "http_authorization"
      crt "/etc/gitlab/ssl/subdomain.mydomain.com.crt"
      key "/etc/gitlab/ssl/subdomain.mydomain.com.key"
      alt_names []
      cn "subdomain.mydomain.com"
      only_if { #code block }
    end
    
    System Info:
    ------------
    chef_version=14.14.29
    platform=ubuntu
    platform_version=18.04
    ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client
    
Recipe: gitlab::gitlab-rails
  * execute[clear the gitlab-rails cache] action run
    - execute /opt/gitlab/bin/gitlab-rake cache:clear
Recipe: gitaly::enable
  * runit_service[gitaly] action hup
    - send hup to runit_service[gitaly]
Recipe: <Dynamically Defined Resource>
  * service[gitlab-workhorse] action restart
    - restart service service[gitlab-workhorse]
  * service[registry] action restart
    - restart service service[registry]
  * service[node-exporter] action restart
    - restart service service[node-exporter]
  * service[gitlab-exporter] action restart
    - restart service service[gitlab-exporter]
  * service[redis-exporter] action restart
    - restart service service[redis-exporter]
  * service[prometheus] action restart
    - restart service service[prometheus]
Recipe: monitoring::prometheus
  * execute[reload prometheus] action run
    - execute /opt/gitlab/bin/gitlab-ctl hup prometheus
Recipe: <Dynamically Defined Resource>
  * service[alertmanager] action restart
    - restart service service[alertmanager]
  * service[postgres-exporter] action restart
    - restart service service[postgres-exporter]
  * service[grafana] action restart
    - restart service service[grafana]

Running handlers:
There was an error running gitlab-ctl reconfigure:

letsencrypt_certificate[subdomain.mydomain.com] (letsencrypt::http_authorization line 5) had an error: RuntimeError: acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 25) had an error: RuntimeError: ruby_block[create certificate for subdomain.mydomain.com] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb line 108) had an error: RuntimeError: [subdomain.mydomain.com] Validation failed, unable to request certificate

Running handlers complete
Chef Client failed. 588 resources updated in 03 minutes 28 seconds
dpkg: error processing package gitlab-ce (--configure):
 installed gitlab-ce package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 gitlab-ce
E: Sub-process /usr/bin/dpkg returned an error code (1)

I have tried the following using ,timeshift to reset my server to an clean install each time:

  • setting up SSL manually over with “sudo certbot --nginx” -> fails
  • changing from Nginx to Apache2 and retry “sudo certbot --apache” -> fails
  • created a nextcloud instance first and ran “sudo certbot --apache” -> success

Then installing GitLab CE as above but with http instead of https ,which does succeed and afterwards setting up SSL and apache manually by:

web_server['external_users'] = ['www-data']
nginx['enable'] = false
letsencrypt['enable'] = true

in /etc/gitlab/gitlab.rb. After that running sudo gitlab-ctl reconfigure again fails with the above error. But it actually is accessible over https now means i can login create repos and stuff all over SSL but when i try to clone a repository to my local machine i get a

fatal: unable to access 'https://subdomain.mydomain.com/user/project.git/': The requested URL returned error: 500

I hope somebody can help me or point to the right direction. If you need more information please tell me i tried to be as complete as i could, but i may have missed something.
Thanks for having a look :slight_smile:

So i tried to follow this “guide”:
https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4900#note_292031191
but using ce instead of ee that did not work either, same error message as above.
So running sudo gitlab-ctl renew-le-certs fails as well as sudo gitlab-ctl reconfigure on 12.8.0-ce.0 and 12.8.1-ce.0 but there are actually SSL-files creates in /etc/gitlab/ssl/ namely:
subdomain.mydomain.com.crt
subdomain.mydomain.com.key
subdomain.mydomain.com.key-staging
first Error i see is:
Error executing action run on resource 'ruby_block[create certificate for subdomain.mydomain.com]' but don’t now what to do with that.
I use the default ports 443 and 80 and the server is reachable from the outside.