Gitlab CE LetsEncrypt behind Apache Proxy unable to create certificates


I am running Gitlab CE 11.11.0 behind an apache proxy with this config

ServerSignature Off

ProxyPreserveHost On
  # Ensure that encoded slashes are not decoded but left in their encoded state.
  AllowEncodedSlashes NoDecode
  <Location />
    # New authorization commands for apache 2.4 and up
    Require all granted
    #Allow forwarding to gitlab-workhorse
    ProxyPassReverse http://{DOMAIN}/
  # Apache equivalent of nginx try files
  RewriteEngine on
  #Forward all requests to gitlab-workhorse except existing files like error documents
  RewriteCond %{REQUEST_URI} ^/uploads/.*
  RewriteRule .*{REQUEST_URI} [P,QSA,NE]
  # needed for downloading attachments
  #DocumentRoot /opt/gitlab/embedded/service/gitlab-rails/public
  #Set up apache error documents, if back end goes down (i.e. 503 error) then a maintenance/deploy page is thrown up.
  ErrorDocument 404 /404.html
  ErrorDocument 422 /422.html
  ErrorDocument 500 /500.html
  ErrorDocument 502 /502.html
  ErrorDocument 503 /503.html

{DOMAIN} = my gitlab domain (

and enabled LE in gitlabconf like so:

## Lets Encrypt
letsencrypt['enable'] = true
letsencrypt['auto_renew_hour'] = "12"
letsencrypt['auto_renew_minute'] = "30"
letsencrypt['auto_renew_day_of_month'] = "*/7"
letsencrypt['auto_renew'] = true

However, when recreating config files with gitlab I receive this error:

    Error executing action `create` on resource 'letsencrypt_certificate[]'

    acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 20) had an error: RuntimeError: [] Validation failed for domain

    Cookbook Trace:
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/providers/certificate.rb:93:in `block (2 levels) in class_from_file'
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/providers/certificate.rb:68:in `map'
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/providers/certificate.rb:68:in `block in class_from_file'

    Resource Declaration:
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/recipes/http_authorization.rb

      3: letsencrypt_certificate site do
      4:   fullchain node['gitlab']['nginx']['ssl_certificate']
      5:   key node['gitlab']['nginx']['ssl_certificate_key']
      6:   notifies :run, "execute[reload nginx]", :immediate
      7:   notifies :run, 'ruby_block[display_le_message]'
      8: end

    Compiled Resource:
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/recipes/http_authorization.rb:3:in `from_file'

    letsencrypt_certificate("") do
      action [:create]
      updated true
      updated_by_last_action true
      default_guard_interpreter :default
      declared_type :letsencrypt_certificate
      cookbook_name "letsencrypt"
      recipe_name "http_authorization"
      fullchain "/etc/gitlab/ssl/"
      key "/etc/gitlab/ssl/"
      alt_names []
      cn ""

    System Info:
    ruby=ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]

Running handlers:
There was an error running gitlab-ctl reconfigure:

letsencrypt_certificate[] (letsencrypt::http_authorization line 3) had an error: RuntimeError: acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 20) had an error: RuntimeError: [] Validation failed for domain

I think its because of the proxy, so that /.well-known is not correctly forwarded to gitlab in the background, but the apache proxy should just pass every request to gitlab if I am not mistaken.

Does anyone know what could be the error?

Hey, could you please check LetsEncrypt certificates fail in domain validation. I was looking if anybody else had a similiar issue but this is the only thing I could find.

Hello, I’ve tried the suggested solutions from the other thread but it didnt do it for me.

This is my config file (I redacted a few things):

This is added at the end of the gitlab.rb file

letsencrypt['enable'] = true
letsencrypt['auto_renew'] = true
letsencrypt['auto_renew_hour'] = 0
letsencrypt['auto_renew_minute'] = 30
letsencrypt['auto_renew_day_of_month'] = "*/4"
nginx['custom_gitlab_server_config'] = "location /.well-known/acme-challenge/ {\n root /var/opt/gitlab/nginx/www/; \n}\n"
nginx['redirect_http_to_https'] = true
nginx['redirect_http_to_https_port'] = 80

Note that I disabled (for whatever reason, I cannot remember) “nginx” in the gitlab.rb file - might be an issue with my setup? Because its now Apache (Proxy Redirect) > Gitlab and not Apache (Proxy) > Nginx > Gitlab.