500 error after Keycloak login -- certificate verify failed unable to get local issuer certificate

I’m integrating a Gitlab CE instance with a Keycloak instance for SSO following the guide below:

I’ve been working at this off and on for a couple of weeks now and I cannot get it to work. The integration is enabled and takes me to the Keycloak login page, but after authenticating and redirecting me to Gitlab I receive a 500 error.

The logs accompanying the 500 errors return the IP of the classic load balancer:

==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/users/auth/oauth2_generic/callback?state=9a763107392395d5def96bb73ed3767c74556c385178ec79&session_state=1df7a541-69ba-47e6-a026-2e952f98241c&code=[FILTERED]" for 192.168.1.15 at 2021-02-16 14:14:19 +0000

<--- [using openid_connect]
OpenIDConnect::Discovery::DiscoveryFailed (SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate)): auth
----- [using oauth2_generic]
Faraday::SSLError (SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate)):
--->

config/initializers_before_autoloader/100_patch_omniauth_oauth2.rb:11:in `callback_phase'
lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'
lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'
lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'
lib/gitlab/metrics/transaction.rb:56:in `run'
lib/gitlab/metrics/rack_middleware.rb:16:in `call'
lib/gitlab/request_profiler/middleware.rb:17:in `call'
lib/gitlab/jira/middleware.rb:19:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:21:in `call'
lib/gitlab/middleware/multipart.rb:172:in `call'
lib/gitlab/middleware/read_only/controller.rb:50:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
lib/gitlab/middleware/request_context.rb:23:in `call'
config/initializers/fix_local_cache_middleware.rb:9:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:76:in `call'
lib/gitlab/middleware/release_env.rb:12:in `call'

Accordingly I’ve tried adding the certs to /etc/gitlab/trusted-certs and I’ve tried terminating ssl at the load balancer with an ACM cert using the prescribed nginx config:

   nginx['listen_port'] = 80
   nginx['listen_https'] = false
   letsencrypt['enable'] = false

as well as passing traffic through to the server unaltered and terminating ssl with a letsencrypt cert.

I don’t know why, but load balancing as prescribed in the official docs for AWS (Installing GitLab on Amazon Web Services (AWS) | GitLab) seems to break this integration. Any insights or advice would be greatly appreciated!

This poster appears to have the same issue under different circumstances. This may be a bug.

I thought the issue might be the call originating from the load balancer’s ip so I added the following to my config without success:

# Each address is added to the the NGINX config as 'set_real_ip_from <address>;'
nginx['real_ip_trusted_addresses'] = [ '192.168.1.0/24' ]
nginx['real_ip_header'] = 'X-Forwarded-For'
nginx['real_ip_recursive'] = 'on'

I fear the issue may be embedded somewhere in the gitlab/middleware libs

I am experiencing the same error, the exact same setup was followed on Amazon Linux 2 and was working with Keycloak, but when I attempted to integrate the 2 on a RHEL 7/8/CentOS7 system I get a 500 error.

I am not using a load balancer, using Nginx to force https; I tried to use LetsEncrypt to get a cert but the RHSM threw an error, and I was not able to login (at the momentt). I did not see any other errors, it appears that Keycloak is setting the session and attempting to exchange the token with GitLab but then I get this annoying 500 error. Any help would be greatly appreciated!

I can corroborate that I’m experiencing this issue on RHEL 7/8. Further Testing eliminated the load balancer/architecture from the list of possible causes

The root cause of the issue is an undefined rails ev, SSL_CERT_FILE.
Adding this line to my configuration resolved the errors:
gitlab_rails[‘env’] = {“SSL_CERT_FILE” => “”}

I have the same error using Keycloak with Gitlab deployed with helm chart. Do you have idea ho this can be fixed in the helm chart?