Login broken after Update

I just updated to the most recent version of gitlab-omnibus. None of my users can seem to login now, and I am not incredibly familiar with troubleshooting issues in this environment.

GitLab 10.1.0
GitLab Shell 5.9.3
GitLab Workhorse v3.2.0
GitLab API v4
Gitaly 0.43.0
Git 2.13.5
Ruby 2.3.5p376
Rails 4.2.8
postgresql 9.6.5

I’m not 100% certain which version of gitlab-omnibus I came from, but it hadn’t been more than a month since I updated, so probably a later 9.x version.

The error I’m getting is

Could not authenticate you from Ldapmain because "Ssl connect returned=1 errno=0 state=error: certificate verify failed".

at the login screen when anyone goes to login. When I tail all of the logs using gitlab-ctl tail, I get the following error on login attempt.

{"method":"POST","path":"/users/auth/ldapmain/callback","format":"html","controller":"OmniauthCallbacksController","action":"failure","status":302,"duration":6.6,"view":0.0,"db":0.0,"location":"https://gitlab.mycompany.com/users/sign_in","time":"2017-10-24T01:37:36.538Z","params":{"utf8":"✓","authenticity_token":"[FILTERED]","username":"my.name","password":"[FILTERED]"},"remote_ip":"192.168.0.81","user_id":null,"username":null}

Any help on the next step in troubleshooting would be helpful. I don’t see any errors coming from anywhere else, and I have no way to login to gitlab currently.

I’ve had a problem similar to this before, and it was due to variables in the gitlab.rb file that had changed names. That doesn’t seem to be the case now, but I’m adding my gitlab.rb settings for LDAP just in case.

My current gitlab.rb ldap

gitlab_rails['ldap_enabled'] = true
gitlab_rails['ldap_servers'] = YAML.load <<-'EOS' # remember to close this block with 'EOS' below
  main: # 'main' is the GitLab 'provider ID' of this LDAP server
    label: 'LDAP'
    host: '192.168.0.101'
    port: 636
    uid: 'sAMAccountName'
    method: 'ssl' # "tls" or "ssl" or "plain"
    bind_dn: 'cn=Ldap_auth,OU=Service Accounts,DC=mycompany,DC=com'
    password: 'superSecurePassword'
    active_directory: true
    allow_username_or_email_login: false
    base: 'ou=mc Users,dc=mycompany,dc=com'
    user_filter: ''
    sync_ssh_keys: false
EOS

Possible Fix (with more questions)

I found this issue, which led me to the release notes here.

It says that verify_certificates in the ldap config is now defaulting to true for security. When I added the line verify_certificates: false to my config file, everything starts working again.

Here’s the question. Is that how I should fix this? In the first post I linked, some people said they were setting a ca_file field. With my configuration above, I’m really not sure which makes more sense.

So that’s a “Fix” the same way disabling certificate validation is a “Fix” when people do it with their Git/Bitbucket/Github SSL keys, or when they do the same thing with security overrides in their browsers. It opens your gitlab up to man in the middle (impersonation) attacks. Something which is NOT your LDAP server will now be able to insert itself in the middle and siphon all your user’s corporate login credentials. What sane corporation operating an LDAP server wants that risk vector left open?

Are you aware that this is what you’re doing? Are you not interested in escalating to your internal IT people to find out why their corporate LDAP server uses a self signed certificate? If that’s what they’re doing? In my opinion the use of a self signed cert is a bad practice. Disabling certificate validation checking is a disastrously bad practice.

Reading the issue it seems it’s better to manually configure a CA file than to disable validation, but it would be better if your IT people actually bought real certificates from a real trustable CA.

Yongsheng Xie @schenkerx commented 2 weeks ago

It seems that adding ca_file works. I just forgot to run gitlab-ctl reconfigure after changing configurations.

That will allow certificate validation to work, and protect you from a MITM attack. You want that right?

Trust chains are key elements of SSL. Without them, you have nothing. Less than nothing, since you think
you have All The Securitay, but you Has Nones.