Old SSL certificate still being served after renewing it

A few weeks ago I installed a new ssl certificate to replace an expiring one. The file name for the .crt and .key remained the same. Only the contents changed.

Now when I go to my site it says the the certificate has expired and I see that it is using the old certificate chain. I can confirm the contents of the crt and key are the updated ones.

Output of gitlab-ctl status

run: gitlab-workhorse: (pid 30354) 18091s; run: log: (pid 15271) 21131317s
run: logrotate: (pid 7332) 91s; run: log: (pid 30308) 21128931s
run: mailroom: (pid 30436) 18060s; run: log: (pid 15266) 21131317s
run: nginx: (pid 3586) 1761s; down: log: 0s, normally up, want up
run: postgresql: (pid 30446) 18059s; run: log: (pid 10566) 13738330s
run: redis: (pid 30448) 18059s; run: log: (pid 10696) 13738292s
run: sidekiq: (pid 30463) 18057s; run: log: (pid 15264) 21131317s
run: unicorn: (pid 30487) 18053s; run: log: (pid 15267) 21131317s

Output of gitlab-ctl tail nginx

2016/11/28 21:26:20 [crit] 6852#0: *13180076 SSL_shutdown() failed (SSL: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init) while SSL handshaking, client: 54.208.238.72, server: 0.0.0.0:443
2016/11/28 21:26:23 [crit] 6852#0: *13180077 SSL_shutdown() failed (SSL: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init) while SSL handshaking, client: 54.208.238.72, server: 0.0.0.0:443

Those critical messages happen every 3 seconds.

I’ve restarted the nginx process but it did nothing. I’ve also ran gitlab-ctl reconfigure but to no avail. My /etc/gitlab/gitlab.rb file has not changed. It still has the nginx ssl path

nginx[‘ssl_certificate’] = “/etc/pki/tls/certs/sitename.com.crt”
nginx[‘ssl_certificate_key’] = “/etc/pki/tls/private/sitename.com.key”

And it also has the external_url

external_url ‘https://sitename.com/’

I’ve run into this same problem. My old cert expired, and I have a new one which I attempted to install using instructions here.

I’m running GitLab Community Edition 10.3.3 (which is up to date).

Cert was put in
/etc/gitlab/trusted-certs/

I then ran
gitlab-ctl reconfigure

The symlink was properly created in
/opt/gitlab/embedded/ssl/certs

When I browse to the site or check the SSL cert using an online checker, however, the old cert is still loaded.

All of the “ssl_certificate” lines in /etc/gitlab/gitlab.rb are commented out.

Is the documentation on this out of date? How do we update our SSL certificates?

I’m having the same problem. I’m on 11.7.0. Any resolution to this?

To those affected:

I suggest trying:
sudo gitlab-ctl restart

If that doesn’t help, please paste the output of:
sudo gitlab-rake gitlab:check
And I will do my best to help troubleshoot.

All the best,
Greg M

My new certificate (created with gitlab-ctl reconfigure) is not showing up as being renewed in browser. I had moved my old cert and key into another folder, and then compared the old one to new one, and yes they are different.

gitlab-ctl status:
run: alertmanager: (pid 2859) 2407s; run: log: (pid 2236) 4137716s
run: crond: (pid 2875) 2407s; run: log: (pid 17147) 3547s
run: gitaly: (pid 2901) 2406s; run: log: (pid 2216) 4137716s
run: gitlab-monitor: (pid 2927) 2405s; run: log: (pid 2223) 4137716s
run: gitlab-workhorse: (pid 2946) 2405s; run: log: (pid 2219) 4137716s
run: grafana: (pid 2960) 2404s; run: log: (pid 2234) 4137716s
run: logrotate: (pid 2981) 2404s; run: log: (pid 2220) 4137716s
run: mattermost: (pid 8839) 0s; run: log: (pid 2221) 4137716s
run: nginx: (pid 8810) 2s; run: log: (pid 17050) 3552s
run: postgresql: (pid 3097) 2402s; run: log: (pid 2217) 4137716s
run: redis: (pid 3108) 2401s; run: log: (pid 2208) 4137716s
run: registry: (pid 3127) 2401s; run: log: (pid 2228) 4137716s
run: sidekiq: (pid 3147) 2400s; run: log: (pid 2211) 4137716s
run: unicorn: (pid 3175) 2399s; run: log: (pid 2210) 4137716s

gitlab-ctl tail nginx:
==> /var/log/gitlab/nginx/gitlab_access.log <==
==> /var/log/gitlab/nginx/error.log <==
==> /var/log/gitlab/nginx/gitlab_error.log <==
==> /var/log/gitlab/nginx/state <==
==> /var/log/gitlab/nginx/current <==
2020-01-08_13:22:10.97316 2020/01/08 14:22:10 [emerg] 429#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2020-01-08_13:22:11.47335 2020/01/08 14:22:10 [emerg] 429#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2020-01-08_13:22:11.97358 2020/01/08 14:22:10 [emerg] 429#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2020-01-08_13:22:12.47375 2020/01/08 14:22:10 [emerg] 429#0: bind() to 0.0.0.0:80 failed (98: Address already in use)
2020-01-08_13:22:12.97393 2020/01/08 14:22:10 [emerg] 429#0: still could not bind()

lsof -i :80
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
nginx 1455 root 13u IPv4 32044 0t0 TCP *:http (LISTEN)
nginx 1459 nginx 13u IPv4 32044 0t0 TCP *:http (LISTEN)
nginx 1460 nginx 13u IPv4 32044 0t0 TCP *:http (LISTEN)
[root@ip-10-150-2-5 gitlab]# ps fax | grep nginx
4132 pts/0 S+ 0:00 | _ grep --color=auto nginx
1455 ? Ss 0:00 nginx: master process /usr/sbin/nginx
1459 ? S 23:59 _ nginx: worker process
1460 ? S 34:17 _ nginx: worker process
17049 ? Ss 0:00 _ runsv nginx
17050 ? S 0:00 | _ svlogd -tt /var/log/gitlab/nginx
4098 ? Ss 0:00 | _ /opt/gitlab/embedded/sbin/nginx -p /var/opt/gitlab/nginx

gitlab-rake gitlab:check

Checking GitLab subtasks …

Checking GitLab Shell …

GitLab Shell: … GitLab Shell version >= 9.3.0 ? … OK (9.3.0)

Running /opt/gitlab/embedded/service/gitlab-shell/bin/check

Check GitLab API access: OK

Redis available via internal API: OK

Access to /var/opt/gitlab/.ssh/authorized_keys: OK

gitlab-shell self-check successful

Checking GitLab Shell … Finished

Checking Gitaly …

Gitaly: … default … OK

Checking Gitaly … Finished

Checking Sidekiq …

Sidekiq: … Running? … yes

Number of Sidekiq processes … 1

Checking Sidekiq … Finished

Checking Incoming Email …

Incoming Email: … Reply by email is disabled in config/gitlab.yml

Checking Incoming Email … Finished

Checking LDAP …

LDAP: … LDAP is disabled in config/gitlab.yml

Checking LDAP … Finished

Checking GitLab App …

Git configured correctly? … yes

Database config exists? … yes

All migrations up? … yes

Database contains orphaned GroupMembers? … no

GitLab config exists? … yes

GitLab config up to date? … yes

Log directory writable? … yes

Tmp directory writable? … yes

Uploads directory exists? … yes

Uploads directory has correct permissions? … yes

Uploads directory tmp has correct permissions? … yes

Init script exists? … skipped (omnibus-gitlab has no init script)

Init script up-to-date? … skipped (omnibus-gitlab has no init script)

Projects have namespace: …

ops / old-deploy … yes

ops / sql-snippets … yes

ops / bash-utilities … yes

ops / ansible … yes

ops / deploy … yes

Admin Name / local-proxy … yes

Another Admin Name / sitename … yes

Another Admin Name / sitename … yes

Another Admin Name / sitename … yes

SiteName / SiteName … yes

SiteName / SiteName … yes

SiteName / SiteName … yes

SiteName / SiteName … yes

Redis version >= 2.8.0? … yes

Ruby version >= 2.5.3 ? … yes (2.6.3)

Git version >= 2.22.0 ? … yes (2.22.0)

Git user has default SSH configuration? … yes

Active users: … 10

Checking GitLab App … Finished

Checking GitLab subtasks … Finished

I’m not too sure why nginx is having an issue binding to port 80… could this possibly be the issue?
Any help here would be greatly appreciated, thanks!

Hi,

it seems that the old Nginx process is still running, having loaded the TLS certificates into memory.

Try forcing the current process to stop first.

gitlab-ctl stop nginx

Then start it again.

gitlab-ctl start nginx

Verify in the logs that binding port 80 errors are gone.

Cheers,
Michael

I tried restarting nginx with no luck, so I just rebooted the aws instance itself, and all was magically just working.

Thanks for your reply, though.