LetsEncrypt certificate renewal issues for Helm-installed Gitlab

I had this issue where the MinIO TLS certificate wasn’t being renewed, I don’t think it is a MinIO-specific problem.

This is my solution for solving it. A lot of the explanations below assume that you know how TLS certificates are issued and how certmanager works.

Anyway: Gitlab uses certmanager under the hood to create and renew the certificates automatically. Certmanager reads Ingress manifests and depending on the annotations and properties, will create the required “challenge solvers” (ingress + pod in kubernetes) to prove you own the domain (in this case minio.domain.tld).

I suppose a poorly timed eviction caused the challenge solving pod to be recreated (and its name changed), and the ingress must have lost track of it. Causing 503 errors whenever the challenge request was sent.

To solve the whole situation, I removed the TLS annotations from the minio ingress (certmanager will automatically remove all other resources created to solve the challenges). And then I added it back in, certmanager will once again create the challenge solvers (properly this time), and by the look of it every works as intended without any dangling resources.

Hope this helps.