Debugging certificate problems in `prepare environment` stage

Infrastructure as Code & Cloud Native
, ,
1

I’m using a self-hosted Gitlab instance with Gitlab Runner, installed by Gitlab itself via a Kubernetes connector, running on an OpenShift Origin 3.11 cluster. The problem described has started after our upgrade to 13.8, however, another Gitlab Runner connected to the same Gitlab instance, but not installed via Gitlab, is not having this problem.

It seem that our jobs can’t properly initialize because of some SSL certificate:

Preparing environment
Waiting for pod gitlab-managed-apps/runner-tyvrmcpx-project-986-concurrent-02gkf4 to be running, status is Pending
Waiting for pod gitlab-managed-apps/runner-tyvrmcpx-project-986-concurrent-02gkf4 to be running, status is Pending
	ContainersNotReady: "containers with unready status: [build helper]"
	ContainersNotReady: "containers with unready status: [build helper]"
ERROR: Job failed (system failure): prepare environment: error dialing backend: x509: certificate has expired or is not yet valid. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information

My problem is that I can’t find out which server that is. Is there any way to found out which command produced that error message?

2

If someone from the future finds this, the problem is described in 1736286 – error dialing backend: remote error: tls: internal error, which is basically about unapproved node certificates in OpenShift. My original error message had probably something to do desynchronised clocks on the nodes, which has cloaked the root problem.

1 Like