Certificate issue with gitlab runner cloning repository

GitLab CI/CD
, ,
1

installed gitlab runner version=11.3.1. registered the runner like

gitlab-runner register --tls-ca-file /etc/gitlab-runner/certs/ca-bundle.cr

then got the error below

2018-12-14T18:45:42.817790090Z standard_init_linux.go:190: exec user process caused "argument list too long

from my reading, this is because ca-bundle.cr has more entries than runner/docker can handle.

If I register runner without specifying --tls-ca-file like gitlab-runner register, got another error as SSL certificate problem: unable to get issuer certificate, so runner was looking for CA.

Tried the option to bypass ssl verification by adding the following in config.toml

environment = ["GIT_SSL_NO_VERIFY=true"]

This option works. However, this poses a security risk that we dont want.

Any advice on how to let runner work with CA?

Thanks.

copying config.toml below as a reference

  name = "maven package docker build"
  url = "https://gitlab.abc.com/"
  token = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
  tls-ca-file = "/etc/gitlab-runner/certs/ca-bundle.crt"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker:latest"
    privileged = true
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
2

you could make a CA file with only the CA root file for the certificate on your gitlab host, to limit the entries/size. Maybe not optimal, but it might work.

1 Like
3

should believe that you mean having the CA root file on gitlab-ruuner (not gitlab host).

I indeed deleted a lot of certs in ca-bundle.crt file, and left a couple of root certs and added intermediate certs in the ca-bundle.crt. This time the error message is:

docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN https://gitlab.abc.com
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://gitlab.abc.com/v2/: x509: certificate signed by unknown authority

Apparently, when login to gitlab internal registry, the ca is unknown. Then i did the following:

(1) on gitlab-runner host, added ca.crt /etc/docker/certs.d/gitlab.abc.com (ca.crt is the same as ca-bundle.crt. but need to change the name to ca.crt according to docker documentation)
(2) change gitlab-config file with volumes = ["/cache", “/etc/docker/certs.d:/etc/docker/certs.d”]
(3) tested it with "docker exec adoring_darwin docker login -u xyz -p xyz gitlab.abc.com (of course, need to create the container of dind first). It worked when login to registry
(4) re-ran the ci/cd job, got the error mentioned above (…certificate signed by unknown authority).

Pls note I am using ca signed cert, not self signed cert. In fact, believe the issue that I encountered is similar to https://gitlab.com/gitlab-org/gitlab-runner/issues/1842. Looks like one can bypass it with --insecure-registry or customized dind images (I havent played with either option)

any ideas. Thanks

4

Updates:

Finally made it work by adding the following in .gitlab-ci.yml

services:
command: ["–insecure-registry", “gitlab.abc.com:4567”]
(credit to https://gitlab.com/gitlab-org/gitlab-runner/issues/1678#note_52465839)

This is a work around.

5

That’s what I meant, you need the root CA cert on the runner, so it can verify the cert of your site, which should already pass the intermediates. If that doesn’t help, you can try to do the docker login with the following option:
–tlscacert yourcafile.pem.
Which still feels a bit hacky, but better then untrusted

6

thanks @bartj

looks like that I have to put root ca certs and intermediate certs together. If i use only root ca certs, there is an error like “SSL certificate problem: unable to get local issuer certificate…”

docker login command itself doesnt have the option of --tlscacert. One can set the repository client up following https://docs.docker.com/engine/security/certificates/. It works outside gitlab-ruuner, even with dind. However, if withing gitlab-runner, still have the same issue as " certificate signed by unknown authority".

seems that we need to use the work around as mentioned in the previous post. Furthermore, seems to me that it might be a bug with the current version of gitlab/gitlab-runner-helper image.