I have configured a self-hosted GitLab with runners and enables the docker registry.
When I try to login from a machine in the same network, all seems to work fine:
sudo docker login -u xxxx -p xxxx my.domain:4567
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
When I try to run it from within a CI runner (also in the same network), there seems to be an issue with the certificates:
$ docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://xxxx/v2/: x509: certificate signed by unknown authority
ERROR: Job failed: exit code 1
Any idea what I am missing?
–
.gitLab-ci.yml:
image: docker:latest
services:
- docker:dind
stages:
- build
- test
- release
variables:
TEST_IMAGE: xxxx:$CI_COMMIT_REF_NAME
RELEASE_IMAGE: xxxx:latest
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
...
You have to configure the runner accordingly, look here.
OK, so I checked that page, but that doesn’t seem to help. So, the main problem seems to be that the CA root certificates are not available in the docker container, while they are in the runner machine itself.
1. Default : GitLab Runner reads system certificate store and verifies the GitLab server against the CA’s stored in system.
Doesn’t seem to be true: it is available in the system certificate store, but in the docket container that’s started from this machine.
2. GitLab Runner reads the PEM ( DER format is not supported ) certificate from predefined file:
Tried that, doesn’t work. There are 2 .crt files, so I concatenated them in one file names /etc/gitlab-runner/certs/my.domain.cert. I still get the same error. gitLab-runner is executed as root
# ps ax -u root | grep gitlab-runner
13155 ? Ssl 56:22 /usr/lib/gitlab-runner/gitlab-runner run --working-directory /home/gitlab-runner --config /etc/gitlab-runner/config.toml --service gitlab-runner --syslog --user gitlab-runner
I also tried to copy the certificates to /home/gitLab/certs , but I still get the same error.
3. GitLab Runner exposes tls-ca-file option during registration and in config.toml under the [[runners]] section which allows you to specify a custom file with certificates. This file will be read every time when runner tries to access the GitLab server.
did that, still the same problem… I used the concatenated .crt file of step 2.
The question now is: how can I add the CA root certificates to a docker runner? (I might add it to variables, drop them into the right directory and run update-ca-certificates , but it seems to be there has to be a better way)
New strategy: I mount the volume with the certificates into the Docker container, and try to install them using update-ca-certificates (Docker:latest-container uses alpine Linux, so I thought that should work. However, when I run update-ca-certificates, they are not installed, due to these warnings:
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
WARNING: ca-certificates-xxx.crt does not contain exactly one certificate or CRL: skipping
How does one install (multiple) ca-certificates in alpine Linux?
more trials, other strategy: Docker registry dind
I’m using your option 3 with tls-ca-file file which includes all root certificates (entire chain) and it works for the git clone and also for the integrated Docker registry. Do not include the server certificate - this is not necessary! I pass this during the runner registration, there is an option.
Enable the debug mode for the CI build to see what is going on in detail.
Good luck!