Hello,
after two days of searching for a working configuration by myself I now decided to ask for help here. Some time ago I installed GitLab on my Debian Jessie server by using the gitlab/gitlab-ce:latest Docker image. I installed a working configuration of two GitLab runners by the gitlab/gitlab-runner:latest Docker image. One of them is specialized on building Docker images:
concurrent = 4
check_interval = 0
[[runners]]
name = "Gitlab-Runner-Docker"
url = "[Anonymous]"
token = "[Anonymous]"
executor = "docker"
limit = 4
privileged = true
cache_dir = "cache"
[runners.docker]
tls_verify = false
image = "docker:latest"
privileged = true
disable_cache = false
volumes = ["/cache"]
shm_size = 0
cpuset_cpus = "4-7"
allowed_images = ["docker:latest"]
[runners.cache]
This runner is able to build images and push them to Docker Hub without any problem (The runner is tagged “docker”):
backend_deploy_master:
stage: deploy
tags:
- docker
only:
- master@[Anonymous]
services:
- docker:dind
environment:
name: backenend_deployment_master
url: [Anonymous]
script:
- docker build -t [Anonymous] .
- docker login -u [Anonymous] -p [Anonymous]
- docker push [Anonymous]
As I activated the Docker Container Registry, too, I now tried to push images to my local Registry instead of to Docker Hub. So i modified the above code to use my repository instead:
backend_deploy_master:
stage: deploy
tags:
- docker
only:
- master@[Anonymous]
services:
- docker:dind
environment:
name: backenend_deployment_master
url: [Anonymous]
script:
- docker build -t [Anonymous] .
- docker login -u [Anonymous] -p [Anonymous] docker.gitlab.[Anonymous].de
- docker push [Anonymous]
This fails at the docker login command with this error message:
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://docker.gitlab.[Anonymous].de/v2/: denied: access forbidden
My first thought was that my Docker internal registry is faulty, so I executed docker login -u [Anonymous] -p [Anonymous] docker.gitlab.[Anonymous].de on my host machine. Result was Login Succeeded and even pushing images was exactly no problem. Just to be sure I tried docker login -u [Anonymous] -p [Anonymous] docker.gitlab.[Anonymous].de on several other Linux machines, too. It always succeeded. Of course the [Anonymous] data was exactly the same all the time.
I decided to take a look at the registry logs for this. This is the log output if I run docker login on my host:
2018-08-22_10:34:15.07620 time="2018-08-22T10:34:15.076167544Z" level=warning msg="error authorizing context: authorization token required" environment=production go.version=go1.10.3 http.request.host=docker.gitlab.[Anonymous].de http.request.id=b33ea6bc-17ae-4a26-b163-0e943b844f5a http.request.method=GET http.request.remoteaddr=[Anonymous] http.request.uri="/v2/" http.request.useragent="docker/17.06.0-ce go/go1.8.3 git-commit/02c1d87 kernel/3.16.0-4-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/17.06.0-ce \\(linux\\))" instance.id=3c80e5c7-932a-417b-b78e-9e8d0604fa07 service=registry version=v2.6.2-2-g91c17ef
2018-08-22_10:34:15.07621 127.0.0.1 - - [22/Aug/2018:10:34:15 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/17.06.0-ce go/go1.8.3 git-commit/02c1d87 kernel/3.16.0-4-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/17.06.0-ce \\(linux\\))"
2018-08-22_10:34:15.21670 time="2018-08-22T10:34:15.21667828Z" level=info msg="response completed" environment=production go.version=go1.10.3 http.request.host=docker.gitlab.[Anonymous].de http.request.id=c28d24b5-8532-4052-910e-1e7ef99c987e http.request.method=GET http.request.remoteaddr=[Anonymous] http.request.uri="/v2/" http.request.useragent="docker/17.06.0-ce go/go1.8.3 git-commit/02c1d87 kernel/3.16.0-4-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/17.06.0-ce \\(linux\\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.23239ms http.response.status=200 http.response.written=2 instance.id=3c80e5c7-932a-417b-b78e-9e8d0604fa07 service=registry version=v2.6.2-2-g91c17ef
2018-08-22_10:34:15.21673 127.0.0.1 - - [22/Aug/2018:10:34:15 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "docker/17.06.0-ce go/go1.8.3 git-commit/02c1d87 kernel/3.16.0-4-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/17.06.0-ce \\(linux\\))"
This was the output if the GitLab runner tries to login:
2018-08-22_10:31:35.41572 time="2018-08-22T10:31:35.415677686Z" level=warning msg="error authorizing context: authorization token required" environment=production go.version=go1.10.3 http.request.host=docker.gitlab.[Anonymous].de http.request.id=e40f6776-e982-4a41-876e-501904e8c067 http.request.method=GET http.request.remoteaddr=172.17.0.8 http.request.uri="/v2/" http.request.useragent="docker/18.06.0-ce go/go1.10.3 git-commit/0ffa825 kernel/3.16.0-4-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.0-ce \\(linux\\))" instance.id=3c80e5c7-932a-417b-b78e-9e8d0604fa07 service=registry version=v2.6.2-2-g91c17ef
2018-08-22_10:31:35.41574 127.0.0.1 - - [22/Aug/2018:10:31:35 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/18.06.0-ce go/go1.10.3 git-commit/0ffa825 kernel/3.16.0-4-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/18.06.0-ce \\(linux\\))"
It seems like the GitLab runner fails to execute the steps 3-5 described in this document https://docs.gitlab.com/omnibus/architecture/registry/, but I have no idea why. Every other client I tried is able to login.
As I am not sure whether I configured something wrong I will paste the registry parts of my gitlab.rb in here, too:
################################################################################
## Container Registry settings
##! Docs: https://docs.gitlab.com/ce/administration/container_registry.html
################################################################################
registry_external_url 'https://docker.gitlab.[Anonymous].de'
### Settings used by GitLab application
gitlab_rails['registry_enabled'] = true
gitlab_rails['registry_host'] = "gitlab.docker.[Anonymous].de"
gitlab_rails['registry_port'] = "80"
gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry"
################################################################################
## Registry NGINX
################################################################################
registry_nginx['enable'] = true
registry_nginx['listen_port'] = 5226
registry_nginx['listen_https'] = false
registry_nginx['proxy_set_headers'] = {
"Host" => "$http_host",
"X-Real-IP" => "$remote_addr",
"X-Forwarded-For" => "$proxy_add_x_forwarded_for",
"X-Forwarded-Proto" => "https",
"X-Forwarded-Ssl" => "on"
}
The main GitLab and the Registry run on the same Docker container started with:
docker run --detach \
--hostname gitlab.[Anonymous].de \
--publish 5223:443 \
--publish 5224:80 \
--publish 5225:22 \
--publish 5226:5226 \
--publish 5227:5227 \
--name gitlab-ce \
--restart always \
--volume ~/gitlab_data/config:/etc/gitlab \
--volume ~/gitlab_data/logs:/var/log/gitlab \
--volume ~/gitlab_data/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
The GitLab runner is started with:
docker run -d --name gitlab-runner-docker --restart always \
-v /home/docker/gitlab-runner/gitlab-runner-docker:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:latest
Both, the main GitLab and the Registry run behind a Apache proxy encrypted with LetsEncrypt:
For gitlab.[Anonymous].de (Main GitLab):
ProxyPreserveHost On
ProxyPassMatch ^\/.well-known\/acme-challenge\/[abcdefghijklmnopqrstuvwxyz\/\-._ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890]*$ !
ProxyPass / http://localhost:5224/
ProxyPassReverse / http://localhost:5224/
For docker.gitlab.[Anonymous].de (Registry):
ProxyPreserveHost On
ProxyPassMatch ^\/.well-known\/acme-challenge\/[abcdefghijklmnopqrstuvwxyz\/\-._ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890]*$ !
ProxyPass / http://localhost:5226/
ProxyPassReverse / http://localhost:5226/
Does anyone of you have any idea what I might want to change to get this running? Thank you in advance.