Docker-machine docker error

GitLab CI/CD
, , ,
6

Hi,

Thanks for the context. The initial question seemed it was referring to an earlier problem you have encountered. Its original state, changes, and potential fixes can help analyze the current problem faster. I’m trying to understand the case for the reverse proxy, and its network flow and help identify the cause for the http-to-https error. I haven’t used this scenario myself - as many details as possible can help me understand while I research possible ways to help mitigate the problem :slight_smile:

While looking for docker-machine gitlab proxy settings I found the earlier forum topic. Docker-machine proxy settings

I set up docker machine using the boot2docker iso, it all runs and deploys a VM but thats where the joy ends. Lo and behold there is no proxy settings so docker can’t pull images.

I’m not sure if boot2docker with its own docker-machine version will work. The documentation refers to installing the GitLab version: Install and register GitLab Runner for autoscaling with Docker Machine | GitLab

squid proxy test

I haven’t used squid as proxy for Docker yet, so I was curious to try the setup. The ubuntu/squid image worked without configuration modifications - which could help your analysis with your squid configuration.

Tests are performed on a Ubuntu 20 LTS VM.

docker run -d --name squid-container -e TZ=UTC -p 3128:3128 ubuntu/squid:5.2-22.04_beta

sudo mkdir -p /etc/systemd/system/docker.service.d
vim /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTPS_PROXY=http://localhost:3128"
Environment="HTTP_PROXY=http://localhost:3128"

 sudo systemctl daemon-reload
 sudo systemctl restart docker

docker rm squid-container
docker run -d --name squid-container -e TZ=UTC -p 3128:3128 ubuntu/squid:5.2-22.04_beta

docker logs -f squid-container

root@legendiary:~# docker pull alpine:latest
latest: Pulling from library/alpine
2408cc74d12b: Pull complete
Digest: sha256:686d8c9dfa6f3ccfc8230bc3178d23f84eeaf7e457f36f271ab1acc53015037c
Status: Downloaded newer image for alpine:latest
docker.io/library/alpine:latest

1655891221.435    499 172.17.0.1 TCP_TUNNEL/200 6087 CONNECT registry-1.docker.io:443 - HIER_DIRECT/3.228.155.36 -
1655891221.872    425 172.17.0.1 TCP_TUNNEL/200 10284 CONNECT auth.docker.io:443 - HIER_DIRECT/3.86.127.18 -
1655891222.347    473 172.17.0.1 TCP_TUNNEL/200 6219 CONNECT registry-1.docker.io:443 - HIER_DIRECT/3.228.155.36 -
1655891235.285    384 172.17.0.1 TCP_TUNNEL/200 6087 CONNECT registry-1.docker.io:443 - HIER_DIRECT/3.228.155.36 -
1655891235.724    428 172.17.0.1 TCP_TUNNEL/200 10284 CONNECT auth.docker.io:443 - HIER_DIRECT/3.86.127.18 -
1655891236.172    445 172.17.0.1 TCP_TUNNEL/200 6219 CONNECT registry-1.docker.io:443 - HIER_DIRECT/3.228.155.36 -
1655891236.631    450 172.17.0.1 TCP_TUNNEL/200 7857 CONNECT registry-1.docker.io:443 - HIER_DIRECT/3.228.155.36 -
1655891237.091    428 172.17.0.1 TCP_TUNNEL/200 6741 CONNECT registry-1.docker.io:443 - HIER_DIRECT/3.228.155.36 -
1655891237.494    388 172.17.0.1 TCP_TUNNEL/200 6139 CONNECT registry-1.docker.io:443 - HIER_DIRECT/3.228.155.36 -
1655891237.510    404 172.17.0.1 TCP_TUNNEL/200 6141 CONNECT registry-1.docker.io:443 - HIER_DIRECT/3.228.155.36 -
1655891237.543     47 172.17.0.1 TCP_TUNNEL/200 5256 CONNECT production.cloudflare.docker.com:443 - HIER_DIRECT/104.18.122.25 -
1655891237.587     72 172.17.0.1 TCP_TUNNEL/200 2809548 CONNECT production.cloudflare.docker.com:443 - HIER_DIRECT/104.18.122.25 -

Docker CLI and squid proxy worked in my tests, I’ve also written a short blog post in

For your setup I’d suggest investigating the proxy logs if they provide more insights when the docker pull command is executed. Once the manual tests work with the CLI, the next step can be looking into docker-machine environment variables for the HTTP proxy settings, see below.

docker-machine with HTTP proxy

Found Set proxy servers on docker-machine (#75) · Issues · GitLab.org / Ops Sub-Department / docker-machine · GitLab

Digging further, I have found 2 things:

Combining this knowledge into

  1. Adding the proxy configuration in the current session
export HTTP_PROXY=http://yourproxy.net:8080
export HTTPS_PROXY=https://yourproxy.net:8080
  1. Re-creating the docker-machine machine
docker-machine rm default

docker-machine create -d virtualbox \
 --engine-env HTTP_PROXY="$HTTP_PROXY" \
 --engine-env HTTPS_PROXY="$HTTPS_PROXY" \
 default

Note: Replace -d virtualbox with the machine driver you are using.

From there, the VM should be using the HTTP proxy for pulling container images

I have not tested the approach but it might be helpful for others finding this topic.

docker-machine for auto-scaling

docker-machine is unfortunately deprecated upstream by Docker, GitLab maintains a fork, and looks for alternatives and better auto-scaling implementations. A blueprint RFC is available at Next Runner Auto-scaling Architecture | GitLab

I’d suggest following Strategy in response to deprecation of Docker Machine by Docker (#341856) · Issues · GitLab.org / GitLab · GitLab and Autoscaling Provider for GitLab Runner to replace Docker Machine (&2502) · Epics · GitLab.org · GitLab and add a comment describing your use case and proposal.

Cheers,
Michael

1 Like