GitLab CI Docker in Docker network inaccessible

Shared GitLab runner container fails to access the port exposed by the container running in it

Hello,

I’m running a docker in docker configuration using GitLab’s shared runners. Following is my gitlab-ci.yml:

image: docker:stable
services:
  - name: docker:dind
variables:
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""

stages:
 - api-test

api-job:
  stage: api-test
  before_script:
    - docker run -d --name api-server -p 5099:8000 <image>

  script:
    # Making API calls here

The docker container starts up just fine in the base container and can be seen using docker ps. It shows the following:

CONTAINER ID        IMAGE       COMMAND                  CREATED             STATUS              PORTS                                                     NAMES
<ID>                <Image>     "/docker-entrypoint.…"   1 seconds ago       Up 1 second         80/tcp, 8001/tcp, 8443-8444/tcp, 0.0.0.0:5099->8000/tcp   api-server

which tells me the port 5099 is accessible. However, when I make API calls to port 5099 to communicate with the web application running inside the (inner) container, it returns an error that the endpoint is not accessible. Moreover, I logged netstat -tulpn which shows no entries.

When I run the same setup on my local ubuntu machine, everything works fine.
I use the same docker image and start it up via:

docker run --privileged --name some-docker -d docker:dind

and exec into it and then run:

docker run -d --name api-server -p 5099:8000 <image>

and make the API calls I want to make and it works. I’m thoroughly confused here. I’m new to DevOps and GitLab so any help would be appreciated.

I know you have posted it long ago but did you resolve this issue if so how can you please explain i’m also struggling with same issue

This YAML defines three containers:

  • main (started automatically from image docker:stable)
  • service (started automatically from image docker:dind)
  • api-server (started in before_script from )

The command docker run starts that api-server container on DOCKER_HOST which specified hostname “docker” which is derived from the name of the service (“docker:dind” without alias). You can find more details about naming here in docs. So the port is published on that service container, not on the main container. That’s why you should make requests to “docker” hostname, not “localhost”. E.g. “http://docker:5099” assuming it’s plain HTTP.

Moreover, I logged netstat -tulpn which shows no entries.

Probably, because the port is published on the service container. Or because they use packet forwarding (e.g. iptables) instead of docker proxy. Doesn’t matter actually.