CI/CD Inter-Service Networking

On the CI/CD documentation on using service, it is clear that the service can be accessed using the alias as the hostname.

However there is no documentation (or I cannot find) on inter-service networking. On the example below, the test code is able to access the database via dbhost and middleware via middlewarehost.

However the middleware cannot connect to the database. I’ve tried using the alias which is dbhost and localhost. Any body know how to connect services together?

I’m getting cannot resolve host when using the alias dbhost from the middleware service.

run-test:
  image: maven:3.3.9-jdk-8
  stage: test
  only:
  - master
  services:
  - name: database
    alias: dbhost
  - name: middleware
    alias: middlewarehost
  variables:
    DBPROFILE: integration-test
    MAX_ATTEMPTS: 30
    SPRING_PROFILES_ACTIVE: docker
  script:
    - mvn test -Dtest=**/**IT
  tags:
  - docker

I was able to replicate the problem in my local setup.

using command : gitlab-runner exec docker run-test.

I connected to the middleware container and I tried connecting using dbhost, container name, localhost, DBHOST (alias in caps), but nothing worked. I then tried the ip-address and it connected fine.

As a quick fix, I just workaround the problem by using the database container’s ip-address. But the test is using the alias.

Lastly, everything tells me that I am doing the right things, alias is working, containers are in the same network (since they can talk to each other via IP), but I guess the hostname resolution is the problem between services. Any ideas?

Hi @gitlab.padawan ,
I encountered the same problem and it surprises me that this issue hasn’t been fixed yet. There is another topic that indicates that people are struggling with this for almost 2 years now. imho service interconnection is an important “feature” one needs for testing.

Just like you I worked around the issue by using ip addresses. It turns out the runner assigns ip addresses starting from 172.17.0.2 incrementally. In your case dbhost would be reachable via 172.17.0.2 and middleware via 172.17.0.3

I don’t know how stable this approach is, especially if the runner is busy and there are many jobs running simultaneously.

Here is a minimal gitlab-ci config to reproduce this behavior:

stages:
  - test

mysql-test:
  stage: test
  services:
    - name: debian
      alias: debian
      command:
        - "sleep"
        - "60"
    - name: debian
      alias: deb-client-1
      command:
        - "ping"
        - "debian"
    - name: debian
      alias: deb-client-2
      command:
        - "ping"
        - "172.17.0.2"
  image: debian
  script:
    - ping debian -c 20
    - sleep 60

The first client fails

Service container logs:
2019-09-26T14:01:47.086531547Z ping: debian: Name or service not known

The container using the address to reach the target succeeds

Service container logs:
2019-09-26T14:01:48.010415693Z PING 172.17.0.2 (172.17.0.2) 56(84) bytes of data.
2019-09-26T14:01:48.010497873Z 64 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.136 ms

and the main environment can resolve the address:

[0K[32;1m$ ping debian -c 20[0;m
PING debian (172.17.0.2) 56(84) bytes of data.
64 bytes from debian (172.17.0.2): icmp_seq=1 ttl=64 time=0.157 ms

There is a related open issue on gitlab.com with many workarounds

There is now a feature flag to enable inter-service communication. Simply add the FF_NETWORK_PER_BUILD variable with a non null value to the job’s variables and they should be able to communicate now.

The reference issue. initially linked by _mark.