Services stop when running multiple pipelines at the same time

Hi,

One of my projects uses GitLab services for CI with the Docker runner. Multiple pipelines regularly run at the same time.

I have noticed that when running multiple pipelines simultaneously, all pipelines fail after one has exited. The application-land error is:

elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7fd86fa41d50>: Failed to establish a new connection: [Errno -2] Name or service not known) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7fd86fa41d50>: Failed to establish a new connection: [Errno -2] Name or service not known)

I indeed do not see the Elasticsearch container in docker ps.

A theory could be that when a pipeline exits, all containers for that service are stopped, even if those containers belong to different pipelines. However, I can reproduce the issue when using different service names as well.

I can confirm that aliases resolve to different service containers inside each container in which the script runs.

Before I investigate any further: is there something obvious that I’m doing wrong?

Configs

/etc/gitlab-runner/config.toml on the runner:

concurrent = 4
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "runner0.cyberfusion.cloud"
  url = "https://vcs.cyberfusion.nl/"
  token = "<stripped>"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.docker]
    image = "docker:stable"
    memory = "12G"
    cpus = "12"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    network_mode = "bridge"
    pull_policy = "if-not-present"
    shm_size = 0

Relevant snippet from .gitlab-ci.yml:

unit test:
  stage: test
  image: python:3.7
  services:
    - name: mariadb:10.3
      alias: mariadb
    - name: elasticsearch:7.16.3
      alias: elasticsearch
      command: [ "bin/elasticsearch", "-Ediscovery.type=single-node", "-Enetwork.host=0.0.0.0" ]
    - name: redis:5.0.14
      alias: redis
      command: [ "redis-server", "--requirepass", "<stripped>" ]
    # https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3163#note_63471739
    - name: rabbitmq:3.9.13-management
      alias: rabbitmq