Hi,

I have a Gitlab CI job (SaaS) using docker to build docker image. When I run it I got this error:

ERROR: error during connect: Get http://docker:2375/v1.40/info: dial tcp: lookup docker on 169.254.169.254:53: no such host

Before gitlab-runner 16.1.0~beta.59.g83c66823 it worked just fine. The last successful run was on:

Running with gitlab-runner 15.9.0~beta.115.g598a7c91 (598a7c91)
on blue-2.shared.runners-manager.gitlab.com/default XxUrkriX, system ID: s_90897a2659b5
feature flags: FF_USE_IMPROVED_URL_MASKING:true

My related gitlab-ci.yaml:

#image: maven:3-openjdk-11
#image: maven:3.8-eclipse-temurin-17-alpine
image: maven:3.9-eclipse-temurin-17-alpine

variables:
  DOCKER_DRIVER: overlay
  ...

services:
  - docker:dind

before_script:
  - docker info

cache:
  paths:
    - .m2/repository

stages:
  - tag
  - sonar
  - build
  - build-docker

semantic_release:
  stage: tag
  image: node:alpine
  variables:
    GITLAB_TOKEN: $ACCESS_TOKEN
  before_script:
    - apk add --no-cache maven git
    - npm install -g semantic-release @semantic-release/gitlab-config @semantic-release/exec @semantic-release/git
  script:
    - semantic-release -e @semantic-release/gitlab-config
  except:
    - tags
  only:
    - master
  tags:
    - docker

build:
  stage: build
  before_script:
    - apk add --no-cache curl jq python3 py3-pip
    - pip install awscli
    - aws --version
    - echo $AWS_A_KEY_ID
    - aws configure set aws_access_key_id $AWS_A_KEY_ID
    - aws configure set aws_secret_access_key $AWS_SECRET_A_KEY
    - aws configure set region $AWS_DEF_REGION
    - export CODEARTIFACT_AUTH_TOKEN=$(aws codeartifact get-authorization-token --domain $CODEARTIFACT_DOMAIN --domain-owner $CODEARTIFACT_DOMAIN_OWNER --query authorizationToken --output text)
    - echo $CODEARTIFACT_AUTH_TOKEN
  script:
    - mvn $MAVEN_CLI_OPTS clean package
  artifacts:
    paths:
      - target/*.jar
  only:
    - tags
  tags:
    - docker

build:docker:
  #image: docker:latest
  image: docker:stable
  before_script:
    - docker info
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
  stage: build-docker
  script:
    - export JAR_PATH=$(ls -1 ./target/)
    - echo $JAR_PATH
    - docker pull ${CI_REGISTRY_IMAGE}:latest || true
    - docker build --cache-from ${CI_REGISTRY_IMAGE}:latest --build-arg JAR_FILE=./target/${JAR_PATH} --tag ${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG} --tag ${CI_REGISTRY_IMAGE}:latest .
    - docker push ${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}
    - docker push ${CI_REGISTRY_IMAGE}:latest
  only:
    - tags
  tags:
    - docker
=========== eof yml ============================

The whole log of failed run:

I had to change https.://gitlab.com/ to [gitlab job] and any xxxx.com to xxxxcom because as a new member I can not post more than 10 links. I’m sorry…

========== log ============================
Running with gitlab-runner 16.1.0~beta.59.g83c66823 (83c66823)

[2]([gitlab job]#L2) on blue-4.saas-linux-small-amd64.runners-manager.gitlab/default J2nyww-s, system ID: s_cf1798852952

[3]([gitlab job]#L3) feature flags: FF_USE_IMPROVED_URL_MASKING:true

[4]([gitlab job]#L4)Preparing the "docker+machine" executor00:32

[5]([gitlab job]#L5)Using Docker executor with image docker:stable ...

[6]([gitlab job]#L6)Starting service docker:dind ...

[7]([gitlab job]#L7)Pulling docker image docker:dind 

...

[13]([gitlab job]#L13)Preparing environment00:02

[14]([gitlab job]#L14)Running on runner-j2nyww-s-project-23767867-concurrent-0 via runner-j2nyww-s-s-l-s-amd64-1691481347-0f8d8a44...

...

[34]([gitlab job]#L34)Using docker image sha256:b0757c55a1fdbb59c378fd34dde3e12bd25f68094dd69546cf5ca00ddbaa7a33 for docker:stable with digest docker@sha256:fd4d028713fd05a1fb896412805daed82c4a0cc84331d8dad00cb596d7ce3e3a ...

[35]([gitlab job]#L35)$ docker info

[36]([gitlab job]/#L36)Client:

[37]([gitlab job]#L37) Debug Mode: false

[38]([gitlab job]#L38)Server:

[39][gitlab job]/#L39)ERROR: error during connect: Get http://docker:2375/v1.40/info: dial tcp: lookup docker on 169.254.169.254:53: no such host

[40]([gitlab job]#L40)errors pretty printing info

[42]([gitlab job]#L42)Cleaning up project directory and file based variables00:01

[44]([gitlab job]#L44)ERROR: Job failed: exit code 1
========== eof log =========================

Do you have any hints/advice how can I solve this issue? I stucked
Thank you!

I haven’t used it in a while. but it should be DOCKER_DRIVER: overlay2. Also variable DOCKER_TLS_CERTDIR: "" used to be required (don’t know if that’s still a thing).

You can also try to be more specific with the service definition

services:
  - name: 'docker:20.10.12-dind'
    command: ['--tls=false', '--host=tcp://0.0.0.0:2375']

I suggest to pick a Docker version and use that instead of the dind tag which is always referencing the latest version and can break things.

I suggest to look for other tool which is designed to build container images in containers and CI like Use kaniko to build Docker images | GitLab

Thank you for your answer. I picked the specific docker and dind version way at first. Kaniko will be the second pick
So, I changed service definition as you wrote and I started to “downgrade” docker version from the latest. I found that the latest version (24.0.5) produces the no-such-host error, but one step down the 23.0.6 doesn’t. (20.10.12 also does not produce the error). But new error came into the picture:

$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY

97WARNING! Using --password via the CLI is insecure. Use --password-stdin.

98Error response from daemon: Get “https://registry-1.docker.io/v2/”: unauthorized: incorrect username or password

I should dig deeper, but If you have any hint, you are most welcome as always…

The CI_BUILD_* variables have been removed. You need to use their new counterparts as documented here

Eh, everything is fine, now. I should have been more careful. Thanks a lot!