I would like to be able to use the gcloud command and also docker in my GitLab CI jobs. I thought that what suited me was using a Cloud SDK Docker image from here and using docker as a service. This is the .gitlab-ci.yml that I’m trying to use:
image: gcr.io/google.com/cloudsdktool/cloud-sdk:346.0.0-alpine
# This file is a template, and might need editing before it works on your project.
# This is a sample GitLab CI/CD configuration file that should run without any modifications.
# It demonstrates a basic 3 stage CI/CD pipeline. Instead of real tests or scripts,
# it uses echo commands to simulate the pipeline execution.
#
# A pipeline is composed of independent jobs that run scripts, grouped into stages.
# Stages run in sequential order, but jobs within stages run in parallel.
#
# For more information, see: https://docs.gitlab.com/ee/ci/yaml/README.html#stages
stages: # List of stages for jobs, and their order of execution
- build
variables:
# When you use the dind service, you must instruct Docker to talk with
# the daemon started inside of the service. The daemon is available
# with a network connection instead of the default
# /var/run/docker.sock socket. Docker 19.03 does this automatically
# by setting the DOCKER_HOST in
# https://github.com/docker-library/docker/blob/d45051476babc297257df490d22cbd806f1b11e4/19.03/docker-entrypoint.sh#L23-L29
#
# The 'docker' hostname is the alias of the service container as described at
# https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services.
#
# Specify to Docker where to create the certificates. Docker
# creates them automatically on boot, and creates
# `/certs/client` to share between the service and job
# container, thanks to volume mount from config.toml
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_HOST: "docker"
services:
- docker:19.03.12-dind
build-job: # This job runs in the build stage, which runs first.
stage: build
script:
- getent hosts
- docker info
However, this is the log from the job:
Running with gitlab-runner 13.12.0-rc1 (b21d5c5b)
on docker-auto-scale fa6cab46
feature flags: FF_GITLAB_REGISTRY_HELPER_IMAGE:true, FF_SKIP_DOCKER_MACHINE_PROVISION_ON_CREATION_FAILURE:true
section_start:1624784010:prepare_executor
Preparing the "docker+machine" executor
Using Docker executor with image gcr.io/google.com/cloudsdktool/cloud-sdk:346.0.0-alpine ...
Starting service docker:19.03.12-dind ...
Pulling docker image docker:19.03.12-dind ...
Using docker image sha256:66dc2d45749a48592f4348fb3d567bdd65c9dbd5402a413b6d169619e32f6bd2 for docker:19.03.12-dind with digest docker@sha256:674f1f40ff7c8ac14f5d8b6b28d8fb1f182647ff75304d018003f1e21a0d8771 ...
Waiting for services to be up and running...
Pulling docker image gcr.io/google.com/cloudsdktool/cloud-sdk:346.0.0-alpine ...
Using docker image sha256:bcb49725a0ab09cb1947f2438c10ee662cfb4b5b9e759e962700777f3bc01877 for gcr.io/google.com/cloudsdktool/cloud-sdk:346.0.0-alpine with digest gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:b299459e2767e431b6dfab9fb852be2ab077791c083be0533b6210c0242183ce ...
section_end:1624784042:prepare_executor
section_start:1624784042:prepare_script
Preparing environment
Running on runner-fa6cab46-project-20298117-concurrent-0 via runner-fa6cab46-srm-1624783966-59f53c1f...
section_end:1624784044:prepare_script
section_start:1624784044:get_sources
Getting source from Git repository
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/andrea-berling/oremassimo/.git/
Created fresh repository.
Checking out 95813b12 as master...
Skipping Git submodules setup
section_end:1624784045:get_sources
section_start:1624784045:step_script
Executing "step_script" stage of the job script
Using docker image sha256:bcb49725a0ab09cb1947f2438c10ee662cfb4b5b9e759e962700777f3bc01877 for gcr.io/google.com/cloudsdktool/cloud-sdk:346.0.0-alpine with digest gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:b299459e2767e431b6dfab9fb852be2ab077791c083be0533b6210c0242183ce ...
$ getent hosts
$ docker info
Client:
Debug Mode: false
Server:
ERROR: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?
errors pretty printing info
section_end:1624784046:step_script
section_start:1624784046:cleanup_file_variables
Cleaning up file based variables
section_end:1624784046:cleanup_file_variables
ERROR: Job failed: exit code 1
As you can see, getent hosts returns nothing, and the job can’t use docker because it can’t connect to the service. However, as soon as I comment out the image line:
#image: gcr.io/google.com/cloudsdktool/cloud-sdk:346.0.0-alpine
#image: docker:19.03.12
# This file is a template, and might need editing before it works on your project.
# This is a sample GitLab CI/CD configuration file that should run without any modifications.
# It demonstrates a basic 3 stage CI/CD pipeline. Instead of real tests or scripts,
# it uses echo commands to simulate the pipeline execution.
#
# A pipeline is composed of independent jobs that run scripts, grouped into stages.
# Stages run in sequential order, but jobs within stages run in parallel.
#
# For more information, see: https://docs.gitlab.com/ee/ci/yaml/README.html#stages
stages: # List of stages for jobs, and their order of execution
- build
variables:
# When you use the dind service, you must instruct Docker to talk with
# the daemon started inside of the service. The daemon is available
# with a network connection instead of the default
# /var/run/docker.sock socket. Docker 19.03 does this automatically
# by setting the DOCKER_HOST in
# https://github.com/docker-library/docker/blob/d45051476babc297257df490d22cbd806f1b11e4/19.03/docker-entrypoint.sh#L23-L29
#
# The 'docker' hostname is the alias of the service container as described at
# https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services.
#
# Specify to Docker where to create the certificates. Docker
# creates them automatically on boot, and creates
# `/certs/client` to share between the service and job
# container, thanks to volume mount from config.toml
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_HOST: "docker"
services:
- docker:19.03.12-dind
build-job: # This job runs in the build stage, which runs first.
stage: build
script:
#- gcloud version
- getent hosts
, this is the output I get:
Running with gitlab-runner 13.12.0-rc1 (b21d5c5b)
on docker-auto-scale 72989761
feature flags: FF_GITLAB_REGISTRY_HELPER_IMAGE:true, FF_SKIP_DOCKER_MACHINE_PROVISION_ON_CREATION_FAILURE:true
section_start:1624784797:prepare_executor
Preparing the "docker+machine" executor
Using Docker executor with image ruby:2.5 ...
Starting service docker:19.03.12-dind ...
Pulling docker image docker:19.03.12-dind ...
Using docker image sha256:66dc2d45749a48592f4348fb3d567bdd65c9dbd5402a413b6d169619e32f6bd2 for docker:19.03.12-dind with digest docker@sha256:674f1f40ff7c8ac14f5d8b6b28d8fb1f182647ff75304d018003f1e21a0d8771 ...
Waiting for services to be up and running...
Pulling docker image ruby:2.5 ...
Using docker image sha256:27d049ce98db4e55ddfaec6cd98c7c9cfd195bc7e994493776959db33522383b for ruby:2.5 with digest ruby@sha256:ecc3e4f5da13d881a415c9692bb52d2b85b090f38f4ad99ae94f932b3598444b ...
section_end:1624784843:prepare_executor
section_start:1624784843:prepare_script
Preparing environment
Running on runner-72989761-project-20298117-concurrent-0 via runner-72989761-srm-1624784753-931e87db...
section_end:1624784846:prepare_script
section_start:1624784846:get_sources
Getting source from Git repository
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/andrea-berling/oremassimo/.git/
Created fresh repository.
Checking out b90351a2 as master...
Skipping Git submodules setup
section_end:1624784847:get_sources
section_start:1624784847:step_script
Executing "step_script" stage of the job script
Using docker image sha256:27d049ce98db4e55ddfaec6cd98c7c9cfd195bc7e994493776959db33522383b for ruby:2.5 with digest ruby@sha256:ecc3e4f5da13d881a415c9692bb52d2b85b090f38f4ad99ae94f932b3598444b ...
$ getent hosts
127.0.0.1 localhost
127.0.0.1 localhost ip6-localhost ip6-loopback
127.0.0.1 0hshit.hopto.org
127.0.0.1 daymndaymn.myftp.org
127.0.0.1 loba.webhop.me
172.17.0.3 docker 3dd539c9395b runner-72989761-project-20298117-concurrent-0-7a95830f0dde86c5-docker-0
172.17.0.4 runner-72989761-project-20298117-concurrent-0
section_end:1624784848:step_script
section_start:1624784848:cleanup_file_variables
Cleaning up file based variables
section_end:1624784848:cleanup_file_variables
Job succeeded
As you can see this time getent hosts returns a list of hosts, among which there is docker from the docker service I included in my .gitlab-ci.yml file. How can I both use an image for my jobs and also have access to the docker service? Or am I doing something wrong?