Docker-machine executor can't build docker image

GitLab CI/CD
,
1

Hello all!
I’m a happy user of this Terraform module: GitHub - npalm/terraform-aws-gitlab-runner: Terraform module for AWS GitLab runners on ec2 (spot) instances ; and I’m now trying to switch most of my builds to those self-hosted runners running on AWS the “docker+machine” executor with this configuration:

concurrent = 10
check_interval = 3
sentry_dsn = ""
log_format = "json"

[[runners]]
  name = "docker-default"
  url = "https://gitlab.com"
  token = "XXX"
  executor = "docker+machine"
  environment = []
  pre_build_script = ""
  post_build_script = ""
  pre_clone_script = ""
  request_concurrency = 1
  output_limit = 4096
  limit = 0
  [runners.docker]
    tls_verify = false
    image = "docker:18.03.1-ce"
    privileged = true
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
    pull_policy = "always"
    runtime = ""
    helper_image = ""
  [runners.docker.tmpfs]

  [runners.docker.services_tmpfs]

  [runners.cache]
   (...)
  [runners.machine]
    IdleCount = 0
    IdleTime = 600

    MachineDriver = "amazonec2"
    MachineName = "runner-%s"
    MachineOptions = [
      (...)

    ]

But I can’t build docker images on those runners, say using this configuration that works fine with the shared runners:

stages:
  - build

docker-build:
  tags:
    - self-hosted
  image: docker:stable
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  stage: build
  script:
    - docker build -t $CI_REGISTRY_IMAGE --build-arg GITLAB_NPM_TOKEN=${CI_JOB_TOKEN} .
    - docker tag $CI_REGISTRY_IMAGE $CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:7}
    - docker push $CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:7}
    - docker push $CI_REGISTRY_IMAGE:latest
  rules:
    - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME'

I’m getting this error:

$ docker build -t $CI_REGISTRY_IMAGE --build-arg GITLAB_NPM_TOKEN=${CI_JOB_TOKEN} .
Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?
Cleaning up project directory and file based variables

I then tried removing the dind service or even the image: docker:stable configuration, but it eventually fails with similar errors…

What can I do to solve this? Is it possible to build a docker image, using docker on a docker+machine executor?

Thank you!

What version are you on? Are you using self-managed or GitLab.com?
Gitlab.com

2

Hi @anthonydahanne I think you are close but need to do a couple of things.

  1. Check out how gitlab configures their shared runners since yours should probably match this pretty closely.
  2. As part of that, consider Use the OverlayFS driver environment = ["DOCKER_DRIVER=overlay2"]
  3. I think you need to Use the Docker executor with Docker socket binding 
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
1 Like
3

hey there @byarbrough !
thanks for your answer, yes the item #3 you listed:

I think you need to Use the Docker executor with Docker socket binding

volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]

did the trick! I updated the config at /etc/gitlab-runner/config.toml and restarted the gitlab-runner service and that was it!

Thank you very much!

4

Glad that helped! Another alternative is to use Buildah, as described in this post: Build Dockerfiles in GitLab CI shared runners the easy way: ditch dind

---
variables:
  IMAGE_TAG: $CI_COMMIT_SHORT_SHA # or $CI_COMMIT_TAG if only building on tags

container-build:
  image: quay.io/buildah/stable:v1.23.3
  before_script:
    - buildah version
    - export HOME=$CI_BUILDS_DIR # needed sometimes to avoid conflict with host
    - buildah login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - buildah build -t $IMAGE_TAG
    - buildah push $IMAGE_TAG docker://$CI_REGISTRY_IMAGE:$IMAGE_TAG
  after_script:
    - buildah logout $CI_REGISTRY

With just these permissions in your runner

cap_add = ["SYS_ADMIN"]
devices = ["/dev/fuse"]
security_opt = ["apparmor:unconfined"] # for Debian host
5

interesting; I’d rather go with a “privileges-less” approach if I were to change the way the Docker images are built, using Kaniko or some programming language native toolkits, such as jib for Java.