CI with docker - am I doing this right?

How to Use GitLab
1

Hi,

I’ve been grappling for a few days with the documentation regarding using a containerised gitlab-runner and running builds through docker containers. I’ve got something running, but I’m feeling that I’m not quite getting the key concepts. I’d appreciate any guidance.

My setup is - perhaps unusually? - that I’m running GitLab itself in a docker container, with a separate gitlab-runner container on the same box, added as per the docs here. That runner was then registered as a “docker” executor.

My gitlab-ci.yml file now comprises a bunch of jobs that themselves each spin up a container of their own in order to run specific build tasks, for instance: a nodejs container to build frontend dependencies, a container with Amazon AWS CLI installed to fetch webserver details, another container to rsync the build up to the webservers. For example:

fetch ec2 instances:
  stage: pre-install
  image: garland/aws-cli-docker
  variables:
    AWS_INSTANCE_TAG_NAMES: "MyApp"
  script:
    - aws ec2 describe-instances --filters "Name=tag:Name,Values=${AWS_INSTANCE_TAG_NAMES}" --query "Reservations[*].Instances[*].[PublicIpAddress]" --output=text > ec2s.txt
  artifacts:
    paths:
      - ec2s.txt
    expire_in: 3 mos

copy files:
  stage: install
  image: instrumentisto/rsync-ssh
  variables:
    target_dir: $SITE_ROOT_PRODUCTION
  script:
    - echo "${SSH_KEY}" > ~/ssh_key.pem
    - chmod 600 ~/ssh_key.pem
    - sh $CI_PROJECT_DIR/scripts/rsync_to_ec2s.sh

A couple of things about this bother me. One is that I seem to have a container running gitlab-runner, whose sole purpose seems to be spawning… sub-containers(?) rather than actually doing any build itself. Another is that I’m aware of “services” being an option in docker integration with GitLab CI, and it certainly seems that what I want to do is spin up a bunch of supporting service containers, but I just don’t get how you’re supposed to use them, or even if this is the right use case for them. I’m also aware of a “docker in docker”… runner? executor? and again, it’s what I seem to be doing, albeit in a roundabout way.

So; am I doing this all wrong? How should i be using a dockerised gitlab-runner to execute the various build and deploy jobs for my pipeline?

Thanks :slight_smile: