Refine .gitlab-ci.yml / best practices

GitLab CI/CD
, , , ,
1

Hello, first post here so my apologies in advance if I am not following the etiquette! I aim to please.

I am staring at a .gitlab-ci.yml file that I put together based on some guides and my own prior experience and trying to wrap my head around how to make it better and more modular. As an overview, this is a Java Maven project with spring boot. there are three stages:

  1. Run tests (this step works great)
  2. Run build (this step works great)
  3. Package into a container (here is where I am having issues)

Before progressing further:

Gitlab version 15.0.0-ee self-hosted

Gitlab runner version 15.0.0

I am gearing up to enable the “review app” function and ideally embrace the AutoDevOps workflow (potentially streamlining this setup so I could even remove my gitlab-ci file seems like a lofty goal, but I never quite understood that system and would love to reuse as much of its methodology as I can). So because of that I have these issues with my current .gitlab-ci.yml

1 - Issues with docker-in-docker

I had to settle on running the package step on a “shell” runner (same host as the usual docker gitlab runner)

I am not quite clear here, which is the recommendation, what is the best guide. I churned through so many combinations of settings for configuring gitlab runner and distros. Started with RHEL, but ran into cgroupsv2 issues and weirdness due to podman vs docker. Then

If you look at the gitlab-ci file it shows a tag on the final stage, so this is all running on the shell of that gitlab runner which I do not like the idea of that. I am worried about race conditions and cache issues and tagging the container as the same thing in two overlapping jobs. Is my concern justified? Should I spend another day and a half trying to get docker-in-docker to work or use the “bind docker socket” hack? What are the shared gitlab runners on gitlab.com using as a best practice to provide docker in docker?

2 - Code duplication

I am not clear here on how to fold my two “package” stages down into one stage so there is less code duplication. I am having trouble understanding the documentation and the distinctions between rules: and environment: nodes in the .yml file and which would be the right path.

3 container sprawl / image lifecycle

In the attached .gitlab-ci.yml you can see that I am building the image and tagging it with the $CI_COMMIT_SHA, then re-tagging as :latest to push to my registry. I am seeing that this leaves a ton of images on my gitlab runner host. I really only need the gitlab package registry to contain the “latest” image of the most recent master branch build, and an image corresponding to any git tags. What is the best way to make sure these numerous images with the $CI_COMMIT_SHA go away? Just delete them in the gitlab pipeline? That will not also delete the :latest though right?

#image: docker:latest
#services:
#  - docker:dind

variables:
  DOCKER_DRIVER: overlay
  #SPRING_PROFILES_ACTIVE: gitlab-ci

stages:
  - test
  - build
  - package

maven-test:
  image: maven:3-openjdk-17
  stage: test
  script: "mvn test"

maven-build:
  image: maven:3-openjdk-17
  stage: build
  only:
    - master
    - tags
  script: "mvn package -B"
  artifacts:
    paths:
      - target/*.jar



# Here, the goal is to tag the "master" branch as "latest"
Push latest:
  variables:
    # We are just playing with Docker here. 
    # We do not need GitLab to clone the source code.
    GIT_STRATEGY: none
  stage: package
  tags:
    - shell
  only:
    # Only "master" should be tagged "latest"
    - master
  script:
    # login to docker
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN [gitlab-registry-host:port]
    # fetches the latest image (not failing if image is not found)
    - docker pull $CI_REGISTRY_IMAGE/container_name:latest || true
    # builds the project, passing proxy variables, using OCI labels
    # notice the cache-from, which is going to use the image we just pulled locally
    # the built image is tagged locally with the commit SHA, and then pushed to 
    # the GitLab registry
    - >
      docker build
      --pull
      --build-arg http_proxy=$http_proxy
      --build-arg https_proxy=$https_proxy
      --build-arg no_proxy=$no_proxy
      --cache-from $CI_REGISTRY_IMAGE/container_name:latest
      --label "org.opencontainers.image.title=$CI_PROJECT_TITLE"
      --label "org.opencontainers.image.url=$CI_PROJECT_URL"
      --label "org.opencontainers.image.created=$CI_JOB_STARTED_AT"
      --label "org.opencontainers.image.revision=$CI_COMMIT_SHA"
      --label "org.opencontainers.image.version=$CI_COMMIT_REF_NAME"
      --tag $CI_REGISTRY_IMAGE/container_name:$CI_COMMIT_SHA
      .

    # Then we tag it "latest"
    - docker tag $CI_REGISTRY_IMAGE/container_name:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE/container_name:latest
    # Annnd we push it.
    - docker push $CI_REGISTRY_IMAGE/container_name:latest

# Finally, the goal here is to Docker tag any Git tag
# GitLab will start a new pipeline everytime a Git tag is created, which is pretty awesome
Push tag:
  variables:
    # Again, we do not need the source code here. Just playing with Docker.
    GIT_STRATEGY: none
  stage: package
  tags:
    - shell
  only:
    # We want this job to be run on tags only.
    - tags
  script:
    # login to docker
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN [gitlab-server]
    # fetches the latest image (not failing if image is not found)
    - docker pull $CI_REGISTRY_IMAGE/container_name:latest || true
    # builds the project, passing proxy variables, using OCI labels
    # notice the cache-from, which is going to use the image we just pulled locally
    # the built image is tagged locally with the commit SHA, and then pushed to 
    # the GitLab registry
    - >
      docker build
      --pull
      --build-arg http_proxy=$http_proxy
      --build-arg https_proxy=$https_proxy
      --build-arg no_proxy=$no_proxy
      --cache-from $CI_REGISTRY_IMAGE/container_name:latest
      --label "org.opencontainers.image.title=$CI_PROJECT_TITLE"
      --label "org.opencontainers.image.url=$CI_PROJECT_URL"
      --label "org.opencontainers.image.created=$CI_JOB_STARTED_AT"
      --label "org.opencontainers.image.revision=$CI_COMMIT_SHA"
      --label "org.opencontainers.image.version=$CI_COMMIT_REF_NAME"
      --tag $CI_REGISTRY_IMAGE/container_name:$CI_COMMIT_SHA
      .

    # Then we tag it
    - docker tag $CI_REGISTRY_IMAGE/container_name:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE/container_name:$CI_COMMIT_REF_NAME
    # Annnd we push it.
    - docker push $CI_REGISTRY_IMAGE/container_name:$CI_COMMIT_REF_NAME

I guess I am just looking for some best practices to build up my knowledge. Since I will be venturing to make the review app function work I really need to have a better foundational understanding of the existing pipeline and my infrastructure. Any advise is very appreciated!

Thanks!

2

1 - Issues with docker-in-docker

  1. What kind cache do you want to use?
  2. Are you worried about race conditions on building the images?

gitlab.com uses Docker Machine executor of GitLab Runner: Install and register GitLab Runner for autoscaling with Docker Machine | GitLab

extends can be used: Optimize GitLab CI/CD configuration files | GitLab .

3 container sprawl / image lifecycle

In the attached .gitlab-ci.yml you can see that I am building the image and tagging it with the $CI_COMMIT_SHA, then re-tagging as :latest to push to my registry. I am seeing that this leaves a ton of images on my gitlab runner host. I really only need the gitlab package registry to contain the “latest” image of the most recent master branch build, and an image corresponding to any git tags. What is the best way to make sure these numerous images with the $CI_COMMIT_SHA go away? Just delete them in the gitlab pipeline? That will not also delete the :latest though right?

I do not think so much images as you are only building with git tag by only:. What container registry are you using?

If GitLab Container Registry, you can use built-in clean up: Reduce Container Registry storage | GitLab

From a viewpoint of practitioner of Auto DevOps just 4 years, I personally cannot recommend that easily. But you can try it out! Enjoy and works well in your applications!!! :rocket: :metal:

1 Like