Advice on setting up a pipeline for Helm packaging and Terraforming?

GitLab CI/CD
, ,
1

I have a project that contains some deployment artifacts, namely a Helm chart I’d like to package, and Terraform I’d like to apply. I’m trying to write a pipeline that allows me to only do each task only when the relevant files change (ie. package helm if ./helm/* changes, apply Terraform if ./terraform/* changes).

I can’t seem to get this to behave how I want. Using a combination of needs and rules my helm-build job doesn’t get created even if a file in ./helm/* changes. Is this because it is in a later stage, and because no jobs rendered in validate or test? When there is a terraform and helm change, all my jobs run as expected.

Is it possible to do this sort of flow control, or should I be in separate repositories?

include:
  - template: Terraform/Base.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml

stages:
  - validate
  - test
  - build
  - deploy
  - cleanup

variables:
  TF_ROOT: ${CI_PROJECT_DIR}/terraform

################
# validate     #
################
terraform-fmt:
  extends: .terraform:fmt
  needs: []
  rules:
    - changes:
      - terraform/*

terraform-validate:
  extends: .terraform:validate
  needs: []
  rules:
    - changes:
      - terraform/*

################
# test         #
################

################
# build        #
################

terraform-build:
  extends: .terraform:build
  needs: ["terraform-fmt", "terraform-validate"]
  environment:
    name: $TF_STATE_NAME
    action: prepare
  rules:
    - changes:
      - terraform/*

helm-build:
  stage: build
  needs: []
  image:
    name: alpine/helm:3.5.3
    entrypoint: [""]
  variables:
    CHART: my-chart-name
  script:
    - helm dependency update helm/${CHART}
    - helm package helm/${CHART}
  artifacts:
    expire_in: 1 week
    paths:
      - ${CHART}*.tgz
  rules:
    - changes:
      - helm/*

################
# deploy       #
################
terraform-deploy:
  extends: .terraform:deploy
  needs: ["terraform-build"]
  environment:
    name: $TF_STATE_NAME
    action: start
  rules:
    - when: manual
      changes:
      - terraform/*

helm-deploy:
  stage: deploy
  needs: ["helm-build"]
  image:
    name: alpine/helm:3.5.3
    entrypoint: [""]
  dependencies:
    - helm-build
  variables:
    CHART: my-chart-name
  before_script:
    - apk add git
    - helm plugin install --version=v0.9.0 https://github.com/chartmuseum/helm-push.git
    - helm repo add my-helm-repo --username ${CI_REGISTRY_USER} --password ${CI_REGISTRY_PASSWORD} ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/stable
  script:
    - helm push ${CHART}*.tgz my-helm-repo
  rules:
    - when: manual
      changes:
      - helm/*

################
# cleanup      #
################
2

Turns out I just didn’t know about child pipelines, which really cleans this up! Breaking down CI/CD complexity with parent-child and multi-project pipelines | GitLab

3

You need to use proper glob pattern in order to cover subdirectories
- helm/* - only covers files directly in helm/ like helm/Chart.yaml, but does not cover files in subdirs like helm/templates/deployment.yaml
- helm/**/* - covers everything in helm/ directory no matter how deep it is

1 Like