Help regarding Parallel Matrix & On_changes rule

GitLab CI/CD
1

Describe your question in as much detail as possible:
Is it possible at all to use the parallel:matrix configuration to manipulate the changes:compare_to feature in the GitLab CI configuration? I have a parallel:matrix overriding and running a job multiple times, but I want to run those matrix items conditionally - do I need a significant overhaul?

What are you seeing, and how does that differ from what you expect to see?
Currently all plan jobs for each three of those run automatically. I want to ensure that when I only change the aws/dev/global/rds folder, I only run the matrix job indicated from that change using the changes:compare_to feature to the main branch.

Instead of all three of those folders running, if I only change a Terraform file in the rds folder, it should only trigger the matrix job that matches.

What version are you on? Are you using self-managed or GitLab.com?

Self managed GitLab running GitLab Enterprise Edition 15.11.11-ee.

Add the CI configuration from .gitlab-ci.yml and other configuration if relevant (e.g. docker-compose.yml)
The following CI is what I have in a shared template library (simplified for ease of reading)

.tf-base:
  before_script:
    ... specific logic here for auth...
    ... other variable configuration ...
    # Run 'terraform init'
    terraform init -upgrade -input=false

tf-plan:
  extends: .tf-base
  rules:
    - if: $CI_COMMIT_TITLE =~ /^release:/
      when: never
    - if: $CI_COMMIT_TAG
      when: never
    - if: '$ENABLE_TF_PLAN != "true"'
      when: never
    - if: $CI_COMMIT_BRANCH
      changes:
        compare_to: 'refs/heads/$MAIN_REF'
        paths:
          - '*/$TF_DIRS/**/*'
    - when: on_success

Below here is the actual implementation of the code above being used in a repository with the parallel:matrix field so that I can dynamically run my Terraform across multiple providers and folder structures.

.dirs-aws: &dirs-aws
  - "dev/global/rds"
  - "dev/global/iam"
  - "dev/global/cluster"

...

tf-plan:
  tags:
    - IaC
  parallel:
    matrix:
      - PROVIDER: aws
        TF_DIRS: *dirs-aws
...

What troubleshooting steps have you already taken? Can you link to any docs or other resources so we know where you have been?
I’ve tried a variety of combos with how to reference only different variables - I’m not sure the optimal approach on how to manage the paths and the rules so that I can reference the important pieces (is it even possible to reference dynamically set variables in the rules section of CI?)

I’ve looked through as many google searches on a variety of keywords relating to the topic as well as the open and closed issues on GitLab

THANK YOU so much for going through this - I would appreciate any suggestions or help here!

2

Hi @mayoraku , came across your post looking for a similar situation.

Here I post you what I did to make it work, hope it helps for your case:

...
terraform format:
  stage: validate
  script:
    - gitlab-terraform fmt
  parallel:
    matrix:
      - TARGET: test
      - TARGET: qa
  environment:
    name: $TARGET
    action: verify
  rules:
    - changes:
        - terraform/$TARGET/**/*
...

With that code, I manage to create specific jobs for a given env(TARGET) only if the terraform/$TARGET/**/* folder contains modifications.
Probably you can add there the compare_to and most probably should also work.

Hope it helps!

1 Like