Unexpected behaviour of "rules" in GitLab CI

GitLab CI/CD
1

Hi. I have some problems with understanding how and why “rules” in GitLab CI work.

I have written a minimal code showing the issue in a GitLab project: Daniel Grabowski / gitlab-ci-rules-problems · GitLab It contains two directories (“files1/” and “files2/”) with some files in them and a .gitlab-ci.yml file.

My configuration

Here’s the CI configuration:

stages:
  - build

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "push"

.job_tpl:
  image: alpine:latest
  stage: build
  variables:
    TARGET_BRANCH: $CI_DEFAULT_BRANCH
  rules:
    - if: $CI_COMMIT_BRANCH == $TARGET_BRANCH
      changes:
        - $FILES_DIR/**/*
      variables:
        JOB_ENV: "prod"
    - if: $CI_COMMIT_BRANCH != $TARGET_BRANCH
      changes:
        - $FILES_DIR/**/*
      when: manual
      allow_failure: true
      variables:
        JOB_ENV: "dev"
  script:
    - echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
    - echo "TARGET_BRANCH=$TARGET_BRANCH"
    - echo "JOB_ENV=$JOB_ENV"

files1 job:
  extends: .job_tpl
  variables:
    FILES_DIR: files1

files2 job:
  extends: .job_tpl
  variables:
    FILES_DIR: files2

As you can see in the above code I’m using workflow to run only “branch pipelines” and have two “twin” jobs configured to watch for changes in one of the project’s directories each. The TARGET_BRANCH variable is of course unnecessary in the demo project but i need something like this in the real one and it shows one of my problems. Additionally the jobs behave differently depending on the branch for which they are run.

My expectaions

What I want to achieve is:

  • Each of the jobs should be added to a pipeline only when I push changes to files1/ or files2/ directory respectively.
  • When I push changes to a branch different then “main” a manual job responsible for the changed directory shoud be added to a pipeline.
  • When I merge changes to the “main” branch a job responsible for the changed directory shoud be added to a pipeline and it should be automatically started.

Test scenario

  1. I’m creating a new branch from “main”, make some change in the file1/test.txt and push the branch to GitLab.
  • what I expect: a pipeline created with only “files1 job” runnable manually
  • what I get: a pipeline with both jobs (both manual). Actually I’ve found explanation of such behaviour here: Choose when to run jobs | GitLab - “The changes rule always evaluates to true when pushing a new branch or a new tag to GitLab.”
  1. On the same branch I make another change in the file1/test.txt and make push.
  • what I expect: a pipeline created with only “files1 job” runnable manually
  • what I get: exactly what I expect since the branch isn’t a “new” one
  1. I create a Merge Request from my branch to main and make the merge.
  • what I expect: a pipeline created with only “files1 job” which starts automatically
  • what I get: a pipeline created with only “files1 job” but a manual one

My questions/problems

  1. Can you suggest me any way to bypass the issue with “changes” evaluating always to “true” on new branches? Actually it behaves exactly as I want it if I don’t use “rules” but let’s assume I need “rules”.
  2. Why the jobs run as “manual” on the main branch in spite of the “if” condition in which both CI_COMMIT_BRANCH and TARGET_BRANCH variables are (or should be) set to “main”. To debug it I’m printing those vars in job’s “script” and when I run it on “main” pipeline I’m getting:
$ echo "CI_COMMIT_BRANCH=$CI_COMMIT_BRANCH"
CI_COMMIT_BRANCH=main
$ echo "TARGET_BRANCH=$TARGET_BRANCH"
TARGET_BRANCH=main
$ echo "JOB_ENV=$JOB_ENV"
JOB_ENV=dev

so theoretically CI should enter into the “automatic” job path.

Generally I find the CI “rules” quite inconvenient and confusing but as I understand it GitLab prefers them to “only/except” solution so I’m trying to refactor my CI/CD to use them which will fail if I don’t find solution for the above difficulties :frowning: