Extend job rules with multiple !reference containing array of rules

Multiple rule extensions with !reference which points to an array of rules

I am trying to create monorepo deployment based on templated jobs, each job in the template already has its own set of rules, which I want to reuse and add a few additional ones inside the job, but it fails every time.

Consider having the following .gitlab-ci-template.yml file which looks something like this

.job:
  stage: some_stage
  script:
    - ./some_nasty_script.sh
  rules:
    - if: $something
      when: always
    - if: $something_else
      when: never

And another .gitlab-ci.yml which will be used to orchestrate the pipeline for your project

job1:
  extends: .job
  rules:
    - !reference [.job, rules]

If I have only 1 reference property inside rules, everything will work fine (side note: when I’ve checked the merged file from GitLab editor, I have noticed that it just spread the included rules without flatting them out so they looked like the following snippet)

job1:
  extends: .job
  rules:
    - - if: $something
       when: always
      - if: $something_else
       when: never

But when I have multiple jobs in my project, where some are being built, test, deploy, etc. If I have an additional set of rules which should be shared among them (but can’t be nested inside the template from 1st snippet because they are exclusive for that project) and i try to include them with an additional reference like in the following snippet, it doesn’t function. Or should I better say, it only evaluates the 1st !reference, 2nd one gets ignored

.default_rules:
  rules:
    - changes:
       - apps/specific-app/*
    - if: $FORCE_DEPLOY
      when: always


job1:
  extends: .job
  rules: 
    - !reference [.job, rules]
    - !reference [.default_rules, rules]

Then the merged yml will look something like this:

job1:
  extends: .job
  rules:
    - - if: $something
        when: always
      - if: $something_else
        when: never
    - - changes:
        - apps/specific-app/*
      - if: $FORCE_DEPLOY
        when: always

I am using our self-managed GitLab 14.10.3-ee

Before posting here I was checking GitLab’s documentation on rules, along with a few issues and PRs in hope that it would help me with my issue, but so far nothing helps.

If anyone has additional ideas for bypassing this limitation at the moment, do tell
Maybe I am doing something so obviously wrong, but I haven’t managed to pinpoint it yet. Unless this actually is a bug/potential feature.

There is always the final option to just write all the rules manually, but that really bums me out, since later maintenance would require much more manual labor.

Thanks for this post - I had this exact same question. The docs for rules say

You can use !reference tags to reuse rules configuration in different jobs.

But and when I click through to reuse rules configuration, I see this example:

.default_rules:
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

job1:
  rules:
    - !reference [.default_rules, rules]
  script:
    - echo "This job runs for the default branch, but not schedules."

job2:
  rules:
    - !reference [.default_rules, rules]
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  script:
    - echo "This job runs for the default branch, but not schedules."
    - echo "It also runs for merge requests."

Putting this in the pipeline editor results in:

".default_rules":
  rules:
  - if: $CI_PIPELINE_SOURCE == "schedule"
    when: never
  - if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
job1:
  rules:
  - - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
  script:
  - echo "This job runs for the default branch, but not schedules."
job2:
  rules:
  - - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
  - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  script:
  - echo "This job runs for the default branch, but not schedules."
  - echo "It also runs for merge requests."

I don’t find any place saying how rules supports nested arrays, so the expected behavior here is ambiguous. I’d like to request that the docs be updated. What is the expected behavior?

The documentation for rules states:

Rules are evaluated when the pipeline is created, and evaluated in order until the first match

Is it possible in your case that either $something or $something_else will be true? In that case, it will ignore rules below that point.

I just created a sample repo that demonstrates that multiple !references of rules works just fine: multi-rules-reference-test. See sample merge requests and their pipelines.