Add conditions to rules array merged with `!reference`

GitLab CI/CD
1

Hello.

In #322992, closed by !67922 and !67922, we have the possibility to merge rules directly with !reference since Gitlab 14.3, like:

.no-schedule:
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never

.on-branch:
  rules:
    - if: $CI_COMMIT_BRANCH
      when: on_success

job-for-branches:
  image: alpine:latest
  rules:
    - !reference [.no-schedule, rules]
    - !reference [.on-branch, rules]
  script:
    - echo "run on branches except schedules"

On gitlab.com, this produced the following merged YAML output:

---
".no-schedule":
  rules:
  - if: $CI_PIPELINE_SOURCE == "schedule"
    when: never
".on-branch":
  rules:
  - if: "$CI_COMMIT_BRANCH"
    when: on_success
job-for-branches:
  image: alpine:latest
  rules:
  - - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
  - - if: "$CI_COMMIT_BRANCH"
      when: on_success
  script:
  - echo "run on branches except schedules"

Now, I would like to add more conditions to the passing rule to check if some file changes or exists, for example, I want the following:

job-for-branches-if-Dockerfile-exists:
  image: alpine:latest
  rules:
  - - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
  - - if: "$CI_COMMIT_BRANCH"
      when: on_success
      exists:
        - Dockerfile
  script:
  - echo "run on branches if Dockerfile exists except schedules"

But I can’t extend what is referenced, something like the following does not work:

.no-schedule:
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never

.on-branch:
  rules:
    - if: $CI_COMMIT_BRANCH
      when: on_success

job-for-branches-if-Dockerfile-exists:
  image: alpine:latest
  rules:
    - !reference [.no-schedule, rules]
    - !reference [.on-branch, rules]
      exists:
        - Dockerfile
  script:
    - echo "run on branches if Dockerfile exists except schedules"

This produce the error:

This GitLab CI configuration is invalid: (<unknown>): did not find expected '-' indicator while parsing a block collection at line 14 column 5

I seems great to be able to reference an item of the rules array to avoid nested array and allow hash merging, something in the idea of:

.no-schedule:
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never

.on-branch:
  rules:
    - if: $CI_COMMIT_BRANCH
      when: on_success

job-for-branches-if-Dockerfile-exists:
  image: alpine:latest
  rules:
    - !reference [.no-schedule, rules]
    - !reference [.on-branch, rules, 0]
      exists:
        - Dockerfile
  script:
    - echo "run on branches if Dockerfile exists except schedules"

But actually, using individual rule hashes does not permit this kind of merging:

.rules-map:
  no-schedule:
    if: $CI_PIPELINE_SOURCE == "schedule"
    when: never
  on-branch:
    if: $CI_COMMIT_BRANCH
    when: on_success

job-for-branches:
  image: alpine:latest
  rules:
    - !reference [.rules-map, no-schedule]
    - !reference [.rules-map, on-branch]
      exists:
          - Dockerfile
  script:
    - echo "run on branches if Dockerfile exists except schedules"

Does someone have any idea or should I open a feature request?

Regards.

2

Hi,

thanks for the detailed question, took me a while to understand the request. I don’t think it is possible to extend rules with !reference and support inline appending an existing rule.

Breaking it down for the feature request:

.on-branch:
  rules:
    - if: $CI_COMMIT_BRANCH
      when: on_success

job1:
  script: echo "test"
  rules:
    - !reference [.on-branch, rules]
      exists:
        - Dockerfile

I’m not sure if the config parser allows for merging the different scopes, but I am not a backend engineer. Let’s see in the new feature proposal you’re creating :slight_smile:

Cheers,
Michael

3

Our current workaround use a .rules-map which simplify a lot custom rules.

Thanks.

4

Thanks, I opened a feature request on the tracker.

2 Likes
5

This discussion came up when I was searching for how to extend a !reference-ed rule with additional job-level predicates. In pseudo-code it’d be something like this:

.on-branch:
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH =~ /^release...a few more ORs and ANDs

job1:
  script: echo "test"
  rules:
    - !reference [.on-branch, rules] && addtional-predicate-1 && addtional-predicate-2

Alternatively, it’d also be ok if I could AND-combine !reference-ed rules such that the job only runs if all rules match (rather than the first that evaluates to true).

I wonder if this warrants its own feature request.

6

Every idea is welcome and appreciated. If you think it matches the other feature request, add a comment into the issue to discuss. Or create a new feature proposal :slight_smile: