How to disable a job conditionally for branch pipelines using rules without accidentally enabling it for merge request pipelines

GitLab CI/CD
,
1

:hugs: Please help fill in this template with all the details to help others help you more efficiently. Use formatting blocks for code, config, logs and ensure to remove sensitive data.

Context

By default a CI configuration that doesn’t use any rules does not launch a merge request pipeline, but does launch for basically every other pipeline source (push, tag, trigger, pipeline, etc). This is fine.

I am developing a number of CI/CD components that developers in my org are encouraged to use to build/test/secure their applications.

I would like to ensure that jobs configured by the components can be disabled individually via job rules but otherwise run in whatever pipeline the developer has configured.

Problem to solve

I have a very basic project with a very basic CI configuration here: https://gitlab.com/duncanmmacleod/test-pipeline-rules. This pipeline has a single job that always runs, which results in pipelines for pushes to all branches, tags, but not merge request pipelines (as per the default).

If I add a second job to the pipeline on a feature branch, a ‘push’ pipeline is launched, but no duplicate ‘merge request’ pipeline is launched when an MR is opened for that branch. That can be seen here: https://gitlab.com/duncanmmacleod/test-pipeline-rules/-/merge_requests/2

However, I would like to configure a job that supports being disabled via rules, e.g. by a variable that specifies disabling it, so that the job can be excluded under certain circumstances (web pipelines, schedules, etc).

My ‘naive’ attempt to achieve this is something like

conditional job:
  script:
    - echo "This is a test"
  rules:
    # allow users to disable it
    - if: $DISABLE_TEST_JOB == 'true'
      when: never
    # but otherwise run it as normal for all types of pipelines where it
    # would run by default
    - when: on_success

However. This rules stanza ‘accidentally’ triggers a merge request pipeline that wasn’t asked for. This can be seen here: https://gitlab.com/duncanmmacleod/test-pipeline-rules/-/merge_requests/1

I understand that I could use the following for the ‘otherwise run’ rule:

    - if: $CI_COMMIT_BRANCH

which is used throughout many of the job templates/components provided by gitlab-org (e.g. here), however that has the negative side effect of meaning that this job is excluded from merge request pipelines if the developer opts into those using workflow:rules. See https://gitlab.com/duncanmmacleod/test-pipeline-rules/-/merge_requests/3

Is there a way to configure a job with rules that effectively achieves the following:

job:
  rules:
    - if: disabled by whatever means
      when: never
    - if: otherwise
      when: on_success

without also having to configure workflow:rules and without spawning merge request pipelines that weren’t asked for?

Any advice is great, including ‘no this isn’t possible’.

Steps to reproduce

See https://gitlab.com/duncanmmacleod/test-pipeline-rules/ and proposed changes in open merge requests.

Configuration

See https://gitlab.com/duncanmmacleod/test-pipeline-rules/ and proposed changes in open merge requests.

Versions

Please select whether options apply, and add the version information.

  • Self-managed
  • GitLab.com SaaS
  • Self-hosted Runners

Versions

Helpful resources

  1. Before opening a new topic, make sure to search for keywords in the forum search
  2. Check the GitLab Runner and GitLab projects for existing issues. If you encounter a bug, please create a bug report issue.
  3. Troubleshooting docs: GitLab Runner, self-managed GitLab instances.

Thanks for taking the time to be thorough in your request, it really helps! :blush:

2

As far as I can tell, this is a non-trivial problem. See also

and probably many more related issues.

3

I get a 404 when I try to look at your test-pipeline-rules branch, did you delete it or is it a permission issue?

Am I understanding you correctly that you would your users to be able to define rules for your components if they need it, but otherwise you do not wish for any rules to be configured?
I was attempting to use rules in my CI/CD components myself but I found out that that simply adding a single “rules:” key in the ci config totally changes how the jobs will be executed.
In the end I gave up and now don’t have any rules in my components but instead I define the rules under the include: if I want to apply some rules to the components.

4

Sorry @gardar, I forgot to check the permissions on the project, please try again.

Your understanding is not quite correct, it’s that I want a default set of rules so that users can disable a job using just variables, but that the ‘positive’ rule match not be tied either to only branch pipelines or to trigger a merge request pipeline. If that makes sense.

5

Ok so what you are looking for is a way to disable the pipeline with a variable, but also avoid duplicate pipelines?

Would this work?

rules:
  # allow users to disable it
  - if: $DISABLE_TEST_JOB == 'true'
    when: never
  # Start a merge request pipeline when in a MR
  - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    when: always
  # Don't start a branch pipeline when in a MR
  - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
    when: never
  # Start a branch pipeline in any other cases
  - if: '$CI_COMMIT_BRANCH'
    when: always
6

Unfortunately, I don’t think this will work.

If I include those rules in my component, because they evaluate to true for the if merge request pipeline rule, they will trigger merge request pipelines for all users whether the user wants one or not. This is not the default.

The problem is basically that there isn’t a when: default rule that says ‘include this job according to the default rules’.

7

Yep, that’s what I was hinting at in my initial reply. That is if you include any rules, or even just a empty rules list rules: [] then it changes the behavior of the workflow.
I gave up myself and removed any rules from my ci components.
I’m not sure if the only/except conditional have the same behavior, did you try using those?

8

@gardar, I have not tried using only/except because those are deprecated and I don’t want to start relying on a feature that is likely to be removed.

I might try translating this to a gitlab.com issue and see if that gets any attention.