Do not run a job if no change outside specific paths

We have a repo for a python package, and a Streamlit UI to expose the functionalities of that package to the end users. The structure is as follows:

./
├── .git/
├── .gitatributes
├── .gitignore
├── .gitlab-ci.yml
├── gitlab-ci-dockerfiles/
├── gitlab-ci-templates/
├── kubeflow-pipeline-components/
├── main-python-sdk/
├── orchestration-scripts/
└── streamlit-ui-scripts/

Flow of the CI is to build a base image with our custom python library, run some tests (unit/integration) and create Kubeflow pipelines. We are in the process of adding a final component of updating UI deployment also as part of CI, which we had been doing manually before.

In order to do that, we realise if a merge request has changes only in the UI folder, we do not need to run the other jobs, as those remain exactly same. So, we are looking for rules which will basically tell this: “do not run job if change is present only in streamlit-ui-scripts”.

We have tried using rules:changes together with when: never, and that helps to solve this problem, but it needs that we set it for every single elements (root level files/folders) except the Streamlit folder. And we need to repeat this rule for all of the non-UI jobs. And if there is ever a change in directory structure, we’ll need to edit all these rules for all these jobs.

So, my question if there a direct opposite of this:

rules:
    changes:
        - streamlit-ui-scripts/**/*

This one runs a job only if there is a change in streamlit-ui-scripts folder, and we want to set a rule that will skip a job if change is present only in streamlit-ui-scripts folder.

Thanks.

1 Like

Hi @anirban.ray

Looking at these docs I think you probably want something like this:

rules:
    - if: true
      changes:
          - streamlit-ui-scripts/**/*
      when: never
    - when: on_success

Hi @snim2, thanks for your answer.

Please correct me if I am wrong, but will it not fail in case changes are present in both within and outside streamlit-ui-scripts? As far as I understand from the documentation, since part of changes will satisfy first rule which has when: never, the job is going to be skipped, which is not what I want.

What I want is as follows:

  1. if changes are made only within streamlit-ui-scripts, skip jobs unrelated to UI and run UI related jobs
  2. if changes are detected only outside streamlit-ui-scripts, run jobs unrelated to UI and skip UI related jobs
  3. if changes are present both within and outside streamlit-ui-scripts, run both types of jobs

Based on my understanding, your solution works for cases 1 and 2, but not 3. If I am missing something, please correct me.

Also, these sort of rules need to be added to every single job. It’d be great if include would support rules:changes or we can define rules on stage/tag. This is not strictly related to my question, but I feel it’d be a great feature to add.

1 Like

Hi @anirban.ray

You are right, my proposal works for 1 and 2 but not 3.

However, you can have any number of if statements in a rules block, and these will be evaluated from top to bottom, so you can just add another rule before mine.

Also, these sort of rules need to be added to every single job…

Yes, but you can use anchors (or extends) to make this easier. For example:

.rules-changes: &rules-changes
    rules:
        - if: true
          changes:
              - streamlit-ui-scripts/**/*
          when: never
        - when: on_success

build:
    stage: build
    <<: *rules-changes
    script:
        - make
    ...

It’d be great if include would support rules:changes or we can define rules on stage/tag.

Yes! If there isn’t one already, please do raise an issue for this.

However, you can have any number of if statements in a rules block, and these will be evaluated from top to bottom, so you can just add another rule before mine.

Does this allow for skipping jobs if there are no changes outside of a specified folder? I can’t see how it would do that from your example

  rules:
        - if: true
          changes:
              - streamlit-ui-scripts/**/*
          when: never

“if invalid expression syntax” -does not even compile for me

I am also looking for such a feature.
Another use case is for example not to run job if the only files that were changed are ‘*.md’ files.
I do not have a solution.

Other people are asking for it too:

  1. In `changes`, add validation of whether only the listed files are changed (#348141) · Issues · GitLab.org / GitLab · GitLab
  2. gitlab-ci "rules" should support extended globbing or some negation of changes/exists (#198688) · Issues · GitLab.org / GitLab · GitLab
    but, as usual, GitLab refuses to add features that its costumers need.

I suggest you to follow the above issues so will get notification if they are fixed. You can also +1 them.