Controlling chil/parent pipelines and triggering

GitLab CI/CD
#1

I have 5 modules (each in their own folder) in a single Gitlab project, and all 5 modules will need their own pipelines. What I need is that

  • when a module is changed, a child pipeline for that module will run
  • if a module is not changed, then the child pipeline for that module will not run
  • one of the modules is building a common base for the others, so if there are changes, the pipeline for this one module needs to be run before the others, but if that one doesn’t have changes, then the pipeline for the base module should not be run at all. The other modules don’t need info about each other and their pipelines can run happily concurrently.

So my questions are:

  • What options are there to control starting a pipeline? Obviously there is the commit message, which I’d prefer using for other things, then there are branches, but that will cause lots of rebases, then there is manual or tags (which is also manual), but are there automated options that would notice for example which folder has changes? Or am I missing some other good option?
  • What is the best way to ensure that the pipeline for the base module is completed (if there were changes to it) before the others start?

Thanks in advance!

#2

Hi @busesblinktwice

I’m not sure whether parent / child pipelines are exactly what you need, although you certainly could use them.

This is the way I would solve your problem. Assuming you have directories like this:

base/
mod1/
mod2/
...

And all modules are built in roughly the same way, I would try something like this:

# Cache dependencies and plugins between builds.
# To keep cache across branches add 'key: "$CI_JOB_NAME"'
cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
        - .m2/repository

stages:
    - buildbase
    - build

workflow:
    rules:
        # Only run one pipeline in an MR.
        - if: $CI_MERGE_REQUEST_ID
          when: never
        - when: always

.build:
    script:
        - cd $MOD_DIR
        - mvn $MVN_BUILD_ARGS compile
    tags:
        - build

build:basemod:
    stage: buildbase
    needs: []
    extends: .build
    variables:
        MVN_BUILD_ARGS: "..."
        MOD_DIR: "base"
    rules:
        - when: always

.buildmod:
    stage: build
    needs: ["build:basemod"]
    extends: .build
    rules:
        - changes:
            - $MOD_DIR/*
          when: on_success

build:mod1:
    extends: .buildmod
    variables:
        MVN_BUILD_ARGS: "..."
        MOD_DIR: "mod1"

build:mod2:
    extends: .buildmod
    variables:
        MVN_BUILD_ARGS: "..."
        MOD_DIR: "mod2"

...

Note that I haven’t tested this!

The example above uses:

You could also use dependencies rather than needs, and there are other options for some of the other keys, also.

However with this definition, you cannot make the build:basemod job optional, until this issue has been resolved, and I can’t think of an easy way to make that first job optional.

HTH,

Sarah