How to setup a child pipeline having multiple jobs with different triggers?

GitLab CI/CD
,
1

Code snippet and background
I have a workflow with multiple jobs as below:

stages:
  - another
  - yet-another

another-job:
  stage: another
  rules:
    - if: $CI_PIPELINE_SOURCE == "push"
      when: always
  script:
    - echo "Another job"

yet-another-job:
  stage: yet-another
  rules:
    - if: $CI_PIPELINE_SOURCE == "push"
      when: manual
      allow_failure: true
  script:
    - echo "Yet another job"

If I write this in the top-level .gitlab-ci.yml file, it works as expected. The first job is triggered on every push, the second job gets added to the pipeline and waits for the manual trigger.

Goal (what I want to do)
Now I need to add workflows for other usecases. Hence, to keep the workflows modular, I want to move the above into another yml file and trigger it using parent-child pipelines.

What I have done so far
So I move it to a file .gitlab-ci/another-workflow/.gitlab-ci.yml, and write the following in the top-level .gitlab-ci.yml (parent):

another-workflow:
  trigger:
    include: .gitlab-ci/another-workflow/.gitlab-ci.yml
  allow_failure: true

What doesn’t work

But this leads to the error:

another-workflow - failed - (downstream pipeline can not be created, Pipeline will not run for the selected trigger. The rules configuration prevented any jobs from being added to the pipeline.)

What I need help with

From the documentation, I figured out that I need to specify rules in the parent pipeline to trigger the child pipeline, but I cannot figure out what it should be.

The above example is clearly a simplified minimal working sample of my actual workflow at work. In my actual workflow, there are some dependencies and shared variables between the various jobs in the child workflow. For that reason, I want to avoid splitting up the child workflow file, if possible.

Question

Hence, I would like to setup the workflow in one of the following ways:

  • In the parent pipeline, specify:
    (1) on triggerA, run child.job1
    (2) on triggerB, run child.job2

  • In the parent pipeline, specify: “don’t ask me anything, just do as the child says”

  • Any other option?

Other information

  • Gitlab runner: gitlab-runner 16.3.0~beta.108.g2b6048b4 (2b6048b4)

  • As you might expect from the post above :), I have read through the documentation, forum posts, stack overflow, etc. and tried out the various suggestions, that are too numerous to be listed here.

  • I have set up a sandbox repository for this purpose here: Files · another-workflow · Aravind Pai / honeycomb · GitLab. Please feel free to use it for any trials.

Thank you for your consideration. Please let me know if any further information is needed from my side.

2

Hi @dragondive

if you use child pipelines in the same project, the pipeline source for jobs in child pipeline is always parent_pipeline.
There are two scenarios I will consider here:

  • all your jobs in the child pipeline have same if: $CI_PIPELINE_SOURCE == "push" condition
  • jobs in the child pipeline have different condition for CI_PIPELINE_SOURCE

For the 1st scenario:

Specifying rules on the trigger job is enough. Remove any CI_PIPELINE_SOURCE conditions on jobs in child pipeline.

another-workflow:
  trigger:
    include: .gitlab-ci/another-workflow/.gitlab-ci.yml
  allow_failure: true
  rules:
    - if: $CI_PIPELINE_SOURCE == "push" # this is enough, because trigger job won't be added so no child pipeline is actually created

Second scenario:

another-workflow:
  trigger:
    include: .gitlab-ci/another-workflow/.gitlab-ci.yml
  allow_failure: true
  variables:
    PARENT_PIPELINE_SOURCE: $CI_PIPELINE_SOURCE
  rules:
    - if: $CI_PIPELINE_SOURCE == "push"
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

and in .gitlab-ci/another-workflow/.gitlab-ci.yml

stages:
  - another
  - yet-another

another-job:
  stage: another
  rules:
    - if: $PARENT_PIPELINE_SOURCE == "push"
      when: always
  script:
    - echo "Another job"

yet-another-job:
  stage: yet-another
  rules:
    - if: $PARENT_PIPELINE_SOURCE == "push"
      when: manual
      allow_failure: true
  script:
    - echo "Yet another job"

yet-another-mr-job:
  stage: yet-another
  rules:
    - if: $PARENT_PIPELINE_SOURCE == "merge_request_event"
  script:
    - echo "MR pipeline job"
1 Like
3

Thank you @balonik. This works well for me.

For anyone else who comes here in future, you may use this commit for reference: Aravind Pai / honeycomb · GitLab I will not delete this branch or force push over it (but if Gitlab gets rid of it, there’s nothing I can do about it. :wink:)