Long story short, I have a pipeline that 1. generates some yml dynamically and 2. triggers that pipeline as a child. This “child pipeline” is a single job that’s essentially running a bunch of other (variable amount) of pipelines
The term “variable count” refers to the fact that the number of “items” is determined at runtime.
FILE: .gitlab-ci.yml
generate-pipeline:
stage: generate-jobs
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE != "web"
artifacts:
public: false
name: "auto job group unreal ci yml"
paths:
- "dynjobs/"
- ".gitlab-ci-group-run.yml"
script:
# some script that generates all the variable count jobs in the "dynjobs" folder.
# the number of jobs created in "dynjobs" is determined at runtime.
# also generates the ".gitlab-ci-group-run-yml" from the "template file" below.
run-pipeline:
stage: run-jobs
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE != "web"
variables:
PARENT_PIPELINE_ID: $CI_PIPELINE_ID
needs: ['generate-pipeline']
trigger:
include:
- artifact: .gitlab-ci-group-run.yml
job: generate-pipeline
strategy: depend
FILE: template file
run-group-pipeline:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE != "web"
needs:
- pipeline: $PARENT_PIPELINE_ID
job: run-pipeline
artifacts: true
trigger:
include:
{0} strategy: depend
The “{0}” is filled in by a string formatter via pwsh, containing a variable amount of repeated entries of the following
- artifact: dynjobs/.gitlab-ci-dynamically-generated-stuff-1.yml
job: generate-pipeline
- artifact: dynjobs/.gitlab-ci-dynamically-generated-stuff-2.yml
job: generate-pipeline
- artifact: dynjobs/.gitlab-ci-dynamically-generated-stuff-3.yml
job: generate-pipeline
# etc
The relationships here seem sound to me, HOWEVER, I’m getting an odd error when I try to run this.
jobs:run-group-pipeline:needs config uses invalid types: cross_dependency
What does this mean, and how do I fix it?
Googling “cross_dependency gitlab ci” turns up 0 results, the things I see have to do with cross-project shenanigans, which isn’t relevant here.
Just to test, I also attempted to comment out trigger,include,strategy, and all the dynamic -artifact/job lines in the generated file “.gitlab-ci-group-run.yml” and add a test script like
script:
- echo "Hello World!"
- dir
- cd dynjobs
- dir
and everything (all the dynamically generated yml files in the dynjobs folder, and also the “.gitlab-ci-group-run.yml” file show up as expected, along with the repo contents.
This indicates to me that the pipeline in the “.gitlab-ci-group-run.yml” is getting evaluated/“executed” in some sense, it’s just that when doing the trigger/include/artifact,job*N,strategy thing causes issues for some reason.
I even modified the template file to simply have something like
trigger:
include:
- .gitlab-ci-test-a.yml
- .gitlab-ci-test-b.yml
where these two a & b files are simple pipeline files in the repo that just echo something, and they both get executed fine.
Thank you for your time!