More than two levels of nested pipelines

We have a monorepo where we make extensive use of nested pipelines for two main purposes:

  • To isolate each sub-project’s pipeline to their own .gitlab-ci.yml, to avoid polluting the global variables and default configurations, and re-use job names without having to worry about collisions.
  • Make re-usable functions (pipelines) with arguments (variables) that can trigger multiple jobs. For instance, a Release pipeline that accepts two variables, PROJECT_NAME and PROJECT_VERSION. This pattern is superior to YAML anchors, !reference tags, and extends:, since it provides proper isolation for the function and lets it create more than one job. The alternative is very verbose and repetitive.

These two patterns naturally combine: a sub-project’s pipeline might trigger a Release pipeline. But at that point, we’re already hit the maximum nesting limit of child pipelines as described in the docs.

Parent and child pipelines were introduced with a maximum depth of one level of child pipelines, which was later increased to two. A parent pipeline can trigger many child pipelines, and these child pipelines can trigger their own child pipelines. It’s not possible to trigger another level of child pipelines.

This limitation seems arbitrary: we can easily have more than one level of nesting in our sub-project hierarchy (that is, a sub-project in our monorepo might have its own sub-projects), and a “function” pipeline should be able to be called from any sub-project depth (and call its own functions in turn).

Allowing an arbitrary amount of nesting can causes issues: a recursion cycle between pipelines is one such example. However, I think a sweet spot can be found that’s larger than 2, but lower than, say, 5 for instance. For self-hosted instances, this could be configurable.