`job is not in any previous stage` error on multiple `needs`

How to Use GitLab
1

I have a repository with tests for frontend and backend. Both are changed only when changes are made in corresponding folders. The deployment is to be manually executed when both jobs are successfully completed. Now I have the problem that when I make changes that only affect the frontend or backend, I get the error message 'production-deploy' job needs 'backend' job, but 'backend' is not in any previous stage.

My .gitlab-ci looks like


stages:
  - run-ci-sub-pipelines
  - deploy

backend:
  stage: run-ci-sub-pipelines
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      changes:
        - backend/**/*

  trigger:
    include: .gitlab-ci/pipeline.backend.yml
    strategy: depend

frontend:
  stage: run-ci-sub-pipelines
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      changes:
        - frontend/**/*

  trigger:
    include: .gitlab-ci/pipeline.frontend.yml
    strategy: depend

production-deploy:
  stage: deploy
  needs: [backend, frontend]

  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      when: manual

  script: "echo deploy production"

If I don’t specify any rules I get the same error.

What is my expected behaviour?

  • For changes in frontend the fontend-ci is started and the job production-deploy is omitted.
  • For changes in backend the backend-ci is started and the job production-deploy is omitted.
  • For changes in both frontend and backend both subpipelines are started and I am able to manually trigger deploy-production.

I have not found a proper workaround for this.

My version: GitLab Community Edition 15.2.0 (self hosted)

2

Hi @techcreation

In this example you don’t need needs in the production-deploy since it is in next stage and it will be always started only after either or both “sub-pipelines” are done. It will work without the needs, the production-deploy will always be there but manual so you can just ignore it and not execute.

Unfortunately, changes are evaluated with OR so I can’t think of a smart way of achieving your expected behaviour.

I can think of several “not-that-smart” solutions like using rules based on commit message or using dynamic child pipelines where you would check what changed and create it only if both backend and frontend are changed.

3

If I remove the needs keyword then, I get two pipelines.
The first one runs frontend and backend and the second one running production-deploy is skipped by default (and can be run manually). Also I am able to start production-deploy regardless of the state of the other pipeline.
I think the behaviour I get is similar to (Pipeline for merge requests split up into two).

4

You get 2 pipelines if you have jobs in MR pipeline and jobs in branch pipeline.

Make sure you have the rule to run production-deploy for MR only.