When: manual in downstream pipeline causes upstream to report failure

I am using a multi-project pipeline to separate my end to end tests from my main application code. My end of end tests, if I run the full suite, can take a significant amount of time. I’ve nicely broken them into various groupings using pytest and it’s mark feature.

I’d like to be able to run specific scenarios from within the pipeline now by setting each of these different scenarios to when: manual. Unfortunately, as soon as I add this, the child pipeline reports it has failed to the parent and progress stops. I can manually run each section, as expected, but even then success is not reported back to the parent pipeline.

This is an example of the pipeline. The Integration Tests step has reported a failure, and I’ve manually run the Fast Tests from the downstream pipeline. It has passed, and as the only job in the pipeline the entire downstream pipeline passes. Yet the parent still reports failure so Deploy doesn’t get run.

If I remove when: manual from the downstream pipeline, Integration Tests will run the full test suite, pass, and Deploy will move on as expected.

This is the parent project pipeline.


image: "python:3.7"

before_script:
  - python --version
  - pip install -r requirements.txt
  - export PYTHONPATH=${PYTHONPATH}:./src
  - python -c "import sys;print(sys.path)"

stages:
  - Static Analysis
  - Local Tests
  - Integration Tests
  - Deploy

mypy:
  stage: Static Analysis
  script:
    - mypy .

flake8:
  stage: Static Analysis
  script:
    - flake8 --max-line-length=88

pytest-smoke:
  stage: Local Tests
  script:
    - pytest -m smoke

pytest-unit:
  stage: Local Tests
  script:
    - pytest -m unittest

pytest-slow:
  stage: Local Tests
  script:
    - pytest -m slow

pytest-fast:
  stage: Local Tests
  script:
    - pytest -m fast

int-tests:
  stage: Integration Tests
  trigger:
    project: andy/gitlab-integration-testing-integration-tests
    strategy: depend

deploy:
  stage: Deploy
  when: manual
  script:
    - echo "Deployed!"

The end to end tests pipeline looks like this:


image: "python:3.7"

before_script:
  - python --version
  - pip install -r requirements.txt
  - export PYTHONPATH=${PYTHONPATH}:./src
  - python -c "import sys;print(sys.path)"

stages:
  - Fast Tests

pytest-smoke:
  stage: Fast Tests
  when: manual
  script:
    - pytest -m smoke

How can I selectively (manually) run downstream jobs and report success back to the parent pipeline? Without when: manual in that last step on the end to end (downstream) pipeline it performs exactly as I want. But, in the real world pipeline I have, I don’t want to run the end to end tests on everything. Usually it is selected scenarios.

1 Like

Hi @AWegnerGitHub

i’m currently struggling at this very spot. Since some time has passed i was woundering if you found a solution for this issue.

Cheers

I have not, but I am hopeful that this feature being worked on for 13.5 (coming out next week), solves the problem. I’ve been following that issue for a couple months now.

13.5 did not resolve this how I expected. The code samples provided above result in the same error as before. I am still investigating to see if there are other options now that this should be supported.

I’ve figured it out! The solution is a few lines of code to change from my original post. This also requires GitLab 13.5, because prior to that these changes would fail the pipeline.

In the main project, I still want the integration job(s) to be manually triggered and block so I added this to my int-tests

  when: manual
  allow_failure: false

Now the entire int-tests looks like this:

int-tests:
  stage: Integration Tests
  when: manual
  allow_failure: false
  trigger:
    project: andy/gitlab-integration-testing-integration-tests
    strategy: depend

Then in the child pipe line, I removed the when: manual line.

Now, the pipeline will run and become blocked, as expected, until the integration stage is manually triggered and passes. My next goal is to figure out how to have multiple jobs in a stage and require that one or more pass before I move on to the next (deploy in my case), but that’s go another time.