Blocked pipeline using rules with environment / review app

We’re having an issue using rules with environments / review apps on gitlab.com where the pipeline becomes blocked if we use rules instead of when. We expect the pipeline to automatically stop using rules, as documented.

Pipeline blocked. The pipeline for this merge request requires a manual action to proceed

If we manually trigger the stop job, the pipeline is considered complete.

Should this work or must we revert to only/except?

only/except syntax:

review:
  environment:
    name: $CI_COMMIT_REF_SLUG
    on_stop: stop:review
  script:
    - echo "review"
  stage: review

stop:review:
  environment:
    action: stop
    name: $CI_COMMIT_REF_SLUG
  except:
    - master
    - tags
  script:
    - echo "stop"
  stage: review
  variables:
    GIT_STRATEGY: none
  when: manual

rules syntax:

review:
  environment:
    name: $CI_COMMIT_REF_SLUG
    on_stop: stop:review
  script:
    - echo "review"
  stage: review

stop:review:
  environment:
    action: stop
    name: $CI_COMMIT_REF_SLUG
  rules:
    # Avoid executing on merge request detached pipelines
    - if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME'
      when: never
    # Execute only on branches (not master)
    - if: '$CI_COMMIT_BRANCH != "" && $CI_COMMIT_BRANCH != "master"'
      when: manual
  script:
    - echo "stop"
  stage: review
  variables:
    GIT_STRATEGY: none

I opened a support issue and they provided the following…

As per the CI docs

When using rules: , allow_failure defaults to false , including for manual jobs.

We can switch the allow_failure based on our needs, in the only syntax the default behaviour of allow_failure is true so no need to specify that explicitly; in the rules syntax the default behaviour is false so explicitly specifying the allow_failure: true should make the pipeline to a non-blocking state.

I think they mean to say that for review apps that use rules, the “on_stop” job must explicitly set allow_failure: true.

@hobti01 - was @anthonymastrean’s post helpful to you? Let us know! Happy to help troubleshoot further if you need!

Thanks for the feedback - unfortunately explicitly allowing failure does not unblock.

Also https://gitlab.com/gitlab-org/gitlab/-/issues/34077 is back and forth between multiple issues which all seem to imply that the behaviour of only is not possible with rules because there is no “implicit” allow_failure: true. So now you must be explicit (which doesn’t work for us) - but this changes the logic of the pipeline step if you want it to succeed when it is run.

:-/