How do I exit a pipeline early without failure?

I have a pipeline where the first job makes a decision about whether the rest are executed. Neither choice is a “failure” in the “send email and alert everyone” sense. It simply does or doesn’t need to be done at that time.

I’m looking for the opposite of allow_failure: true which allows the pipeline to continue regardless of the exit status of the job. Instead, I want determine the continuation status.

Is there an easy or common way to do this? Or am I, once again, stuck with generating yaml files?

This is an interesting use-case.

I would be inclined to put the jobs and stages that you want to execute “sometimes” in a child pipeline rather than trying to exit early.

The other option might be to do something clever with rules and exists or variable, but I think that would be less clean.

I have used artifacts before child pipelines.
First job does some logic as yours and then generates a simple text file that is passed to next jobs thanks to artifacts. Each subsequent job checks the content of the file and exits if it should.

It has it’s disadvantages, like jobs are still scheduled and executed and take some time to just exit, but if it is not a concern this is one option :slight_smile:

I don’t like the artifacts approach for two reasons. First, yes, those jobs still get scheduled and executed and there can be a bunch of them. And second, you can’t easily tell from the initial UI which ones did anything and which ones did not.

Thinking about recursions lead me to a different thought from children, which is basically just recursion on the initial pipeline. First instance, (submission), spawns the second, (singularity check), potentially spawns the third, (common work). It’s a bunch of additional pipelines being spawned but they should all succeed and they should all be easily identifiable from the UI.

I’ll need to try that.

1 Like

This seems to be working so thanks for the provocation and the hint about recursion. :).

To reiterate, my solution for now is that submissions run some checks, then curl a pipeline start of a second phase, keyed by a local variable.

The second phase checks whether it’s the eldest running second or third phase. If not, it does nothing but exit. If so, it curls a pipeline start of the third phase.

Third phase runs longer term checks that would take too long to run on every submission. Since they run back-to-back, the third phases are essentially piggy backing all changes up to the point that the third phase starts. Once complete, the third phase checks to see whether the sha it just built is still current. if so, it exits. If not, it curls a pipeline start of a fresh second phase.

The answer to my original question is that “you can’t” in gitlab. Gitlab isn’t that sophisticated. It has many special cases for unusual cases but simple stuff like this isn’t handled as a first class feature. So instead, I’ve split my moral pipeline into two different passes or phases through the gitlab pipeline. The UI is a little confused in that there are now three different things showing up on the pipelines page, (really more like a half dozen including branch builds, nightly builds, etc), and no easy way to see that they are three different moral pipelines. But they all pass in the routine case so the only failures that should show are actual run time failures.

I wish gitlab had a way to declare a comment for a pass through the pipeline that would show up on the pipelines UI page. That would make the pipelines UI page much easier to read.

1 Like

Interesting, I think you should raise an issue for this feature…

pipeline comments to identify distinct passes through the .gitlab-ci.yml pipelines (#329215) · Issues · GitLab.org / GitLab · GitLab for pipeline comments.

1 Like

early (successful) pipeline exit (#329217) · Issues · GitLab.org / GitLab · GitLab for early exit.

2 Likes

Another way to do this: make an API call to cancel the pipeline. I have pipelines that auto-delete themselves for similar reasons. Works just fine.

3 Likes

This is indeed a needed feature.

I will try @clement.moyroud suggestion (thanks) for now

My solution doesn’t actually work in my context. It does result in a decision being made with displaying any failures. But it opens a concurrency window in the context I’d intended to use it.

I’ll have to try merging the two phases back together and using the self cancel trick.

I tried canceling the pipeline, but this marks all remanent jobs as canceled as well.

I only cancel the decision-making job, which is a blocking job, this way the rest of the pipeline is marked as skipped.

@josetruyol may I ask, when you cancel the decision-making job, how do you know it needs to be cancelled?

I will cancel when It’s not the eldest.

And I want all the rest of the jobs in the pipeline to be canceled. That’s sort of the point. If you don’t need that, if you just want a job to end, you can exit.

Canceling is not working for me, though. I apparently don’t have the authority to cancel jobs. But it’s hard to tell. I often get the same message when the call is bugged or misformed or any number of other reasons.

Sure would be nice if the protections and security stuff were documented somewhere.

If the job is part of a deployment, I think you can configure this in a couple of ways. If the job is not deploying anything, then I guess this is hard.

Looks like

Resource_Group:

is more general that just deployments, but it just serializes. I actually want the second one to terminate.

And I want all the rest of the jobs in the pipeline to be canceled. That’s sort of the point. If you don’t need that, if you just want a job to end, you can exit.

@agapanthusblue it depends, if I just exits the job, I have two options:

  1. Error code: marks the pipeline as Failed
  2. Success code: continue with the next stages

That’s why I cancel the job. In my pipeline, if this job does not succeed, the rest of the pipeline should not be executed. Of course, cancel the pipeline has the same effect.

@snim2 My decision-making job is a semantic release with a dry run. If there is no new version, I don’t want to generate a new release, nor new deployment to our test environment.

1 Like

Thanks, that’s a really interesting use-case.

Agreed. Self-cancel seems to be what I was originally looking for… a way to stop the pipeline without raising a failure.