How to block pipeline steps from running in parallel across different pipelines

Hi all,

I understand that gitlab-ci divides it’s pipeline into steps, which can be organises into stages. In each stage all steps can run in parallel, while each stage has to wait for the previous one to finish. So far so good. My question is not about stages, but multiple build pipelines running in parallel. My problem is that I can’t figure out how I can tell the gitlab-ci to please wait with the execution of stage X (or step Y) if the same stage (or step) is already running in any other pipeline (on a different gitlab-runner).

For example: I have a pipeline that automatically builds and deploys the master branch to staging. Our deploy step can run into trouble if it is executed by two gitlab-runners in parallel. When two commits arrive on master within a short period of time, two runners will start pipelines in parallel and the deploy stages will overlap, causing issues.

I assume one option would be to simply just use a single gitlab-runner, but that will slow the rest of our pipeline down as well and all steps in each stage will not be able to run in parallel.

I previously used concourse-ci heavily for this exact same use-case and they have an options to control which pipeline steps (they call them jobs) are allowed to run in parallel:
https://concourse-ci.org/jobs.html#job-serial
https://concourse-ci.org/jobs.html#job-serial-groups

Is there a similar option in gitlab-ci ?

2 Likes

Our solution is to use a single runner with a ‘deploy’ tag and then tag our deploy jobs that can’t run in parallel. Everything else can run in parallel on all of our other runners. It’s worked great for us and seems like a reasonable approach.