GitLab shared runner: sequential pipeline builds

I set up some CI for my company’s Java projects last year on GitLab. We’re using a shared runner, that executes a couple of phases per project pipeline. This worked well, but due to a few unforeseen reasons we had to bench it for awhile.

An example of the ordering on GitLab and how this worked (simple phases with Java Maven projects):

  • Project A: Run tests (Pipeline 1, Job 1)
  • Project A: Deploy jar to artifactory (Pipeline 1, Job 2)
  • Project B: Run tests (Pipeline 2, Job 3)
  • Project B: Deploy jar to artifactory (Pipeline 2, Job 4)

I’ve picked it back up recently and it seems like after a certain GitLab upgrade the behaviour has changed slightly – the pipeline no longer executes in the order above. I’ve done some research and I think it may be due to how the shared runner utilises a ‘shared usage queue’ (I don’t think it did this last year, on our version at least) – https://docs.gitlab.com/ee/ci/runners/#how-shared-runners-pick-jobs

The execution now goes like this:

  • Project A: Run tests (Pipeline 1, Job 1)
  • Project B: Run tests (Pipeline 2, Job 3)
  • Project A: Deploy jar to artifactory (Pipeline 1, Job 2)
  • Project B: Deploy jar to artifactory (Pipeline 2, Job 4)

This introduces problems where Project B may depend on Project A, and if API breaking changes have been introduced in Project A, it needs to be deployed to the artifactory before Project B starts running its tests.

I’ve tried changing settings in the gitlab-runner config.toml, like making sure concurrent builds are turned off, but nothing seems to have helped.

I’ve also tried renaming the phases from ‘test’ & ‘deploy’ to other random words, to make sure it wasn’t grouping phases together with the same name from different pipelines.

I’m wondering if there is a way to turn off the ‘shared usage queue’ of the shared runner and have pipelines executed sequentially in the order of their job number?