Parts of my pipeline run tests that have to run on certain bits of real hardware. Obviously if I’m doing this, I need to prevent multiple jobs attempting to use the hardware at the same time. What is the best way of managing the potential conflict? Is there some form of mutex or resource that I can specify in my gitlab-ci.yml?
I’ve come up with two potential options:
Create a runner that I associate with the hardware and give it a limit of one job at once, and then ensure all jobs that touch that hardware run on that runner. This is fine as long as the relationship of job to hardware is one to one, but doesn’t cope so well down if two jobs use two overlapping but independent sets of hardware.
Create an internal web service that can act as a ‘mutex’ on the hardware and get that mutex at the start of each job, or block until available. This is going to mean that quite a few jobs can be sitting idling using up resource, and giving misleading indications of how long they took to run.
Is there a better way of doing this?