Is it possible to limit permissions on the job/pipeline retry button?

We are enforcing a single pipeline created(and running) at any given time per project via Webhook triggers and custom queue logic. This is because, as we benefit from parallel jobs and there’s no way to tell the CI to have a cap on the number of running pipelines, we can’t ensure that a new pipeline starts before the previous one ends(and thus invalidating the cache mechanism in some cases).
This is fine if every pipeline is triggered via Webhook as we can handle the time it’s created, but the Retry button in the UI(Cancel is ok) allows to rerun directly that pipeline(we have no control over it, a Pipeline trigger is sent instead, there’s no such Retry trigger).
Is there a way to at least limit for Developers the ability to retry Jobs from the UI ? I would rather remove completely the Retry button as an option though.

Thanks !