Do not stop environment when new pipeline is triggered in same branch

We have an environment that has an on_stop job configured.
We also have the option Auto-cancel redundant pipelines enabled.
Now whenever a new pipeline starts in the same branch, the on_stop job gets triggered, stopping our environment.
Is it possible to configure environments so that they are not automatically shut down on new pipelines?
We also want to keep Auto-cancel redundant pipelines enabled.