Is there a way to test changes to the pipeline jobs?

situation is we’ve built out a multi pipeline library of jobs, generic jobs that can be included into any and all of the pipelines.

but this causes an issue. any time we make a change to one of the jobs, we need to make sure that all of the pipelines that use it still work. which means a run of every pipeline that uses that step.

in an ideal world there’d be a way of testing each job. mock the inputs, run the script, test the outputs. but i can’t work out how to do that.

i can sort of see a way of running every job inside a container, with only inputs passed on the command line, but ive no idea how we’d get outputs out of the container.

has anyone implemented any form of testing for the jobs that constitute pipelines? any way of unit testing the steps?

TIA

1 Like