How to avoid testing in Ci after merge

I’m using CI for developing a solution using a monorepo. It has two custom python libraries, and high level orchestration scripts using these packages.

The CI is split into 3 parts: build, test and deploy. During build, I create an image (fixed name plus pipeline id) in ECR (staging repository) with kaniko, which is used in test stage to run unit tests and integration tests. These two run always for a merge request. If both these pass, I’ll merge the development branch into main branch, and a new CI gets triggered. It repeats the build and test stage, and then deploy stage starts. I add some very high level wrappers on the image created in build stage and publish it with a version tag (identified by merge request id) and latest in ECR (deploy repository).

Everything works in this approach, but the build and test stages repeat for the CI that got triggered before merge (as a merge request pipeline) and for the one that got triggered after merge (as a branch pipeline). Since these two steps take a considerable time, and also since it’s sort of pointless, I’d like to avoid this repetition. Is there a way to handle this scenario?

The main challenge I’m facing is how to identify the base image created during build stage (identified by pipeline id). If I can do that, it’s simple. But I do not know how to get that pipeline id. I have to use an identifier in image tag as a lot of people are working and there can be simultaneous pipelines by other people.

I also considered using commit id instead of pipeline id, but can’t use it as merge creates a commit of its own and hence the id changes. And trying to access the last commit id before the latest seemed too much of a hack, so I’m hoping there is a better simpler solution.

Thanks

I wouldn’t say that these steps are useless. Imagine the scenario where you merge two MRs.
So you would have the following “flow” where you test every time you merge something.

          <main>
           /   \
        <MR1> <MR2>
          |     |
        test1  test2
          |     |
        <main>  | 
          |     /
        test3  /
       /  |   /
deploy  <main>
          |
        test4
       /
deploy

This is good, because in the case where changes in MR2 don’t play well with changes in MR1, your tests in test4 will fail and the deploy job will not run.
There’s a reason the term “integration hell” exists. Things may work by themself, but when you put them together, everything goes up in flames :stuck_out_tongue:

1 Like

Thanks for the reply.

We have set it up so that only fast forward merges are allowed. So, if MR1 is merged, MR2 is non-mergeable instantly. After someone handles it, that’s a new commit and tests do run it before merge itself.

Do you think there’s a reason to repeat tests after merge in this situation?

If I understand it correctly, I do agree with you that in this case, running the tests again should not be necessary.