I’m using CI for developing a solution using a monorepo. It has two custom python libraries, and high level orchestration scripts using these packages.
The CI is split into 3 parts: build, test and deploy. During build, I create an image (fixed name plus pipeline id) in ECR (staging repository) with kaniko, which is used in test stage to run unit tests and integration tests. These two run always for a merge request. If both these pass, I’ll merge the development branch into main branch, and a new CI gets triggered. It repeats the build and test stage, and then deploy stage starts. I add some very high level wrappers on the image created in build stage and publish it with a version tag (identified by merge request id) and latest in ECR (deploy repository).
Everything works in this approach, but the build and test stages repeat for the CI that got triggered before merge (as a merge request pipeline) and for the one that got triggered after merge (as a branch pipeline). Since these two steps take a considerable time, and also since it’s sort of pointless, I’d like to avoid this repetition. Is there a way to handle this scenario?
The main challenge I’m facing is how to identify the base image created during build stage (identified by pipeline id). If I can do that, it’s simple. But I do not know how to get that pipeline id. I have to use an identifier in image tag as a lot of people are working and there can be simultaneous pipelines by other people.
I also considered using commit id instead of pipeline id, but can’t use it as merge creates a commit of its own and hence the id changes. And trying to access the last commit id before the latest seemed too much of a hack, so I’m hoping there is a better simpler solution.
Thanks
