Unit testing of GitLab CI/CD pipelines themselves (like Jenkins Shared Pipeline Libraries Unit Tests)?

GitLab CI/CD
1

I am an experience Jenkins user and I am wondering if there is anything in GitLab analogous to Jenkins Pipeline Unit Testing capabilities?

How does one go about applying CI/CD best practices for GitLab ‘pipeline’ themselves? Or, how does gitlab facilitate ‘eating its own dog-food’ vis-a-vis CI/CD pipelines themselves.

Presumably, gitlab allows reuse of shared ‘code’ across multiple pipelines; so how are these to be themselves tested?

2 Likes
2

This is resurecting an old question, I know, but I’ve asked myself this several times. For any sufficiently complicated automation pipeline—especially those that interact with multiple repositories or call the Gitlab API—I have yet to find a workflow that doesn’t involve pushing commits (with names like “CI fix”, “Maybe this time?”, and “FUUUUUUU”) and alt-tabbing to a browser to initiate something like an MR or branch creation.

I’ve seen a few things like gitlab-ci-local, but as far as I can tell they all just reimplement the CI yaml parsing with their own quirks and limitations and don’t provide a way to mock the Gitlab API, which is not going to give full fidelity.

I’m close to trying to build a docker compose that stands up a minimal Gitlab instance and CI runner, but that seems like an overengineered act of desperation and I anticipate the test speed would be awful.

I wish Gitlab would provide us something with the library code that parses the CI YMLs that would let me test assertions like “with these environment variables/files changed/etc., I expect these jobs to be run”. (I may dig into this myself, but obviously I’m not an expert on the codebase.) At least that way I could unit test the script steps like conventional software and then be able to check that pipelines do the right things depending on how they’re triggered, which is probably at least 20-40% of the development time I spend on CI automation. Add a mock API server (I think there are a few tools that will create one from an OpenAPI spec) and we have another 20-40%.

Anyway, sorry for anyone hoping for an answer here.

3

I spent yesterday seeing if I could implement something myself on this. The goal was to have a Docker container where I could bind mount Gitlab CI templates and Rspecs that would be run against them.

I found a spec within the Gitlab codebase that does the kinds of tests I’d like to run against my own CI templates. Unfortunately, it doesn’t seem like it’s easy to extract it from the rest of the codebase, but I’ll drop some breadcrumbs in case anyone with more context on the Gitlab codebase wants to follow up:

  • That spec calls CI::CreatePipelineService
  • That returns a CI::Pipeline ActiveRecord object from execute which holds references to the builds, variables, etc. that I’d like to run assertions on.
  • Inside of execute, the CreatePipelineService passes the Pipeline through a chain of responsibility under lib/gitlab/ci/pipeline/chain

So, at the very least, the Rails environment needs to setup so we have a database connection for Ci::Pipeline. I don’t know if that means that the entire Gitlab stack needs to be up or if there’s a way to start a minimal subset (or mock environment).

I ran into a bunch of issues around bundle installing the source code, but they may have been local problems. Toward the end of the day I tried installing GDK, hoping maybe I could whittle it down to something smaller from there, but I ran into problems with mise.