I am managing the pipeline of a monorepo containing a design system. This means that there’s a lot of git activity with corresponding pipelines/jobs. A while ago we split our
test job into separate jobs;
yarn types:check/etc, because this gave us a clean overview regarding job status directly from the MR, and it improved the speed of the pipeline. 5x small parallel jobs run faster than 1 large slow job.
However, lately we’ve been running low on GitLab Runners. Or better said, more and more projects in our organisation demand CI/CD. This causes our jobs to often show up as waiting for several minutes, sometimes up to 15/20 minutes before it gets picked up by a runner.
Now, I was wondering if there’s a functional difference under the hood between stages and using the parallel + matrix keyword to run parallel jobs. Since the latter clearly states in the documentation:
Multiple runners must exist, or a single runner must be configured to run multiple jobs concurrently.
I indeed see that with this solution the same runner is being re-used. But I am curious if the same applies with simply using stages. This because stages feel more intuitive:
danger ci: extends: .test script: - yarn danger ci --failOnErrors flow checking: extends: .test script: - yarn run flow linting: extends: .test script: - yarn run lint
test: extends: .test script: - 'eval "$JOB_CMD"' parallel: matrix: - JOB_CMD: - 'yarn run packages:check' - 'yarn danger ci --failOnErrors' - 'yarn run lint' - 'yarn run types:check' - 'yarn run test:coverage' artifacts: reports: junit: - coverage/junit.xml paths: - coverage
Which also falsely looks for coverage in the non-relevant jobs.
Ideally we would want to spin up a single docker instance and in parallel on that instance run the 5 jobs. But as far as I have read, that does not seem possible with GitLab CI. But please correct me if I am wrong.
- What version are you on? Are you using self-managed or GitLab.com?
- GitLab: 14.7.2, self-managed
- Runner: AWS