Using different Docker images during a single Gitlab-CI job

GitLab CI/CD
,
1

I’m facing a probably classical problem, but I don’t know how to deal with, mostly due to the reset during job switching.

In my pipeline, I wish to run instrumented test and collect these data to display them on a dashboard (let’s call it SonarQube :wink: ) In order to do this, I would appreciate to use (at least) two distinct Docker environments:

  • one with the build requirements (dependencies, compilers, etc.)
  • one with the collector

I wish to use two container images because there are really distinct in term of dependencies.

Of course, the collector has to work on specific files generated by the execution of tests. If I run each step in different stages, I will lost these files.

Is there a way to describe a job using different images for specific steps?
Or is there any way to tight two jobs/stages ensuring the second jobs inherit a non-cleaned workspace?

2

For example, something like (pictures/examples worth thousands of words):

job:
  scripts:
  - image: instrumented-build-env
    script:
    - make
    - make test
  - image: collector
    script:
    - collect
3

hey @gbonnefille , did you ever figure out how to do this?

4

Oh, just answered my own question: https://docs.gitlab.com/ee/ci/yaml/#image. Looks like the answer is yes. Yay!

1 Like
5

@benswift can you provide a piece of code as example? I can’t find anything related to multiple images in the same job in the doc you provided.

Thanks a lot

6

Sure - you can just add an image key to the specific job (which will override the top-level image specified in your CI file, e.g.

image: ubuntu:latest

run_tests:
  image: ubuntu:someothertag
  stage: test
  script:
    - echo "woohoo, tests pass"
7

@benswift Hey, so we can only use single image per job ?

What if I want multiple images, is there a syntax for that ?

I am thinking something like

run_tests:
  image: [ubuntu:someothertag , node:image]
  stage: test
  script:
    - echo "woohoo, tests pass"
8

No. All the documentation leads one to conclude the following without actually saying it directly:

Gitlab CI supports up to 1 image per “ci layer”, e.g. pipeline, job, …

All of the language and yaml spec indicate at any layer image is a single string, and cannot be a list like services supports.

I researched this a bit while preparing to migrate a multi-image, one-service-per-image docker compose project to run in gitlab CI. I have concluded that my best option is to merge my 2 images to include the superset of dependencies required by both services and specify both. This isn’t really that bad of an option because the 2 services always go together. I would never want to run the 2nd image and service without running the 1st in tandem.

References:

p.s. I should mention that since ive been Looking into migrating the whole doctor composed project into a k8s project, I will next look into the k8s executor to see if there’s a better solution there.

9

You can do it with parallel:matrix

10

Yeah, you can definitely pull this off. Just use the services keyword in your .gitlab-ci.yml to spin up a sidecar container with your collector image. This way, both your build and collector can share the same workspace during the job. Alternatively, you can use artifacts to pass the files between stages, but if you want everything in the same job, sidecar is probably your best bet.