How to matrix subsequent jobs referencing artifacts from previous matrix job?

GitLab CI/CD
1

Although I know of an undocumented way to pick one artifact from a previous matrixed job (dependencies: ["matrixed_job_name: [varName]"] CI: Specifying artifact dependencies when using `parallel: matrix:`), is there a way to make the subsequent job matrixed in a way it starts a job for each artifact from the previous job ? Each such job should pick one artifact from the previous job.

E.g.

stages:
  - build
  - deploy

build:
  stage: build
  script:
    - bash -c $BUILD_TARGET.sh
  parallel:
    matrix:
      - BUILD_TARGET: ['linux', 'windows', 'mac']
  artifacts:
    expire_in: 1hrs
    when: always
    paths:
      - ./builds

deploy:
  stage: deploy
  script: 
    - deploy.sh ./builds/$BUILD_TARGET
  parallel:
    matrix:
      dependencies: build

If not it means I either have to put everything in one job or write a deploy job for each build, neither seem too clean.

2

@nurdofitru Welcome to the forum!

Unfortunately right now I think you need to use a different job for each matrix dependency (now documented).

I created a new issue for a feature request if you want to give it a :+1: because I didn’t see an existing one.

-James H, GitLab Product Manager, Verify:Pipeline Execution

3

Hello!

Indeed sharing artifacts between matrix jobs is not straightforward but it’s possible to do it in a clean way. Maybe the following solution would solve the issue?

stages:
  - build
  - deploy

.targets:
  parallel:
    matrix:
      - BUILD_TARGET: windows
      - BUILD_TARGET: linux
      - BUILD_TARGET: mac

build:
  stage: build
  extends:
    - .targets
  environment: $BUILD_TARGET
  script:
    - bash -c $BUILD_TARGET.sh
    - mv build build-$BUILD_TARGET # update `build` to reflect your case
  artifacts:
    paths:
      - build-$BUILD_TARGET
    expire_in: 1 hour

deploy:
  stage: deploy
  extends:
    - .targets
  environment: $BUILD_TARGET
  needs:
    - build
  script: 
    - cd build-$BUILD_TARGET
    - ls # all build artifacts for $TARGET are available in `build-$BUILD_TARGET`
    - deploy.sh ./$BUILD_TARGET
4

I really like how you’ve structured it with the .targets extension, it definitely keeps things clean and organized. Using needs to reference the build artifacts for each target is a clever approach.

1 Like
5

Thanks, @brewbalt

If anyone wants to know more about this technique, feel free to check our blog post:
u11d .com/blog/sharing-artifacts-between-git-lab-ci-matrix-jobs-react-build-example/

Added spaces to URL because I couldn’t post with real link. Please copy the link and remove the space before .com :slight_smile: