Difficulty with CI/CD Job artifacts and dependencies
Hello - I’m working on some updates to a gitlab-ci.yml config which we have been using for some time, and we are seeing some behaviour with Job artifacts which I don’t understand and, at first impressions, seems inconsistent.
We’re using GitLab.com, and are working from the docs at:
Here’s a snippet from our CI file:
process openapi:
stage: Build
image:
name: mikefarah/yq
entrypoint: [""]
before_script:
- yq --version
script:
# Create and process yaml file (removing x-logo)
- mkdir apispecout
- cat openapi/api-spec.yaml | yq 'del(.info.x-logo)' >> apispecout/api-spec-out.yaml
after_script:
- ls apispecout/
tags:
- 3di-linux
artifacts:
paths:
- apispecout/
build openapi ref:
stage: Build
image: node:latest
before_script:
# Install the npm package dependencies, with cache, using package-lock.json
- npm ci --cache .npm --prefer-offline
- ls
script:
- npx redoc-cli bundle openapi/api-spec.yaml -t openapi/custom.hbs
# Move the output to output folder
- mkdir output
- mv redoc-static.html output/index.html
dependencies:
- process openapi
tags:
- 3di-linux
artifacts:
paths:
- output/
build hugo project:
stage: Build
image:
name: klakegg/hugo:latest-ext
entrypoint: [""]
before_script:
- hugo version
# Install theme package dependencies, using cache
- cd themes/3di
- npm ci --cache ${CI_PROJECT_DIR}/.npm --prefer-offline
- cd -
- ls
script:
- hugo
variables:
# Retrieve all submodules configured in .gitmodules
GIT_SUBMODULE_STRATEGY: recursive
dependencies:
- build openapi ref
tags:
- 3di-linux
artifacts:
paths:
- public/
test html output:
stage: Test
image:
name: wjdp/htmltest
entrypoint: [""]
# Temporarily allow this to fail
allow_failure: true
before_script:
- htmltest -v
- ls
script:
# Run at warning and skip externals (for API links)
- htmltest -l 2 -s public
dependencies:
- build hugo project
tags:
- 3di-linux
What we’re finding is that only the job test html output (from the Test stage) is able to see the artifacts from it’s declared dependencies build hugo project, in this case the public/ folder:
None of the other jobs in the Build stage are able to see their declared dependencies. For example, the build openapi ref job cannot see the artifacts from the process openapi job, in this case the apispecout/ folder:
We know that the apispecout/ folder, and the file it contains, are being created by process openapi, because the quick debug there is showing it:
As far I can see, we’re following the docs correctly for the artifacts and dependencies keywords… but of course, I’m bound to be doing something wrong.
Can anyone help?
Epic thanks,
Keith