TLDR;
Including an external repository that contains pipeline jobs causes the build to fail with the error
invalid yaml, even when the ci.skip push option is passed.
What’s Happening
I am using an annotated git tag, and pushing it with the option ci.skip from inside the CI job.
The tag is created with the command
git tag -a $tag -m "Version created by gitlab-ci"
and is pushed with the command
git push tag-origin $tag --follow-tags -o ci.skip
To enable the CI job to tag the build, an Access Token is created with the permissions:
api, write_repository
When the CI process uses the include property to get common jobs from an external build repository,
the skip property doesn’t take effect, and the job throws an error of invalid yaml because it is unable
to pull the build repository, as the user that kicked off the job (the access token) does not have access
to the external repository.
When I push the tag and my CI process does not use the include option to include external jobs, the
skip option is honored and works as expected.
Working with no includes
The CI file shown below, will work correctly.
image: "gitlab-ci-docker-builder:ubuntu_18.04"
stages:
- build
- deploy
variables:
CICD_URL: "https://${GITLAB_USER_LOGIN}:${CICD_AT}@HOST.com/GROUP/cicd-build.git"
workflow:
rules:
- if: $CI_COMMIT_TAG
when: never
- when: always
default:
before_script:
- if [ "$CI_COMMIT_REF_NAME" == "master" ]; then export DOCKER_IMAGE_TAG="rc"; else export DOCKER_IMAGE_TAG="alpha"; fi
- git clone --single-branch --branch deploy $CICD_URL $CICD_DIR && cd $CICD_DIR && pwsh ./install.ps1 && cd ..
- export TAG_VERSION=`grep -m1 -oP '"version"\s*:\s*"\K[^"]+' component.json`-$DOCKER_IMAGE_TAG
cicd-build:
stage: build
services:
- name: docker:dind
alias: proxydocker
script:
# Other steps have been removed
- echo "TAG_VERSION=$TAG_VERSION" >> build.env
artifacts:
reports:
dotenv: build.env
cicd-publish:
stage: publish
services:
- name: docker:dind
alias: proxydocker
dependencies:
- cicd-build
needs:
- job: cicd-build
artifacts: true
rules:
- if: $CI_COMMIT_BRANCH == "master" || $CI_COMMIT_BRANCH == "dev"
when: on_success
script:
# Other steps have been removed
# This script will run the tag operations
- pwsh -NoProfile -ExecutionPolicy Unrestricted -Command "& { Start-Tag }"
Running a build using the CI above will produce the following
Failing when includes is specified
The CI file shown below will throw an error of invalid yaml not able to access cicd-build repository
image: "gitlab-ci-docker-builder:ubuntu_18.04"
include:
- project: 'cicd-build'
ref: deploy
file:
- '/gitlab-ci/gitlab-ci-scan.yml'
- '/gitlab-ci/gitlab-ci-deploy.yml'
stages:
- build
- scan
- publish
- deploy
variables:
CICD_URL: "https://${GITLAB_USER_LOGIN}:${CICD_AT}@HOST.com/GROUP/cicd-build.git"
workflow:
rules:
- if: $CI_COMMIT_TAG
when: never
- when: always
default:
before_script:
- if [ "$CI_COMMIT_REF_NAME" == "master" ]; then export DOCKER_IMAGE_TAG="rc"; else export DOCKER_IMAGE_TAG="alpha"; fi
- git clone --single-branch --branch deploy $CICD_URL $CICD_DIR && cd $CICD_DIR && pwsh ./install.ps1 && cd ..
- export TAG_VERSION=`grep -m1 -oP '"version"\s*:\s*"\K[^"]+' component.json`-$DOCKER_IMAGE_TAG
cicd-build:
stage: build
services:
- name: docker:dind
alias: proxydocker
script:
# Other steps have been removed
- echo "TAG_VERSION=$TAG_VERSION" >> build.env
artifacts:
reports:
dotenv: build.env
cicd-publish:
stage: publish
services:
- name: docker:dind
alias: proxydocker
dependencies:
- cicd-build
- cicd-scan
needs:
- job: cicd-build
artifacts: true
- job: cicd-scan
artifacts: false
rules:
- if: $CI_COMMIT_BRANCH == "master" || $CI_COMMIT_BRANCH == "dev"
when: on_success
script:
# Other steps have been removed
# This script will run the tag operations
- pwsh -NoProfile -ExecutionPolicy Unrestricted -Command "& { Start-Tag }"
Running a build using the CI above will produce the following
Don’t mind that the last step (deploy) is failing, the tag is kicked off from step 3
What’s Expected
My expectation was that the build would be skipped, an not try and run the includes operation to get the jobs from the external repository.