Use CI commit in current execution

Hi everyone,

I have a Gitlab CI on my Typescript library, and I want it to autoupdate the version on push.

So for now, here is how it works :

.gitlab-ci.yml
include:
  - template: Jobs/Build.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml
  - template: Jobs/Test.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Test.gitlab-ci.yml
  
image: node:latest

stages:
  - prepare
  - build
  - test
  # - review
  # - dast
  - deploy

variables:
  REPOSITORY_URL: "https://$TOKEN_NAME:$ACCESS_TOKEN@gitlab.com/<my-project-path>.git"

workflow:
  rules:
    - if: "$CI_COMMIT_BRANCH == 'master'"
      variables:
        ENV_NAME: production

    - if: "$CI_COMMIT_BRANCH == 'develop'"
      variables:
        ENV_NAME: staging

    # https://github.com/heroku/heroku-buildpack-nodejs
    - exists:
        - package.json

prepare:
  # https://docs.npmjs.com/cli/v9/commands/npm-version#preid
  # https://stackoverflow.com/questions/51716044/how-do-i-push-to-a-repo-from-within-a-gitlab-ci-pipeline
  # https://gitlab.com/gitlab-org/gitlab/-/issues/389060/?_gl=1*1cbzdf5*_ga*NjA0NDA1NzM2LjE2NzkyNjI5Njc.*_ga_ENFH3X7M5Y*MTY4OTE0Nzk4MS4xOC4wLjE2ODkxNDc5ODIuMC4wLjA.
  stage: prepare
  except:
    - release
  script:
    - git config user.email "ci-pipeline@gitlab.com"
    - git config user.name "CI Pipeline"
    - git remote set-url --push origin "$REPOSITORY_URL"
    - npm version prerelease --message "CI Pipeline - v%s"
    - git push origin HEAD:$CI_COMMIT_REF_NAME -o ci.skip

prepare-release:
  stage: prepare
  only:
    - release
  script:
    - git config user.email "ci-bot@gitlab.com"
    - git config user.name "ci-bot"
    - git remote set-url --push origin "$REPOSITORY_URL"
    - tag=$(npm version patch -m "CI Pipeline - v%s")
    - git push origin HEAD:$CI_COMMIT_REF_NAME -o ci.skip
    - git push origin "$tag" -o ci.skip

deploy:
  stage: deploy
  environment: $ENV_NAME
  needs: ['build', 'test']
  script:
    - echo "//${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">.npmrc
    - npm publish --verbose

So when I push the new version, I ask to skip the job execution for this commit.
But in the next step (build / test / deploy), the version is not the new one (and so I get conflict on version number for deploy).

So is it possible to reuse the commit done by the CI in next steps ?
Or do I have to kill workflow when I update version, and skip prepare when I am in a commit pushed by CI ? (I don’t know how to do it by the way :confused: )

Another thing : I want the test to wait for prepare stage (I don’t know why build wait but not test)
I you know something about this, I take ! :slight_smile:

Thank you !

release is not supported condition for only or except. I think you need to use tags in there instead.

If you use needs in a job stages are ignored for that job.

1 Like

Hi @balonik, and thank you for taking time to answer me.

So you’re saying I have to tag each version (even the alpha), and only publish when CI is running on a tag ?

Yes I know, and I already use it for the deploy stage.
But for build and test I am using GitLab CI jobs, so if I override it, the stages imported will not be used, doesn’t it ?

It just occurred to me that the release is actually name of a branch. Sorry, I haven’t used only/except for a long time since it’s deprecated.
I get it now, you want to run the deploy job only if it’s a commit from the prepare or prepare-release jobs. In that case you need to drop the -o ci.skip from the last lines so the commits actually trigger new pipeline and then play with the conditions on the jobs so they are executed when needed. The easiest way would be to build the conditions around commit message. For advanced cases I will replace the only/except with rules:

prepare:
  rules:
    - if: $CI_COMMIT_MESSAGE =~ /^CI Pipeline/   # don't run if the commit message starts with 'CI Pipeline', preventing a loop
      when: never
    - if: $CI_COMMIT_REF_NAME != "release"       # run if the branch is not 'release'

prepare-release:
  rules:
    - if: $CI_COMMIT_MESSAGE =~ /^CI Pipeline/   # don't run if the commit message starts with 'CI Pipeline', preventing a loop
      when: never
    - if: $CI_COMMIT_REF_NAME == "release"       # run if the branch is 'release'

deploy:
  rules:
    - if: $CI_COMMIT_MESSAGE =~ /^CI Pipeline/  # run only if the commit comes from 'prepare' jobs
    - if: $CI_COMMIT_TAG                        # run for tags

The reason why job from test stage is not “waiting” for previous stages is caused by the Job definition of the included GitLab provided template Test.gitlab-ci.yml. The job has needs: [] so it ignores your stages definition. You need to override it with needs: null in order to remove the needs: []

1 Like

Oh I don’t know it was deprecated !

I tried to simplify what you suggested, but I do not works as expected. Indeed, the prepare stage is never run…

Here is the file :

include:
  - template: Jobs/Build.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml
  - template: Jobs/Test.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Test.gitlab-ci.yml
  
image: node:latest

stages:
  - prepare
  - build
  - test
  # - review
  # - dast
  - deploy
  # - performance

variables:
  REPOSITORY_URL: "https://$TOKEN_NAME:$ACCESS_TOKEN@gitlab.com/<my-project-path>.git"

workflow:
  rules:
    - if: $CI_COMMIT_BRANCH

    - if: $CI_COMMIT_REF_NAME == "release"
      variables:
        NPM_ARGS: 'patch -m "CI Pipeline - v%s"'

    - if: $CI_COMMIT_REF_NAME != "release"
      variables:
        NPM_ARGS: 'prerelease --message "CI Pipeline - v%s'

    - exists:
        - package.json

prepare:
  # https://docs.npmjs.com/cli/v9/commands/npm-version
  # https://stackoverflow.com/questions/51716044/how-do-i-push-to-a-repo-from-within-a-gitlab-ci-pipeline
  # https://gitlab.com/gitlab-org/gitlab/-/issues/389060/?_gl=1*1cbzdf5*_ga*NjA0NDA1NzM2LjE2NzkyNjI5Njc.*_ga_ENFH3X7M5Y*MTY4OTE0Nzk4MS4xOC4wLjE2ODkxNDc5ODIuMC4wLjA.
  stage: prepare
  rules:
    - if: $CI_COMMIT_MESSAGE =~ /^CI Pipeline/
      when: never
  script:
    - git config user.email "ci-pipeline@gitlab.com"
    - git config user.name "CI Pipeline"
    - git remote set-url --push origin "$REPOSITORY_URL"
    - tag=$(npm version $NPM_ARGS)
    - git push origin HEAD:$CI_COMMIT_REF_NAME
    - if [ $CI_COMMIT_REF_NAME == "release" ]; then git push origin "$tag" -o ci.skip; fi

test:
  needs: null

deploy:
  stage: deploy
  needs: ['build', 'test']
  rules:
    - if: $CI_COMMIT_MESSAGE =~ /^CI Pipeline/  # run only if the commit comes from 'prepare' jobs
  script:
    - echo "//${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">.npmrc
    - npm publish --verbose

If no statement in rules evaluates to true the job is not added as described in docs. Just add some dummy statement at the end like - if: $CI_COMMIT_REF_NAME

1 Like

Oh well, I have misunderstand the when: never, thank you.

After a discussion with a friend, we find a better/lighter solution.

I will only update version if not in release (so the prerelase one), and the release version will be done by the semantic release (not already implemented, I will see this later).
So the prepare stage is moved inside the deploy stage.

And that way, we only have one CI execution (for the moment - maybe more when the semrel will be implemented). No tag to handle, build and test runs together and the workflow end by deploy with package registry publishing and git push.

Here is the new .gitlab-ci.yml :

include:
  - template: Jobs/Build.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml
  - template: Jobs/Test.gitlab-ci.yml  # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Test.gitlab-ci.yml
  
image: node:latest

stages:
  - build
  - test
  # - review
  # - dast
  - deploy
  # - performance

variables:
  REPOSITORY_URL: "https://$TOKEN_NAME:$ACCESS_TOKEN@gitlab.com/<my-project-path>.git"

workflow:
  rules:
    - if: $CI_COMMIT_BRANCH
    - exists:
      - package.json

deploy:
  stage: deploy
  needs: ['build', 'test']
  script:
    # https://docs.npmjs.com/cli/v9/commands/npm-version
    # https://stackoverflow.com/questions/51716044/how-do-i-push-to-a-repo-from-within-a-gitlab-ci-pipeline
    # https://gitlab.com/gitlab-org/gitlab/-/issues/389060/?_gl=1*1cbzdf5*_ga*NjA0NDA1NzM2LjE2NzkyNjI5Njc.*_ga_ENFH3X7M5Y*MTY4OTE0Nzk4MS4xOC4wLjE2ODkxNDc5ODIuMC4wLjA.
    - |
      if [ $CI_COMMIT_REF_NAME != "release" ]
      then
        git config user.email "ci-pipeline@gitlab.com"
        git config user.name "CI Pipeline"
        git remote set-url --push origin "$REPOSITORY_URL"
        tag=$(npm version prerelease --message "v%s (CI Pipeline)")
        echo "Updating version to '$tag'"
        git push origin HEAD:$CI_COMMIT_REF_NAME -o ci.skip
      fi
    - echo "//${CI_SERVER_HOST}/api/v4/projects/${CI_PROJECT_ID}/packages/npm/:_authToken=${CI_JOB_TOKEN}">.npmrc
    - npm publish --verbose

We also think about a generic before_script that could do the job too (update version in before_script but git push only in deploy) - but as I do not need the new version in other stage, I choose the other proposal.

Thank you again @balonik for your help and yours answers, I learn a lot thanks to you ! :slight_smile: