Seeking help for pipeline stage rules regex

Trouble getting rules to work in a pipeline stage

After reading the relevant documentation section I want to run one of the stages only under certain conditions. In short, the following represents my problem:

variables:
  CI_COMMIT_MESSAGE: $CI_COMMIT_MESSAGE

build:
  stage: integration
  rules:
    - if: $CI_COMMIT_MESSAGE !~ /^wip/i
  script:
    - echo $CI_COMMIT_MESSAGE
    - echo "Do docker stuff if rule is true"

Expected behavior

  • With commit message “feat (api) #99: my commit subject” the stage should be run
  • With commit message “wip (api) #99: my work in progress” the stage should not be run

Actual behavior

No matter what I’m doing, I can’t get different behaviors for the two cases. To me it looks like somehow the regex is interpreted as a string and not as a regex pattern.

Conditions

  • $CI_COMMIT_MESSAGE is is available and prints out the expected commit message
  • I’m not getting any syntax working for the .gitlab-ci.yml file
  • All scripts and other fields are interpreted as expected.

Does any body see what I’m doing wrong or missing?

Any hint or idea would be appreciated.

Thank you very much!
Daniel

I haven’t tried this out, but I guess you want something like this:

variables:
  CI_COMMIT_MESSAGE: $CI_COMMIT_MESSAGE

build:
  stage: integration
  rules:
    - if: '$CI_COMMIT_MESSAGE !~ /^wip/i'
      when: on_success
    - when: never
  script:
    - echo $CI_COMMIT_MESSAGE
    - echo "Do docker stuff if rule is true"

I notice in the docs that reg exps in rules tend to be in single quotes, like this:

rules:
    if: '$CI_COMMIT_BRANCH =~ /^version\/[0-9-\.]+$/'
    ...

Hi snim2,

thanks a lot for your input! Unfortunately, your two hints do not solve the problem.

Since the rule should only render as true if the pattern /^wip/i/ is not found, the when: never doesn’t make a difference. Of course I’ve tried anyway. It doesn’t matter if the commit message starts with wip or another string, the evaluation is always the same.

Putting the the whole statements into quotes doesn’t seem to make any difference either.

Thanks again for your effort!

Best regards
Daniel

Oh, then I think you want something like:

  rules:
    - if: '$CI_COMMIT_MESSAGE =~ /^wip/i/'
      when: never
    - when: on_success

If that doesn’t work I’ll have a go in a test repo!

I’m still getting the same behavior. When I apply your suggestion the stage is still added to the pipeline, even if the commit matches the pattern, i. e. CI_COMMIT_MESSAGE starts with wip.

Somehow the patterns seems to be disregarded. Adding - when: on_success makes it run every time, no matter if the pattern is matched or not.

p.s. I had to leave out the extra slash after at the end, since it gitlab reported a syntax error in the yml. So I used

  - if:  '$CI_COMMIT_MESSAGE =~ /^wip/i'

instead of

  - if:  '$CI_COMMIT_MESSAGE =~ /^wip/i/'

… and if I omit the when: on_success this build stage is never added. It is as if the pattern is somehow either disregarded or always matched. That gave me the idea that maybe there is no regex but only a simple string matching.

I think the issue is probably your regexp. This seems to work for me:

stages:
    - hello
    - test

hello:
    stage: hello
    rules:
        - when: on_success
    script:
        - echo "HELLO"
test:
    stage: test
    rules:
        - if: '$CI_COMMIT_MESSAGE =~ /^wip.*/'
          when: never
        - when: on_success
    script:
        - echo "$CI_COMMIT_MESSAGE"

I did try this and quite some other patterns before. Gave it another shot, matching exactly your suggestion but I’m still getting the same behavior. The commit message pattern doesn’t seem to make any difference.

Also shouldn’t it be anyway sufficient to simple use

    - if: $CI_COMMIT_MESSAGE !~ /^wip.*/

At least this is what I’m understanding from the documentation as mentioned here

  • if: $CUSTOM_VARIABLE !~ /regex-expression/ : If the custom variable CUSTOM_VARIABLE does not match a regular expression.

Thanks again for your effort, I really appreciate that!

I’m still getting the same behavior regardless of the $CI_COMMIT_MESSAGE pattern. In the following screenshot the latest pipeline should not have the build stage, since it includes:

  rules:
    - if: $CI_COMMIT_MESSAGE =~ /^wip.*/
      when: never
    - when: on_success

However the build stage is added anyway in both cases.

The variable $CI_COMMIT_MESSAGE is available in any case. I echo it in the following stage.

I simply can’t spot where I make a mistake or where I might be missing something. The rest of the .gitlab-ci.yml works fine, so I suppose it really comes down to misunderstanding the rules syntax or a possible bug in how a correct rule and regex pattern is interpreted and applied.

Just for the record I include the whole .gitlab-ci.yml:

stages:
  - test
  - integration
  - staging
  - production

variables:
  CI_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
  CI_COMMIT_TITLE: $CI_COMMIT_TITLE
  CI_COMMIT_MESSAGE: $CI_COMMIT_MESSAGE

build:
  stage: integration
  # variables:
  #  CI_COMMIT_TITLE: $CI_COMMIT_TITLE
  #  CI_COMMIT_MESSAGE: $CI_COMMIT_MESSAGE
  rules:
    - if: $CI_COMMIT_MESSAGE =~ /^wip.*/
      when: never
    - when: on_success
  script:
    - echo "building containers for integration environment if commit message DOES NOT start with WIP"
    # do we have previous integration containers running? If so stop and delete them
    - if [[ -n $(docker ps -q -f name=integration) ]]; then docker -v rm $(docker stop $(docker ps -q -f name=integration)); fi;
    # Set env variable API_BASE_URL
    - export API_BASE_URL="https://integration.api.ifc-hive.karo.design"
    # build services from scratch
    - docker-compose -f docker-compose.yml -f docker-compose.integration.yml up -d --build
    # remove dangling images
    - docker rmi $(sudo docker images -f "dangling=true" -q) -f

# complete integration deployment
test-api:
  stage: test
  script:
    - echo "Run api tests"

deploy-integration:
  stage: integration
  script:
    - echo "Deploy to inegration environment"
    - echo "$CI_COMMIT_MESSAGE"

# deploy to staging
deploy-staging:
  stage: staging
  when: manual
  environment:
    name: "staging"
    url: "https://staging.docs.ifc-hive.karo.design"
  only:
    - master
  script:
    - echo "Deploy to staging environment"
    - git -C $STAGING_DIR_PATH pull
    # do we have staging containers running? If so stop and delete them
    - if [[ -n $(docker ps -q -f name=staging) ]]; then docker -v rm $(docker stop $(docker ps -q -f name=staging)); fi;
    # Set env variable API_BASE_URL
    - export API_BASE_URL="https://staging.api.ifc-hive.karo.design"
    # build services from scratch
    - docker-compose -f $STAGING_DIR_PATH/docker-compose.yml -f $STAGING_DIR_PATH/docker-compose.staging.yml up -d --build

# deploy to production
deploy-production:
  stage: production
  when: manual
  environment:
    name: "production"
    url: "https://ifc-hive.karo.design"
  only:
    - master
  script:
    - echo "Deploy to production environment"
    - git -C $PRODUCTION_DIR_PATH pull
    # do we have production containers running? If so stop and delete them
    - if [[ -n $(docker ps -q -f name=production) ]]; then docker -v rm $(docker stop $(docker ps -q -f name=production)); fi;
    # build services from scratch
    - docker-compose -f $PRODUCTION_DIR_PATH/docker-compose.yml -f $PRODUCTION_DIR_PATH/docker-compose.production.yml up -d --build

So, one difference between your config and mine is this section:

variables:
  CI_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
  CI_COMMIT_TITLE: $CI_COMMIT_TITLE
  CI_COMMIT_MESSAGE: $CI_COMMIT_MESSAGE

I don’t think you need that, the $CI_ variables are provided automatically, and I wonder whether not quoting those variables has messed something up? What happens if you remove that section?

That’s it. You are a star!

Though I don’t quite get why this messes up the behavior of the rule section. I understand that I do need the variables section if I want to make those variables available to my services. So in case one of my services would need $CI_COMMIT_MESSAGE as an environmental variable, it would be in conflict with my current use case.

I don’t know about services, but you don’t need that block for the jobs you are defining, so if you can get away without some of those lines, so much the better. Otherwise quoting should help a bit:

variables:
  CI_COMMIT_MESSAGE: '${CI_COMMIT_MESSAGE}'

or some such…

Hi Sarah,

ok, thanks for the hint!

Best regards
Daniel

1 Like