Testing varaible against regular expression in before_script

I am trying to manipulate a custom variable in my before_script based on the commit branch.

The TAG variable is set on the runner server in my process and is used as the version of the docker image that is created by the runners. I want to prepend ‘TEST-’ to $TAG if the commit branch is in the test/ namespace.

Relevant parts of my gitlab_ci.yml file are:

before_script:
  - export TAG=$(date +%y.%.m.%d)
  - |-
      if [[ "$CI_COMMIT_BRANCH" =~ "/^test/\.*$/i" ]]; then
        export TAG="TEST-${TAG}";
     fi;
  - echo $TAG

I’m reasonably certain that the regex test is correct because the build won’t happen unless that specific test passes in the rules: section of my build stage. As in:

rules:
  - if: '$CI_COMMIT_BRANCH =~ /^test\/.*$/i'

The process runs, but the $TAG variable is set only to the date without the ‘TEST-’ prefix.

I’m certain this is something very minor that I have missed. Any help is greatly appreciated.

Hi @jordonedavidson

This is maybe a stupid question, but if you have

rules:
  - if: '$CI_COMMIT_BRANCH =~ /^test\/.*$/i'

surely you always want the value of TAG to start with TEST-?

Yes, that is correct. I am attempting to set that in the before_script section.

I’m sure I’m just off on the syntax for that part, because the same regex works in the rules: but not in the script.

Right, so I guess my question is, why have the if statement at all, if it will always be triggered?

How else can I change the value on the fly? The goal is to have this happen automatically. and TAG is set dynamically at run-time.

Well, there are a number of ways! But I would be inclined to do something like this:

stages:
    - test

.job: &job
    stage: test
    script:
        - echo ${TAG}

test-job:
    <<: *job
    rules:
        - if: '$CI_COMMIT_BRANCH =~ /^test\/.*$/i'
          when: on_success
        - when: never
    before_script:
        - export TAG=TEST-$(date +%y.%.m.%d)
        - echo ${TAG}

This does mean that you need to export the TAG variable in every job that uses the .job anchor, but you can probably get away with only implementing rules and before_script in the concrete jobs.

Unfortunately you can’t do this yet:

.job: &job
    variables:
         TAG: $(date +%y.%.m.%d)

due to this upstream bug.

Yes, if that rule was for a specific job I would do that. What isn’t explained in the original question is that I’ve only showed the minimal code to explain my issue. The jobs in question have several rules that control when they run. This is an additional case.

Also I imagine if I can’t get it working in the before_script, that test likely won’t work in the script part of the job either.

1 Like

OK, this works for me:

test-job:
    stage: test
    rules:
        - if: '$CI_COMMIT_BRANCH =~ /^test\/.*$/i'
          when: on_success
        - when: never
    before_script:
        - export TAG=$(date +%y.%.m.%d)
        - |-
            if [[ "$CI_COMMIT_BRANCH" =~ ^test/.*$ ]]; then
              export TAG="TEST-${TAG}";
            fi;
    script:
        - echo ${TAG}

You may well know about this already, but just FYI I tested this in an empty repo by adding some echo statements:

    before_script:
        - export TAG=$(date +%y.%.m.%d)
        - echo "${CI_COMMIT_BRANCH}"
        - echo ${TAG}
        - |-
            if [[ "$CI_COMMIT_BRANCH" =~ ^test/.*$ ]]; then
              echo "PING"
              export TAG="TEST-${TAG}"
              echo ${TAG}
            fi;
        - echo ${TAG}

Then running the YAML from the command line:

$ export CI_COMMIT_BRANCH=test/branch

$ gitlab-runner exec shell test-job
Runtime platform                                    arch=amd64 os=linux pid=353031 revision=58ba2b95 version=14.2.0
WARNING: You most probably have uncommitted changes. 
WARNING: These changes will not be tested.         
Running with gitlab-runner 14.2.0 (58ba2b95)
Preparing the "shell" executor
Using Shell executor...
executor not supported                              job=1 project=0 referee=metrics
Preparing environment
Running on eascsl05010nl...
Getting source from Git repository
Fetching changes...
Initialised empty Git repository in .../scratch-space/builds/0/project-0/.git/
Created fresh repository.
Checking out 24726fd3 as test/branch...

Skipping Git submodules setup
Executing "step_script" stage of the job script
$ export TAG=$(date +%y.%.m.%d)
$ echo "${CI_COMMIT_BRANCH}"
test/branch
$ echo ${TAG}
21.%.m.10
$ if [[ "$CI_COMMIT_BRANCH" =~ ^test/.*$ ]]; then # collapsed multi-line command
PING
TEST-21.%.m.10
$ echo ${TAG}
TEST-21.%.m.10
$ echo ${TAG}
TEST-21.%.m.10
Job succeeded

$

Which is a bit easier than pushing to a repo every time.

Well, that may be the trick. I notice that you don’t have quotes or / / around the regex in your conditional.

Yes, that seemed to do the trick. Bash is very picky!

1 Like

Hello @snim2
thanks for above examples.
I am trying to use it in my solution with a bit more complex regex. Unfortunately can’t push it to work :frowning:

I need to check if commit tag is in format sometext_x.x.x where x is a number (eg. test_0.1.1)

so I’ce created following condition:

stages:

  • test
stages:
  - test

test-job:
    stage: test
    rules:
        - if: '$CI_COMMIT_TAG =~ /^\w+_[0-9]+(?:.[0-9]+)+$/'
          when: on_success
        - when: never
    before_script:
        - export TAG=$(date +%y.%.m.%d)
        - |-
            if [[ "$CI_COMMIT_BRANCH" =~ ^test/.*$ ]]; then
              export TAG="TEST-${TAG}";
            fi;
         - |-
            if [[ "$CI_COMMIT_TAG" =~ ^\w+_[0-9]+(?:.[0-9]+)+$ ]]; then
              export TAG="TEST2-${TAG}";
              export VERSION=$(echo "$CI_COMMIT_TAG" | sed 's/.*_//');
            fi;
script:
        - echo ${CI_COMMIT_BRANCH}
        - echo ${CI_COMMIT_TAG}
        - echo ${TAG}
        - echo ${VERSION}

Unfortunately it doesn’t work:

May I kindly ask your advice?

Thanks in advance!

Hi @smarthomepch

I think this is probably a reg exp question, more than a GitLab question. Personally, I usually try an online reg exp tester when I have problems like this. There are many online (this one seems OK) and you should be able to use them to test with a variety of test strings, which should give you some confidence that you’ve ended up with a good solution.

Thank you for prompt response.
I think I do something wrong in GitLab because reg exp is created properly.
The first what I did was to check it in regex101.com

on top on this I have exactly the same reg exp in the rule:

test-job:
    stage: test
    rules:
        - if: '$CI_COMMIT_TAG =~ /^\w+_[0-9]+(?:.[0-9]+)+$/'

and it works perfectly fine.

This made me suprise why it doesn’t work in before_script if statment.

So, this worked for me, just in my terminal:

if [[ "$EXAMPLE_TAG" =~  ^[0-9a-zA-Z]+_[0-9](\.[0-9]){0,2}$ ]] ; then echo Y ; else echo N ; fi

I’ve messed around with this quite a bit, but it was the \w at the start that Bash seemed to object to.

1 Like

This works perfectly fine!
Thanks a lot!
Kudos to you!

1 Like