How to use value from version file in repository, as image variable?

GitLab CI/CD
1

hi,

First post here, and 3 days fresh into gitlab. I might not use the right terminology, please be understanding :smiley:

In short, I would like to use a value from a file in my repository, as a variable to an image during ci pipeline.
reading the documentation and several bugs/requests, i could not make an understanding how is this achievable.

In details:

  1. /config folder contains all the properties definitions required for running home assistant.
  2. all these files are pushed to my gitlab repository, including a file called .HA_VERSION with contains only the version of the currently running home assistant (e.g. 0.91.2)
  3. i have a .gitlab-ci.yml defining the stages of the pipeline (currently only test) + the 2 jobs… and i would like to test my commits (e.g. hass --check_config) against the current version but also against the latest.
before_script:
  - export VERSION=`cat .HA_VERSION`
  - echo $VERSION

stages:
  - test

test:
  stage: test
  image: homeassistant/amd64-homeassistant:$VERSION
  script:
    - hass --script check_config -c .

testlatest:
  stage: test
  image: homeassistant/amd64-homeassistant
  script:
    - hass --script check_config -c .

reading the documentation:

i understand that i cannot use inside the runner, a variable exported in the execution shell environment.
indeed, if i try to use it, the variable is not expanded.

How to solve this, so that i will not make manual changes to this file each time i upgrade to a newer version?

thank you
-d

2

Solution 1:
You could template the file and render it as part of a pre commit hook. To verify this is being done, you can have your first job in your pipeline do the same thing, and diff to what’s checked in. If they are different, fail the pipeline.

Solution 2:
Use docker in docker to spin up the correct image

Solution 3:
Spin up a kubernetes job using the right image and wait for it to complete.

Solution 4:
Farm out to a separate workflow engine like Argo that has these dynamic capabilities

1 Like
3

thanks @ghostsquad

not yet investigated the solution 1.
I assume solution 2 dind supposed to be similar with solution 3.

I thought I already have something in the lines of solution 3, my runner is on a minikube kubernetes cluster. unless i am missing something, this does not work.
For example, the job starts and shows this message:

Using Kubernetes executor with image homeassistant/amd64-homeassistant:$IMAGE_VERSION

Then keeps repeating the below message:

Waiting for pod gitlab-managed-apps/runner-xxx to be running. status is Pending

I assume this is happening because the variable is not expanded so it’s not able to download the image…

Will also check the argo option…

thanks again

4

Please upvote https://gitlab.com/gitlab-org/gitlab/issues/34202

1 Like
5

I also wanted to use a version string in a .gitlab-ci.yml file for appending it to a Docker image name. I did it like this:

build:
  stage: build_images
  script:
    - API_VERSION=v$(grep -E -o "(version = )(.*)" pyproject.toml | cut -d\" -f2)
    - echo $API_VERSION
    # Build and push new images for staging.
    - docker pull $API_STAGING:latest
    - docker build --cache-from $API_STAGING:latest >-
        -t $API_STAGING:latest >-
        -t $API_STAGING:$CI_COMMIT_SHORT_SHA >-
        -t $API_STAGING:$API_VERSION >-
        -f dockerfiles/Dockerfile.staging .
    - docker push $API_STAGING
  tags:
    - build

The key line here is API_VERSION=v$(grep -E -o "(version = )(.*)" pyproject.toml | cut -d\" -f2) .

Explanation: the string I’m trying to read in pyproject.toml is something like version = "0.17.1" , and the result I wanted was v0.17.1

  • v is just a string I want to prepend to my version number
  • -E ( --extended-regexp ) invokes grep as egrep ; allows use of special regexp characters
  • -o ( --only-matching ) doesn’t make a difference in my use-case, but might be helpful in other cases (I’m not sure)
  • (version = ) and (.*) : the two capture groups; the latter one captures anything after the space after the equal sign
  • Running just $ grep -E -o "(version = )(.*)" pyproject.toml will result in version = "0.1.0" , so I’m not using the capture groups; instead, I’m using cut
  • cut “cut[s] out selected portions of each line of a file”
  • -d\" sets the delimiter to a double-quote instead of the default (tab)
  • -f2 specifies the fields to return; a value of 1 would return everything before the first quote, i.e., version = , so 2 returns everything before the second quote and after the first, and 3 returns nothing in this example since there is no third double-quote-separated field
  • echo $API_VERSION just to see that it’s working
1 Like
6

One solution is to generate on the fly a new gitlab ci pipeline form a template that is stored in the git repository and trigger it as child pipeline:

Build:
  script:
    - ./gradlew --build-cache -x test build
    - export VERSION_TAG="$(./gradlew printVersion -x test --console plain --quiet)"
    - echo "VERSION_TAG=${VERSION_TAG}" > version.env
  artifacts:
    reports:
      dotenv: version.env
    when: always
    expire_in: 1 week

Test Bootstrap:
  needs:
    - Build
  script:
    - <.gitlab/templates/test.gitlab-ci.yml sed -e "s/%VERSION_TAG%/${VERSION_TAG}/" >.gitlab/ci/test.gitlab-ci.yml
  artifacts:
    paths:
      - .gitlab/ci/test.gitlab-ci.yml
    expire_in: 30 days

Test:
  needs:
    - Test Bootstrap
  trigger:
    include:
      - artifact: .gitlab/ci/kind.gitlab-ci.yml
        job: Test Bootstrap
    strategy: depend