Dynamic Hostname for ssh executor

GitLab CI/CD
,
1

connect to dynamic hostname with SSH executor

Hi folks!
We are currently experimenting with a pipeline that provides test coverage for a complete deployment of a virtual appliance we ship to customers.

The pipeline has the following Jobs:

  • create a new KVM VM
  • run the installer using our custom image
  • run post-installation tasks on the VM
  • run external tests to see if all went well from outside the VM
  • run internal tests from inside the VM

The post-installation tasks and internal tests are run on the newly created VM themselves, using the ssh executor. All works well, as long there’s only only pipeline is running at the same time, with hostname, user and password hardcoded to config.toml. However, we want to provide multiple test instances running at the same time, as multiple people are working in parallel on issues that need this pipeline.

The VMs get hostnames assigned by the first job (naming template foobar-X) and get IP-addresses via DHCP (with dynamic DNS so we can talk to the hostnames).

We tried to use env vars from the pipeline in the config.toml, but those do not get interpolated (as expected, tbh).

Is there a way to do this? Can one dynamically set the SSH login vars for ssh executors?

Versions

  • gitlab: 16.1.1-ee self-hosted
  • runner: 16.1.0

config files / things we tested

Working, but only 1 VM possible (hostname fixed)

# config.toml
[[runners]]
  name = "runner-name"
  url = "https://gitlab.company.domain/"
  token = "nice-try-token"
  executor = "ssh"
  [runners.ssh]
    host = "hardcoded-vm-name.domain"
    user = "loginuser"
    password = "loginpassword"
    disable_strict_host_key_checking = true

# .gitlab-ci.yml
stage-one:
  tags:
     - kvm
  script:
      - .snip.
stage-two:
  tags:
    - ssh
  script:
    - echo "This works"

not working, use hostname from env-vars

# config.toml
[[runners]]
  name = "runner-name
  url = "https://gitlab.company.domain/"
  token = "nice-try-token"
  executor = "ssh"
  [runners.ssh]
    host = "$VARIABLE_HOSTNAME"
    user = "loginuser"  # static username
    password = "$VARIABLE_PASSWORD"  # no priority, could use keys as well
    disable_strict_host_key_checking = true

# .gitlab-ci.yml
stage-one:
  tags:
     - kvm
  script:
      - .snip.
stage-two:
  tags:
    - ssh
  variables:
    VARIABLE_HOSTNAME: "my-hostname"
    VARIABLE_PASSWORD: "my-password"
  script:
    - echo "This ain't working"

Error:

Running with gitlab-runner 16.1.0 (865283c5)
 on runner_name token, system ID: s_systemid
Resolving secrets
Preparing the "ssh" executor
Using SSH executor...
ERROR: Preparation failed: ssh command Connect() error: ssh Dial() error: dial tcp: lookup $VARIABLE_HOSTNAME: no such host