GitLab CI Terraform init requires backend configuration via CLI

GitLab CI/CD
,
1

I have my Terraform backend configured in root configuration. When I try to use Terraform from GitLab CI, terraform init without -backend-config fails, whereas from my localhost CLI with the same configuration works.

I am trying to figure out how to pass the value of TF_HTTP_PASSWORD variable to Terraform in GitLab CI, because I would like to use it in terraform_remote_state data source. According to documentation https://www.terraform.io/docs/language/settings/backends/http.html variable TF_HTTP_PASSWORD should work, but in GtLab CI it doesn’t.

My main.tf file:

terraform {
  backend "http" {
    address        = "https://gitlab.com/api/v4/projects/28190009/terraform/state/test"
    lock_address   = "https://gitlab.com/api/v4/projects/28190009/terraform/state/test/lock"
    unlock_address = "https://gitlab.com/api/v4/projects/28190009/terraform/state/test/lock"
    username       = "gitlab"
    # password      = not in configuration
    lock_method    = "POST"
    unlock_method  = "DELETE"
  }
}

Localhost CLI use works without any issue:

export TF_HTTP_PASSWORD="<GitLab Personal Access Token>"
terraform init

GitLab CI use fails:

---
image:
  name: hashicorp/terraform:light
  entrypoint:
    - "/usr/bin/env"
    - "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

variables:
  TF_IN_AUTOMATION: "true"
  TF_HTTP_PASSWORD: ${CI_JOB_TOKEN}

before_script:
  - terraform --version
  - terraform init -reconfigure -input=false

stages:
  - apply

tf-apply:
  stage: apply
  script:
    - terraform apply -auto-approve -input=false

2021/07/16 09:33:49 [DEBUG] GET https://gitlab.com/api/v4/projects/28190009/terraform/state/test** **Error refreshing state: HTTP remote state endpoint requires auth

If I replace

  • - terraform init

line with

  • - terraform init -reconfigure -backend-config="address=https://gitlab.com/api/v4/projects/28190009/terraform/state/test" -backend-config="lock_address=https://gitlab.com/api/v4/projects/28190009/terraform/state/test/lock" -backend-config="unlock_address=https://gitlab.com/api/v4/projects/28190009/terraform/state/test/lock" -backend-config="username=gitlab-ci-token" -backend-config="password=$CI_JOB_TOKEN" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" -input=false

it works in GitLab CI!

The project is here: https://gitlab.com/xebis/gitlab-terraform-test

3

Hi,

GitLab provides a quickstart CI/CD template including a container image, which wraps the required terraform CLI calls accordingly for CI/CD as gitlab-terraform. This is described in the docs: Infrastructure as code with Terraform and GitLab | GitLab

The following CI templates come into use:

I’d recommend using the wrapper script / container with the template, as otherwise you’ll need to pass this manually on your own.

An example how to do it is a community project which deploys k3s and uses the image & CLI commands: .gitlab-ci.yml · main · everyonecancontribute / kubernetes / k3s Demo · GitLab

Cheers,
Michael

1 Like
4

Solved, the problem was that I used username = "gitlab" instead of username = "gitlab-ci-token". With the gitlab-ci-token it works on localhost (because I use GitLab Personal Access Token, and in this case GitLab Terraform backend ignores username) and at CI as well. Then passing the password as CLI argument or environment variable works.

It is mentioned in GitLab Docs: GitLab managed Terraform State - Using a GitLab managed Terraform state backend as a remote data source

@dnsmichi thank you for your fast reply

2 Likes
5

Thanks for sharing the solution, learned something new myself today :sunglasses: