Terraforming with Gitlab CI locking and naming issues

GitLab CI/CD
,
1

Hi, I’m using GitLab Enterprise Edition 14.0.12-ee and I’m trying to set a tf plan pipeline across several stacks with shared dynamodb_table state lock on AWS. It looks something like:

image:
  name: hashicorp/terraform:latest
  entrypoint: [""]

stages:
  - plan

include:
  - local: terraform.gitlab-ci.yml
  - local: stacks/cen/01-foo/.gitlab-ci.yml
  - local: stacks/cen/02-bar/.gitlab-ci.yml

terraform.gitlab-ci.yml looks like:

.terraform:plan: &terraform_plan
  stage: plan
  before_script:
    - cd "${TF_STACK}"
    - terraform fmt
    - terraform init
    - terraform validate
  script:
    - terraform plan
  only:
    changes:
      - "${TF_STACK}/*"

stacks/cen/{01-foo,02-bar}.gitlab-ci.yml are intended to be as minimal as possible and look very similar except for TF_STACK, something like:

variables:
  TF_STACK: stacks/cen/02-bar

plan:${TF_STACK}:
  extends: .terraform:plan
  needs: []

  1. Problem is that if the stages run at the same time, one will inevitably fail with a state lock issue! Terraform plan should not lock state files by default since it no longer writes changes to state · Issue #28130 · hashicorp/terraform · GitHub How do I make the plan in each stack run sequentially? Or better still make terraform ignore the lock?

  2. It’s hard to know which plan is running, I try to name the stage with plan:${TF_STACK} but that doesn’t work. Is there another way?

2

Hi @hendry

starting with the quick one - job names can’t have variables. You need to name your jobs with static names.

as far as the job order goes, GitLab by default does not guarantee job order execution in a single stage. If you need jobs to be executed in a specific order you can either use different stages or needs.
If you have many stacks or they have dynamic lifetime you can also look at Dynamic Child Pipelines to generate your jobs instead of static includes and files.

Another option is to use terraform plan -lock=false, but there is a warning associated

Don't hold a state lock during the operation. This is
dangerous if others might concurrently run commands
against the same workspace.

And another option is to use Terraform Workspaces with named states, basically having a state per stack.

1 Like
3

Thanks the Dynamic Child Pipelines feature looks like what I want as echoed by Reddit! https://www.reddit.com/r/gitlab/comments/ww9jhq/how_do_i_name_included_ci_stage_by_variable/ilk2mft/

But why is it dangerous? It’s just reading and comparing the state. I’m not using the plan.