OpenTofu - Unsupported state file format: can not be read without an encryption configuration

GitLab CI/CD
1

Problem to solve

I got this error, Error refreshing state: Unsupported state file format: This state file is encrypted and can not be read without an encryption configuration, when following the documents from GitLab and OpenTofu (https://opentofu.org/docs/language/state/encryption/).

I tried to deploy locally and all .tf files work just fine.
All tfvars are supplied through GitLab CI/CD Variables.

I only observe this error message in test and validate stages, for now.
Other stages, like fmt and graph completed without any issue.

Steps to reproduce

  1. Enable encryption and backend http.
  2. Trigger GitLab CI first to initialize environment and Terraform state.
  3. Trigger GitLab CI again to deploy changes. <= ERROR

Configuration

.gitlab-ci.yml:

include:
  - component: gitlab.com/components/opentofu/job-templates@main

fmt:
  stage: validate
  extends: [.opentofu:fmt]

validate:
  stage: validate
  environment: stg
  extends: [.opentofu:validate]

graph:
  stage: validate
  environment: stg
  extends: [.opentofu:graph]

test:
  stage: test
  environment: stg
  extends: [.opentofu:test]

backend block:

terraform {
  backend "http" {}
}

encryption block:

terraform {
  encryption {
    key_provider "pbkdf2" "default" {
      passphrase = var.passphrase
    }

    method "aes_gcm" "default" {
      keys = key_provider.pbkdf2.default
    }

    state {
      method = method.aes_gcm.default
      enforced = true
    }

    plan {
      method = method.aes_gcm.default
      enforced = true
    }
  }
}

Versions

  • GitLab.com SaaS: GitLab Enterprise Edition 17.9.0-pre dcf7ed9f83d
  • GitLab OpenTofu CI/CD component: 0.50.0
  • OpenTofu: v1.9.0

Helpful resources

After some search, I found backend is explicit disabled in test and validate. Ref: gitlab-tofu.sh L416-425

And, I tried to clear cache and it works, but only works for the next run.
If there is no cache, both test and validate work just fine.
However, as soon as .terraform/terraform.tfstate exists, they both fail.

I also did a reset by:

  • Clear runner caches.
  • Stop and delete all environments.
  • Remove all Terraform states.

Which didn’t help at all.

2

I am running into a really similar problem. It worked initially but when I started importing things from my infrastructure locally it stopped building in the cloud with this error.

I wonder if my local environment doesn’t have the same encryption settings as the gitlab runner and that’s why the state is reading as unencrypted in the CI/CD runner.

Did you re-initialize your local environment with the same encryption keys? Or are you even using a local environment?

3

OK, I found out some things.

About 2 months ago auto_encryption was introduced: State and plan encryption support (#83) · Issues · GitLab components / OpenTofu · GitLab

I am using it, but I didn’t have my local environment setup to use encryption so I think when I imported my infrastructure it created an unencrypted state file. I added
auto_encryption_enable_migration_from_unencrypted: true to my .gitlab-ci.yml

.gitlab-ci.yml:

include:
  - component: gitlab.com/components/opentofu/full-pipeline@0.50.0
    inputs:
      # The version must currently be specified explicitly as an input,
      # to find the correctly associated images. # This can be removed
      # once https://gitlab.com/gitlab-org/gitlab/-/issues/438275 is solved.
      version: 0.50.0 # component version
      opentofu_version: 1.9.0
      auto_encryption: true
      auto_encryption_passphrase: $PASSPHRASE
      auto_encryption_enable_migration_from_unencrypted: true


stages: [validate, test, build, deploy, cleanup]

plan:
  artifacts:
    access: 'developer'

That seems to have solved things so far, but I won’t remove that line until I have a successful apply working and the state actually encrypted again.

I set up my TF_ENCRYPTION environment variable the same way it’s setup here and it is working with my encrypted state.

4

There are some amazing folks figured out a workaround and are working on a fix to the template.

And the workaround does work.

1 Like