GitLab CI Vault Integration does not work as expected

GitLab CI/CD
1

Problem

Today I tried the GitLab CI Vault integration to replace manual commands as shown below. The gitlab-integration job fails. The manual jobs succeeds with the correct data shown. I get a permissio denied error, but the same data is requested with commands which work with the JWT as shown below. Do I need to add some information for the Vault Integration or is my .gitlab-ci.yml faulty?

Thanks for any help!

Error Message from gitlab-integration job

Resolving secrets
Resolving secret "cert"...
Using "vault" secret resolver...
ERROR: Job failed (system failure): resolving secrets: reading secret: reading from Vault: api error: status code 403: 1 error occurred:
	* permission denied

Output from working manual job

$ export VAULT_ADDR=https://vault.+++.de:443
$ export VAULT_TOKEN="$(vault write -field=token auth/jwt/login jwt=$CI_JOB_JWT)"
$ vault kv get certificates/+++.de
====== Metadata ======
Key              Value
---              -----
created_time     2020-12-06T19:11:21.705026445Z
deletion_time    n/a
destroyed        false
version          1
======= Data =======
Key            Value
---            -----
certificate    -----BEGIN CERTIFICATE-----
+++
-----END CERTIFICATE-----
key            -----BEGIN RSA PRIVATE KEY-----
+++
-----END RSA PRIVATE KEY-----

Example

gitlab-integration: # Does not work
  stage: build
  variables:
    VAULT_SERVER_URL: https://vault.+++.de:443
# With or without these variables, it does not change the output
#    VAULT_AUTH_PATH: jwt 
#    VAULT_AUTH_ROLE: gitlab-ci
  secrets:
    cert:
        vault: certificates/+++.de
  image: registry.+++.de/+++/images/cpi
  script:
    - cat $cert

manual:  # Works without problems
  stage: build
  image: registry.###.de/+++/images/cpi # Image with Vault installed
  script:
    - export VAULT_ADDR=https://vault.+++.de:443
    - export VAULT_TOKEN="$(vault write -field=token auth/jwt/login jwt=$CI_JOB_JWT)"
    - vault kv get certificates/+++.de

For privacy reasons, some values are removed and replaced with +++.

2

Hi, I’m also facing a similar problem, I’m always getting permission denied message and I can’t figure out why.
This is my policy:

vault policy read redis-test
path "redis/data/redis/test/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
  }

And role:

vault read auth/jwt/role/redis-test
Key                        Value
---                        -----
allowed_redirect_uris      <nil>
bound_audiences            <nil>
bound_claims               map[project_id:487]
bound_claims_type          glob
bound_subject              n/a
claim_mappings             <nil>
clock_skew_leeway          0
expiration_leeway          0
groups_claim               n/a
max_age                    0
not_before_leeway          0
oidc_scopes                <nil>
policies                   [redis-test]
role_type                  jwt
token_bound_cidrs          []
token_explicit_max_ttl     1m
token_max_ttl              0s
token_no_default_policy    false
token_num_uses             0
token_period               0s
token_policies             [redis-test]
token_ttl                  0s
token_type                 default
user_claim                 user_email
verbose_oidc_logging       false

Vault server log:

{
  "time": "2021-05-10T12:55:47.054394541Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:43b40851925cd687b062781bdc1c43381653c6757afc00f87a3c504da203b9c3",
    "accessor": "hmac-sha256:6426fec8dc45ea95d2643639d81a3c39c83ff5bcc9a427ae78ef85116e9a55cd",
    "display_name": "jwt-carlos.acedo@swisssign.com",
    "policies": [
      "default",
      "redis-test"
    ],
    "token_policies": [
      "default",
      "redis-test"
    ],
    "metadata": {
      "role": "redis-test"
    },
    "entity_id": "df6578c7-fe68-c8fc-7204-c3a288f1719b",
    "token_type": "service",
    "token_ttl": 60,
    "token_issue_time": "2021-05-10T12:55:47Z"
  },
  "request": {
    "id": "92cd43c7-1f8d-1b47-33c5-29b0bbafa5f9",
    "operation": "read",
    "mount_type": "kv",
    "client_token": "hmac-sha256:43b40851925cd687b062781bdc1c43381653c6757afc00f87a3c504da203b9c3",
    "client_token_accessor": "hmac-sha256:6426fec8dc45ea95d2643639d81a3c39c83ff5bcc9a427ae78ef85116e9a55cd",
    "namespace": {
      "id": "root"
    },
    "path": "redis/data/redis/test",
    "remote_address": "192.10.10.13"
  },
  "response": {
    "mount_type": "kv",
    "data": {
      "error": "hmac-sha256:dd5bd5e670898cfa44985d52ba34ecb55e716c78ed1e2b62e4dd884f087b7778"
    }
  },
  "error": "1 error occurred:\n\t* permission denied\n\n"
}

CI/CD env variables:

VAULT_AUTH_ROLE: redis-test
VAULT_AUTH_PATH: jwt
VAULT_SERVER_URL: http://vault:8200/

Job:

deploy_test:
  secrets:
    REDIS_PASSWORD:
      vault:
        engine:
          name: kv-v2
          path: redis
        path: redis/test
        field: password

Path looks correct to me, not sure why I keep getting permission denied…

3

I also have exactly the same issue that the 2 previous users mentioned. Any advice?

4

For future reference my issue was that Gitlab does not support kv1

5

So how did you solve it?

I upgraded to kv-v2 and added “data” prefix to my policy, but it still does not work.

6

I saw your comment in the other issue on the Secret File Variable. I wasn’t able to solve it at that time and since you have the same error still I assume that Gitlab still didn’t fix the issue. The only other option is to contact Vault manually via script.