Storing credentials on self hosted instance?

How to Use GitLab Self-managed
,
1

Problem to solve

TL;DR: From where does Git on gitlab runner fetch credentials?

I’m using Terraform in a CI/CD pipeline to manage our infrastructure. We use a self managed GitLab instance (on-prem). In my Terraform code, I’ve got basically this:

module "mod" {
  source = "git::https://gitlab-ci-token@git.mycompany/terraform/mod.git"
}

The “mod.git” repository is private. So I need to supply a username and password (or rather: any username and a PAT - personal access token).

On my notebook, I’ve created a credentials file like this:

echo "https://gitlab-ci-token:secret-password@git.mycompany" >> ~/.git-credentials

This makes Terraform happy, when it uses git to clone “https://git.mycompany/terraform/mod.git”.

I added the echo … >> … line to my .gitlab-ci.yml in a job, but this doesn’t seem to be used:

╷
│ Error: Failed to download module
│ 
│   on main.tf line 4:
│    4: module "mod" {
│ 
│ Could not download module "mod" (main.tf:4) source code from
│ "git::https://gitlab-ci-token@git.mycompany/terraform/mod.git":
│ error downloading
│ 'https://gitlab-ci-token@git.mycompany/terraform/mod.git':
│ /usr/bin/git exited with 128: Cloning into '.terraform/modules/mod'...
│ fatal: could not read Password for
│ 'https://gitlab-ci-token@git.mycompany': No such device or address
│ 
╵

I have to use https access — ssh is disabled and cannot be enabled.

The “username” and password/PAT are supplied in GitLab CI/CD variables. I’m using these variables to fill the ~/.git-credentials file.

Versions

Please select whether options apply, and add the version information.

Versions

2

You would be better to use CI/CD variables: GitLab CI/CD variables | GitLab

3

Could you please expand?

Which variables should I use in what way? Actually, the PAT is already in a variable (“The “username” and password/PAT are supplied in GitLab CI/CD variables.”). But Terraform cannot use environment variables in a module … { source = … } line.

In the Terraform docs, they instruct to use Git Credentials Storage:

If using the HTTP/HTTPS protocol, or any other protocol that uses username/password credentials, configure Git Credentials Storage to select a suitable source of credentials for your environment.

Thanks :slight_smile:

4

In your .gitlab_ci.yml you’ll prob need to be making references to TF_TOKEN variable.

My initial reply probably won’t help much, since those variables are configured in Gitlab itself. You’ll need to be pulling variables/tokens from Terraform, from your .gitlab_ci.yml file, so this is more a Terraform forum question. Unfortunately I don’t see much in Gitlab documentation relating to Terraform.

Google did give me some results like this though: How to Implement GitLab CI/CD Pipeline with Terraform so whether that will help explain more or not I’ve no idea - I don’t use Terraform. But it looks promising.

5

What would I do with $TF_TOKEN? Where should it be stored, so that git from gitlab runner uses it? Doing echo "https://gitlab-ci-token:$TF_TOKEN@git.mycompany" >> ~/.git-credentials didn’t help (see the error message I posted: “could not read Password…”).

I’m aware about How to Implement GitLab CI/CD Pipeline with Terraform and other docs like that. That goes in the wrong direction — I don’t need help in setting up Terraform (or, do I? possible — but not in this forum here :wink:). I need help in setting up Git on GitLab runner, so that it uses the credentials.

Terraform doesn’t pull environment variables. They are, at best, a fallback. And in that use case (module source line), it’s plainly not supported. So this is not a Terraform forum question.

Long story, short: Where to store credentials, so that Git on a self-hosted GitLab (and also GitLab runner) instance makes use of them?

I cannot make the project/repository public. The group, to which it must belong, prevents that.
I cannot use SSH access. SSH is disabled - probably for “security reasons”.
Neither of that can be changed. That’s set in stone.

6

A ~/.netrc file like the following works fine:

machine git.mycompany
login pat
password glpat-…

Another option: Very simple GIT_ASKPASS script which just prints the “password” (or PAT). As simple as this (eg. stored as /tmp/GIT_ASKPASS.sh):

#!/bin/sh

echo glpat-…

And before invoking terraform init and the like, export GIT_ASKPASS variable:

GIT_ASKPASS=/tmp/GIT_ASKPASS.sh
export GIT_ASKPASS

In my Terraform source code, I then have something like this:

module "mod" {
  source = "git::https://git.mycompany/terraform/mod.git"
}

Ie. neither “username” nor password.