Repository/files API and private_token parameters

TLDR: I’m still confirming a couple of things, but my initial investigation leaves me raising eyebrows at the use of a query parameter with a sensitive secret. The lack of Basic authn header support in the GET repository/files API seems like an oversight needing some immediate attention.

Is there a Basic Auth format of username and password, where the password is the API Token, that works? Or is this just not supported?

Update: it seems this is a legacy token vs. current state. I generated a new read_api token in my account with privileges to read the repository in question BASIC auth works with and without a username. This suggests that an AWS::CloudFormation:Authentication resource could solve the problem!

Update: it’s not the legacy token format, so much as the legacy token leader string that is the issue with Basic auth. The legacy token in question uses nyUS. Tokens today, use glpat. I changed the leader and Basic auth began working. Now, off to find the documentation in the CHANGELOGs (I know I saw something about PAT token changes… :thinking:

Update: found the discussion in the Version History of Personal access token prefix docs.


I’m refactoring some work in pipelines that leverage AWS CloudFormation Init scripts to download artifacts from a specific GitLab repository, deposit them in a specific locations, then populate the “mustache” keys with parameter overrides.

"files": {
  "/opt/service/conf/configuration.xml": {
    "source": {"Fn::Sub": "https://gitlab.example.com/api/v4/projects/12345678/repository/files/configuration.xml/raw?ref=${configRef}&private_token=${api_token}"},
    "owner": "root",
    "mode": "000755",
    "context": {
      "submEndpoint": {"Fn::FindInMap": [{"Ref": "environment"}, "subm", "endpoint"] },
      ...
      ...

This puzzled me. “Wait, I pass the private_token ON THE query string?” Now, I’m not a security professional, but that can’t be right.

I started my investigation beginning with the CloudFormation cfn-init.log on AWS. And my concern was realized: the token in clear text in the log. The first concern is that a sufficiently privileged token could be dangerous.

Recalling that parameters can be masked in CFT parameters with NoEcho: true was the first route to correction. I thought to myself, “Surely, the original developers knew of this property.” And indeed they did. Many of the passwords for databases and the like have NoEcho: true set, save api_token. Thinking, “Then why, on all that is holy, was this omitted here?” Forgetfulness? Testing and forgot to clean it up? Or something more difficult?"

I’m in the process of doing some of that investigation, i.e., did NoEcho here not work as expected in the original effort? The developers of that effort are no longer with the organization, and I suspect they wouldn’t recall if asked.

I’ve tried a couple of alternatives in Postman with the API that initially seemed promising, but could not be implemented in the CFT. For example, OAuth2 with a Bearer token header in Postman works, but the AWS::CloudFormation::Authentication object does not support an “OAuth2” Authorization header. AWS CloudFormation only supports Basic and AWS Authentication. So I tried using Basic auth in Postman (with various values for username and the token as password). No luck.

In conclusion, the support of Basic auth seems like an oversight here (OP note: this conclusion may be wrong. See TLDR: update above). The use of and continued support for a private_token query parameter seems…dangerous. My guess is a properly set CI/CD variable with “masked” is not an issue. But when the value is passed to tools like the AWS CLI, the masking is MAY be lost. Or when the value is encrypted in a “vault” and passed along following decryption the use of a query parameter must be reviewed for immediate deprecation.

You can use multiple ways how to authenticate with GitLab API using tokens. Pasting it as parameter is just one of them.

If you are running your self-hosted version it also depends which version you run.