CI_JOB_TOKEN access denied despite having token access configured

Hi,

I am using the CI_JOB_TOKEN with bazel to clone a second git repository within the same gitlab group. The basic setup for this to work is

git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/".insteadOf "git@gitlab.com:"

which rewirtes all ssh urls of gitlab to https urls with basic authentication using the job token.

Up until a few days ago this worked without problems. I am aware, that with the 16.0 update the project (B), which is accessed needs to define a token access for the project which wants to access (A) it via the job token.

So far I’ve added the A to the token access list of B. I still get the access denied error, when running the CI job:

Error in fail: error running 'git fetch origin refs/heads/*:refs/remotes/origin/* refs/tags/*:refs/tags/*' while working with @hugo_geekdoc_theme:
remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://gitlab.com/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied
fatal: Authentication failed for 'https://gitlab.com/exb/documentation/hugo-geekdoc-exb.git/'

I am running this on Gitlab Saas with custom runners.

Running with gitlab-runner 15.11.0 (v15.11.0)

I did not find it in the changelog but is it necessary for the runners to also be on version 16.0 for this to work? I am asking because I have no access to the runner setup and would like to understand if this is the cause.

Same error here, all of a sudden my git clones fail with the same HTTP Basic: Access denied. despite having the projects in the allowlist.

I’ve been experiencing the exact same problem as @bcmedeiros since this morning. All my CI jobs are crashing, reporting an authentication error. The Group Access Token is still valid, and I can still use it to carry out the same operations from my machine.

Edit : same issue with both a newly created group access token and a personal access token.

Hi,

Same error since this night during scheduled pipeline.
Do you know why ?

No replies from Gitlab support yet.

Are you guys using Alpine image and installing latest curl in there?

Try any curl version before 8.2.0 (i.e. 8.1.2). That helped us.

Well, you seems to be onto something. Our jobs are indeed using Alpine Linux. Git package have libcurl=8.2.0-r0 dependency since 19 hours ago.

curl 8.2.0 released yesterday (July 19th), and we can see failures on pipelines which install latest version of it since this morning. No idea why it breaks git actions.

It might be related to this issue: Regression 8.2.0: Basic authentication does not follow with -L · Issue #11486 · curl/curl · GitHub. I am able to reproduce the issue on Arch Linux (curl 8.2.0 is in testing repository), so it’s definitely curl related.

1 Like

I use that image => python:3.12.0b4-alpine3.18

Still an Alpine image that installs curl in version 8.2.0. You’d need to switch to different image, build your own using alpine/curl:8.1.2 as base, or install the curl from source.

Incident link for Gitlab: 2023-07-20: Pipeline using CI_JOB_TOKEN to run git clone are failling with HTTP Basic: Access denied (#16066) · Issues · GitLab.com / GitLab Infrastructure Team / production · GitLab

Seems to be fixed in 8.2.0-r1.