Downloading raw file over HTTPS using gitlab-ci-token recently stopped working

GitLab CI/CD
1

I have a pipeline job that downloads a file from another repo, and this has been working until about two days ago:

    const url = `https://gitlab.com/<namespace>/<repo>/-/raw/<branch>/path/to/file.txt`;
    const headers = { Authorization: `Basic ${btoa(`gitlab-ci-token:${process.env.CI_JOB_TOKEN}`)}` };
    const res = await fetch(url, { headers });
    const contents = await res.text();

Equivalent to curl https://gitlab-ci-token:{$CI_JOB_TOKEN}@gitlab.com/.../-/raw/... (notice I’m fetching from …/-/raw/…), which also seems not to work. Using git clone over HTTPS, which presumably uses the smart HTTP git protocol, still works in the pipeline, but now when I try to fetch file contents using /raw endpoints, I get redirected to my company’s SSO login page.

Did something change recently? I suspect what I’ve been doing isn’t officially supported and a loophole got closed, but I’d welcome any more info. As an alternative, I’ll have my script do a shallow clone of the whole repo to try to get the file’s contents, but if there’s a more efficient way to get it, I’m open to ideas.

2

Same here. I used the same method as yours, but to check for semver collision during merge request. At first, I thought it was just a redirect error as it was failing at:

<html><body>You are being <a href="https://gitlab.com/users/sign_in">redirected</a>.</body></html>

Appending -L to curl doesn’t work either, as it redirects to the sign-in page, instead of the raw file content.

Edit: Tested with get file using Repository API returns:

{"message":"404 Project Not Found"}

In the meantime, I’ve reverted to using git fetch source/target file. I really don’t want to fetch everything though, especially with a large repo.

1 Like