The CI_JOB_TOKEN can be used to upload a new package but not to list or delete packages

Using CI_JOB_TOKEN to look up and delete packages

I have a project where I deploy a package to the package registry. More specifically it’s a Terraform module deployed to the Infrastruction Registry, but that’s not really important as it’s the same package repository underneath as all other packages use I believe and I’m using the packages API endpoint to do this.

So deploying works fine. I’ve basically followed instructions here:

The following project (referenced in the documentation above) was very helpful:

Basically, all it does is user curl in the CI scripts to upload a tgz file containing the Terraform module and it uses the JOB-TOKEN: $CI_JOB_TOKEN header to do that.

However, I need to be able to delete and re-upload the same versions of the modules multiple times. For that, I need to use the API to search for the package of that particular name and version and then use the API again to delete that package before uploading it.

This all works when I use my private token, but if I try to use the same token as I used for uploading the module ($CI_JOB_TOKEN) it doesn’t work. I just get a 404 response with a message saying that the project doesn’t exist. I’ve made absolutely sure that everything is as it’s supposed to be in my CI scripts, so I’m 99% sure that the problem is that the CI_JOB_TOKEN just doesn’t have read and delete API access, while it seems to have write access.

So I ask, is there anything that I can configure to enable read and delete API access when using the CI_JOB_TOKEN to authenticate? It seems weird that the CI of a project is unable to read and delete packages from its own package registry.

Hi, @StFS.

There are security limitations to the CI_JOB_TOKEN.

You might want to create and use a project access token for this purpose.

Thanks. That would require me to define a token in every project that needs to do this (basically all projects) which is something I’d very much like to avoid.

But also, it’s not clear to me from the documentation how that token then makes it into the CI? Would I have to 1) create the token for the project and 2) then put that token (manually) into the CI variables in that project? If the token then expires or is updated, I would have to manually keep it in sync there.

If that’s the case, it’s even less desirable.

But thanks for your suggestion. I guess I have a workaround using this although I would argue that defining a project access token like this, which is a long lived token, is less secure than allowing the CI_JOB_TOKEN access to the API (as the CI token is short lived).

But after discussing this quite a bit at Unable to delete Terraform module from the infrastructure registry via API (#348690) · Issues · GitLab.org / GitLab · GitLab there seems to be some interest in allowing the CI_JOB_TOKEN access to the API (at least to the project owned resources).

I’ll hope for the best but I guess I’ll have to find an alternative for now (such as S3).

@StFS

Would I have to 1) create the token for the project and 2) then put that token (manually) into the CI variables in that project? If the token then expires or is updated, I would have to manually keep it in sync there.

That’s my understanding, but don’t take my word for it - my only experience in this area is as a fellow user.

One way I can think of reducing overhead is to use group CI variables. In this case, using a project access token won’t work, so I would probably go the personal access token route you suggested initially but use a dedicated bot account for it.

defining a project access token like this, which is a long lived token, is less secure than allowing the CI_JOB_TOKEN access to the API (as the CI token is short lived).

Yes, I agree with you there. In a previous job I wrote a small app that allowed us to automatically set variables at instance, group and project levels, as well as rotate the secrets and reset them ad hoc.

Thanks for creating the follow-up issues. You might want to ping the product manager for the package group in Allow the CI_JOB_TOKEN access to the package API (at least on a project level) to DELETE and list (GET) (#349418) · Issues · GitLab.org / GitLab · GitLab for visibility, although I see that David has already commented on it.

1 Like