Triggering a specific job of the pipeline via API

I’ve managed to start the job I want, but it fails immediately

Steps I’ve followed:

• Created an access token for the project and placed it in my script.
• curl --insecure -X POST --header “PRIVATE-TOKEN: $PROJECT_ACCESS_TOKEN” “https://gitlab.tld/api/v4/projects/$PROJECT_ID/pipeline?ref=$BRANCH” - this successfully triggers the pipeline as the <token_name> user. All jobs within the pipeline are set to when: manual.
• curl --insecure --header “PRIVATE-TOKEN: $PROJECT_ACCESS_TOKEN” “https://gitlab.tld/api/v4/projects/$PROJECT_ID/pipelines/$PIPELINE_ID/jobs” - this gets the list of jobs that I then use to extract the specific ID of the job.
• curl --insecure -X POST --header “PRIVATE-TOKEN: $DEPLOY_TOKEN” https://gitlab.tld/api/v4/projects/$PROJECT_ID/jobs/$JOB_ID/play - this successfully launches the desired job.

After the job is launched, it almost immediately fails with the following:

Reinitialized existing Git repository in /builds/groupname/projectname/.git/
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'https://gitlab.tld/groupname/projectname.git/'
ERROR: Job failed: exit code 1

• What version are you on? Are you using self-managed or GitLab.com?
  • GitLab Community Edition 13.3.5
  • Runner (Hint: /admin/runners): 13.3.1

Things I’ve tried:

• Generating a token with api only permissions
• Generating a token with api and repo read/write permissions
• Generating a token with all the permissions
• Generating an impersonation token
• Tailing logs on the runner itself while launched as gitlab-runner --debug run - did not prove to be useful
• Retrying the job via GUI - that then changes the “owner” to me and the jobs goes on to succeed.

One more point to make - this is an otherwise fully functional pipeline - all the steps are set to manual and it’s been tested multiple times. This smells like some sort of internal access token user’s permission issue that I cannot pint point.

Thank you in advance

Solved by choosing a different approach.

I’ve created a job that only becomes part of the pipeline when the pipeline itself is triggered via API. Otherwise it’s as if it didn’t exist in the first place. Here’s how I’ve done it:

000-colo-switch:prod:
  stage: deploy
  variables:
    PROJECT: "projectname"
  rules: # whether to include the job in the pipeline
    - if: '$CI_PIPELINE_SOURCE == "trigger"'
      when: always
    - when: never

The job is triggered using a Trigger Token obtained in project settings -> CI CD. The curl command looks as follows:

curl -k -X POST \
     -F token=$TRIGGER_TOKEN \
     -F "ref=$BRANCH" \
     -F "variables[COLO]=$COLO" \
     https://gitlab.tld/api/v4/projects/$PROJECT_ID/trigger/pipeline