Gitlab Changelog API not able to generate changelog via CI/CD pipeline

I am trying to automatically generate Changelog and perform release using CI/CD pipeline. I am following the guide: Tutorial: Automate releases and release notes with GitLab

My .gitlab-ci.yml script:

# List of stages for jobs, and their order of execution
stages:
  - prepare
  - release


prepare_job:
  stage: prepare
  image: alpine:latest
  rules:
  - if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
  script:
    - apk add curl jq
    - 'curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG" | jq -r .notes > release_notes.md'
    - cat release_notes.md
  artifacts:
    paths:
    - release_notes.md

release_job:
  stage: release
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  needs:
    - job: prepare_job
      artifacts: true
  rules:
  - if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
  script:
    - echo "Creating release"
  release:
    name: 'Release $CI_COMMIT_TAG'
    description: release_notes.md
    tag_name: '$CI_COMMIT_TAG'
    ref: '$CI_COMMIT_SHA'
    assets:
      links:
        - name: 'Container Image $CI_COMMIT_TAG'
          url: "https://$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA"

Both jobs ran succesfully but it generated empty changelog (“null”).

Output of prepare job:

Running with gitlab-runner 16.6.1 (f5da3c5a)
  on Runner for changelog generator QxsfarD_, system ID: s_1de69ef528ed
Preparing the "docker" executor
Using Docker executor with image alpine:latest ...
Pulling docker image alpine:latest ...
Using docker image sha256:f8c20f8bbcb684055b4fea470fdd169c86e87786940b3262335b12ec3adef418 for alpine:latest with digest alpine@sha256:51b67269f354137895d43f3b3d810bfacd3945438e94dc5ac55fdac340352f48 ...
Preparing environment
00:01
Running on runner-qxsfard-project-121-concurrent-0 via ems-gitlab...
Getting source from Git repository
00:01
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/embeded-programmers/gitchangelog-test/.git/
Checking out 74047d7e as detached HEAD (ref is 2.2.0)...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:04
Using docker image sha256:f8c20f8bbcb684055b4fea470fdd169c86e87786940b3262335b12ec3adef418 for alpine:latest with digest alpine@sha256:51b67269f354137895d43f3b3d810bfacd3945438e94dc5ac55fdac340352f48 ...
$ apk add curl jq
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
(1/10) Installing ca-certificates (20230506-r0)
(2/10) Installing brotli-libs (1.1.0-r1)
(3/10) Installing c-ares (1.22.1-r0)
(4/10) Installing libunistring (1.1-r2)
(5/10) Installing libidn2 (2.3.4-r4)
(6/10) Installing nghttp2-libs (1.58.0-r0)
(7/10) Installing libcurl (8.5.0-r0)
(8/10) Installing curl (8.5.0-r0)
(9/10) Installing oniguruma (6.9.9-r0)
(10/10) Installing jq (1.7.1-r0)
Executing busybox-1.36.1-r15.trigger
Executing ca-certificates-20230506-r0.trigger
OK: 13 MiB in 25 packages
$ curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG" | jq -r .notes > release_notes.md
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    35  100    35    0     0    298      0 --:--:-- --:--:-- --:--:--   301
$ cat release_notes.md
null
Uploading artifacts for successful job
00:02
Uploading artifacts...
release_notes.md: found 1 matching artifact files and directories 
Uploading artifacts as "archive" to coordinator... 201 Created  id=479 responseStatus=201 Created token=64_Nzwyd
Cleaning up project directory and file based variables

Output of release job:

Running with gitlab-runner 16.6.1 (f5da3c5a)
  on Runner for changelog generator QxsfarD_, system ID: s_1de69ef528ed
Preparing the "docker" executor
00:04
Using Docker executor with image registry.gitlab.com/gitlab-org/release-cli:latest ...
Pulling docker image registry.gitlab.com/gitlab-org/release-cli:latest ...
Using docker image sha256:a187ed282417311621b2295e2655ab2c2c80fc55a1f9d2a734ec0cf39e66afed for registry.gitlab.com/gitlab-org/release-cli:latest with digest registry.gitlab.com/gitlab-org/release-cli@sha256:5a71acbadc47c1971100f5246b09f88ba09e84ebe7769e425475dce85245a2bf ...
Preparing environment
00:01
Running on runner-qxsfard-project-121-concurrent-0 via ems-gitlab...
Getting source from Git repository
00:01
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/embeded-programmers/gitchangelog-test/.git/
Checking out 74047d7e as detached HEAD (ref is 2.2.0)...
Skipping Git submodules setup
Downloading artifacts
00:02
Downloading artifacts for prepare_job (479)...
Downloading artifacts from coordinator... ok        host=xxx-gitlab.xxxxxxx.lt id=479 responseStatus=200 OK token=64_t87k-
Executing "step_script" stage of the job script
00:01
Using docker image sha256:a187ed282417311621b2295e2655ab2c2c80fc55a1f9d2a734ec0cf39e66afed for registry.gitlab.com/gitlab-org/release-cli:latest with digest registry.gitlab.com/gitlab-org/release-cli@sha256:5a71acbadc47c1971100f5246b09f88ba09e84ebe7769e425475dce85245a2bf ...
$ echo "Creating release"
Creating release
Executing "step_release" stage of the job script
00:01
$ release-cli create --name "Release 2.2.0" --description "release_notes.md" --tag-name "2.2.0" --ref "74047d7e7a42c1aa90d84a59557507a3e117f607" --assets-link "{\"url\":\"https:///2-2-0:74047d7e7a42c1aa90d84a59557507a3e117f607\",\"name\":\"Container Image 2.2.0\"}"
time="2023-12-18T08:29:11Z" level=info msg="Creating Release..." cli=release-cli command=create name="Release 2.2.0" project-id=121 ref=74047d7e7a42c1aa90d84a59557507a3e117f607 server-url="https://xxx-gitlab.xxxxxxx.lt.lt" tag-message= tag-name=2.2.0 version=0.16.0
Tag: 2.2.0
Name: Release 2.2.0
Description: null
Created At: 2023-12-18 08:29:12.131 +0000 UTC
Released At: 2023-12-18 08:29:12.131 +0000 UTC
Asset::Link::Name: Container Image 2.2.0
Asset::Link::URL: https:///2-2-0:74047d7e7a42c1aa90d84a59557507a3e117f607
See all available releases here: https://xxx-gitlab.xxxxxxx.lt.lt/embeded-programmers/gitchangelog-test/-/releases
time="2023-12-18T08:29:12Z" level=info msg="release created successfully!" cli=release-cli command=create name="Release 2.2.0" project-id=121 ref=74047d7e7a42c1aa90d84a59557507a3e117f607 server-url="https://xxx-gitlab.xxxxxxx.lt.lt" tag-message= tag-name=2.2.0 version=0.16.0
Cleaning up project directory and file based variables
00:01
Job succeeded

I am not too sure why it was not able to generate the changelog. If I simply type the URL to the browser:

https://xxx-gitlab.xxxxxxx.lt/api/v4/projects/121/repository/changelog?version=2.2.0

(Which is what pretty much the above job supposed to do), I am getting the changelog:

I have even tried manual curl command as shown below:

PS C:\Users\petrikas.lu\Desktop\WORK\gitchangelog-test\my_project> curl --header "PRIVATE-TOKEN:xxxxxxxxxxxxxxxxxxx-H" "https://xxx-gitlab.xxxxxxx.lt/api/v4/projects/121/repository/changelog?version=2.3.0"
{"notes":"## 2.3.0 (2023-12-18)\n\n### added (2 changes)\n\n- [another new feature for 2.3.0](embeded-programmers/gitchangelog-test@e78e3550505b7eeabf5483c7613ff92c9ad3bd5c)\n- [new feature for 2.3.0](embeded-programmers/gitchangelog-test@5b0b425b92c8cea49b34ad3fbd8f9a8572e47d6e)\n"}

And as you can see from above that seems to return some kind of changelog.

I would appreciate if someone could help me understand why the CI/CD job failed to generate the changelog. Thanks in advance.

Seems that the curl request does not succeed. I’d suggest splitting up the commands to debug further, i.e. adding a new line just with curl, but removed | jq block.

    - curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG"

    - 'curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG" | jq -r .notes > release_notes.md'

I suspect either a permission issue with the API token, or a different return format without the notes key. Maybe the surrounding single quotes also have an impact.

Thanks for response. I believe the issue was due to the following setting:

After I unchecked Protect variable I was able to get the changelog via CI/CD pipeline but I am not certain why.

Perhaps because the gitlab-runner is not running the jobs on the protected branch (main), instead it is running on its own local branch?

Glad you could figure it out yourself.

If the changelog script is executed on a “normal” unprotected branch, for example feature-backend-1234, the protected variable will be empty. It will only be filled with its actual value on a protected branch, which by default is only the default branch (main or $CI_DEFAULT_BRANCH if you need to use a variable in CI/CD rules).

So when the release job is running in a MR, or release branch which is not protected, the variable is empty and as such, the curl call will not work.

Thanks for your clarifications. I have been further investigating this but I am very confused.

I am initiating a CI/CD pipeline using main branch that is protected :

and trying to execute this pipeline:

# List of stages for jobs, and their order of execution
stages:
  - prepare
  - release


prepare_job:
  stage: prepare
  image: alpine:latest
  rules:
  - if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
  script:
    - apk add curl jq
    - 'curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG" | jq -r .notes > release_notes.md'
    - cat release_notes.md
  after_script:
    - touch JOB_ID.txt
    - echo GE_JOB_ID=$CI_JOB_ID >> JOB_ID.txt
    - cat JOB_ID.txt
  artifacts:
    paths:
      - release_notes.md
    reports:
      # To ensure we've access to this file in the next stage
      dotenv: JOB_ID.txt

release_job:
  stage: release
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  needs:
    - job: prepare_job
      artifacts: true
  rules:
  - if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
  script:
    - echo "Creating release"
  release:
    name: 'Release $CI_COMMIT_TAG'
    description: release_notes.md
    tag_name: '$CI_COMMIT_TAG'
    ref: '$CI_COMMIT_SHA'
    assets:
      links:
        - name: 'Container Image $CI_COMMIT_TAG'
          url: "https://$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA"
        - name: 'Changelog'
          url: '$CI_PROJECT_URL/-/jobs/${GE_JOB_ID}/artifacts/file/release_notes.md'


When this is checked :

It will generate null changelog:

However, Disabling the Protect Variable setting and running the same pipeline again:

As you can see it is able to generate changelog. Perhaps this is worth investigating further? Is there any other reasons why it would fail? I would expect this to work since I am on my main protected branch.

The release job is only triggered by a Git tag. My guess is that there is no relation to a protected branch here. But I am on my ipad, cannot test this now.

Well the way I trigger the pipeline currently is very simple.

I am on my main branch and I do the following:

  1. Make some changes in the code and commit them :

git add .
git commit -m "I made some changes"; --trailer "Changelog: changed"
git push

  1. Create a tag

git tag 3.9.0

  1. Push the tag

git push origin --tags

The above will create a tag and then I will push it to trigger the pipeline. Do I need to add some additional information in the CI/CD to ensure it works while the protect variable setting is checked?

From what I understand I can trigger the pipeline from any branch. But the changelog generation job should only go through and generate a changelog when pipeline is initiated from the main branch (Which is what I am doing) if the protect variable is checked but that is not the case.

I think you have to make your tags protected for the CI/CD protected variable to be usable in that case.
Check and configure Settings -> Repository -> Protected tags