Gitlab Changelog API not able to generate changelog via CI/CD pipeline

Krupis · December 18, 2023, 8:46am

I am trying to automatically generate Changelog and perform release using CI/CD pipeline. I am following the guide: Tutorial: Automate releases and release notes with GitLab

My .gitlab-ci.yml script:

# List of stages for jobs, and their order of execution
stages:
  - prepare
  - release


prepare_job:
  stage: prepare
  image: alpine:latest
  rules:
  - if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
  script:
    - apk add curl jq
    - 'curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG" | jq -r .notes > release_notes.md'
    - cat release_notes.md
  artifacts:
    paths:
    - release_notes.md

release_job:
  stage: release
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  needs:
    - job: prepare_job
      artifacts: true
  rules:
  - if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
  script:
    - echo "Creating release"
  release:
    name: 'Release $CI_COMMIT_TAG'
    description: release_notes.md
    tag_name: '$CI_COMMIT_TAG'
    ref: '$CI_COMMIT_SHA'
    assets:
      links:
        - name: 'Container Image $CI_COMMIT_TAG'
          url: "https://$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA"

Both jobs ran succesfully but it generated empty changelog (“null”).

Output of prepare job:

Running with gitlab-runner 16.6.1 (f5da3c5a)
  on Runner for changelog generator QxsfarD_, system ID: s_1de69ef528ed
Preparing the "docker" executor
Using Docker executor with image alpine:latest ...
Pulling docker image alpine:latest ...
Using docker image sha256:f8c20f8bbcb684055b4fea470fdd169c86e87786940b3262335b12ec3adef418 for alpine:latest with digest alpine@sha256:51b67269f354137895d43f3b3d810bfacd3945438e94dc5ac55fdac340352f48 ...
Preparing environment
00:01
Running on runner-qxsfard-project-121-concurrent-0 via ems-gitlab...
Getting source from Git repository
00:01
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/embeded-programmers/gitchangelog-test/.git/
Checking out 74047d7e as detached HEAD (ref is 2.2.0)...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:04
Using docker image sha256:f8c20f8bbcb684055b4fea470fdd169c86e87786940b3262335b12ec3adef418 for alpine:latest with digest alpine@sha256:51b67269f354137895d43f3b3d810bfacd3945438e94dc5ac55fdac340352f48 ...
$ apk add curl jq
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
(1/10) Installing ca-certificates (20230506-r0)
(2/10) Installing brotli-libs (1.1.0-r1)
(3/10) Installing c-ares (1.22.1-r0)
(4/10) Installing libunistring (1.1-r2)
(5/10) Installing libidn2 (2.3.4-r4)
(6/10) Installing nghttp2-libs (1.58.0-r0)
(7/10) Installing libcurl (8.5.0-r0)
(8/10) Installing curl (8.5.0-r0)
(9/10) Installing oniguruma (6.9.9-r0)
(10/10) Installing jq (1.7.1-r0)
Executing busybox-1.36.1-r15.trigger
Executing ca-certificates-20230506-r0.trigger
OK: 13 MiB in 25 packages
$ curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG" | jq -r .notes > release_notes.md
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    35  100    35    0     0    298      0 --:--:-- --:--:-- --:--:--   301
$ cat release_notes.md
null
Uploading artifacts for successful job
00:02
Uploading artifacts...
release_notes.md: found 1 matching artifact files and directories 
Uploading artifacts as "archive" to coordinator... 201 Created  id=479 responseStatus=201 Created token=64_Nzwyd
Cleaning up project directory and file based variables

Output of release job:

Running with gitlab-runner 16.6.1 (f5da3c5a)
  on Runner for changelog generator QxsfarD_, system ID: s_1de69ef528ed
Preparing the "docker" executor
00:04
Using Docker executor with image registry.gitlab.com/gitlab-org/release-cli:latest ...
Pulling docker image registry.gitlab.com/gitlab-org/release-cli:latest ...
Using docker image sha256:a187ed282417311621b2295e2655ab2c2c80fc55a1f9d2a734ec0cf39e66afed for registry.gitlab.com/gitlab-org/release-cli:latest with digest registry.gitlab.com/gitlab-org/release-cli@sha256:5a71acbadc47c1971100f5246b09f88ba09e84ebe7769e425475dce85245a2bf ...
Preparing environment
00:01
Running on runner-qxsfard-project-121-concurrent-0 via ems-gitlab...
Getting source from Git repository
00:01
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/embeded-programmers/gitchangelog-test/.git/
Checking out 74047d7e as detached HEAD (ref is 2.2.0)...
Skipping Git submodules setup
Downloading artifacts
00:02
Downloading artifacts for prepare_job (479)...
Downloading artifacts from coordinator... ok        host=xxx-gitlab.xxxxxxx.lt id=479 responseStatus=200 OK token=64_t87k-
Executing "step_script" stage of the job script
00:01
Using docker image sha256:a187ed282417311621b2295e2655ab2c2c80fc55a1f9d2a734ec0cf39e66afed for registry.gitlab.com/gitlab-org/release-cli:latest with digest registry.gitlab.com/gitlab-org/release-cli@sha256:5a71acbadc47c1971100f5246b09f88ba09e84ebe7769e425475dce85245a2bf ...
$ echo "Creating release"
Creating release
Executing "step_release" stage of the job script
00:01
$ release-cli create --name "Release 2.2.0" --description "release_notes.md" --tag-name "2.2.0" --ref "74047d7e7a42c1aa90d84a59557507a3e117f607" --assets-link "{\"url\":\"https:///2-2-0:74047d7e7a42c1aa90d84a59557507a3e117f607\",\"name\":\"Container Image 2.2.0\"}"
time="2023-12-18T08:29:11Z" level=info msg="Creating Release..." cli=release-cli command=create name="Release 2.2.0" project-id=121 ref=74047d7e7a42c1aa90d84a59557507a3e117f607 server-url="https://xxx-gitlab.xxxxxxx.lt.lt" tag-message= tag-name=2.2.0 version=0.16.0
Tag: 2.2.0
Name: Release 2.2.0
Description: null
Created At: 2023-12-18 08:29:12.131 +0000 UTC
Released At: 2023-12-18 08:29:12.131 +0000 UTC
Asset::Link::Name: Container Image 2.2.0
Asset::Link::URL: https:///2-2-0:74047d7e7a42c1aa90d84a59557507a3e117f607
See all available releases here: https://xxx-gitlab.xxxxxxx.lt.lt/embeded-programmers/gitchangelog-test/-/releases
time="2023-12-18T08:29:12Z" level=info msg="release created successfully!" cli=release-cli command=create name="Release 2.2.0" project-id=121 ref=74047d7e7a42c1aa90d84a59557507a3e117f607 server-url="https://xxx-gitlab.xxxxxxx.lt.lt" tag-message= tag-name=2.2.0 version=0.16.0
Cleaning up project directory and file based variables
00:01
Job succeeded

I am not too sure why it was not able to generate the changelog. If I simply type the URL to the browser:

https://xxx-gitlab.xxxxxxx.lt/api/v4/projects/121/repository/changelog?version=2.2.0

(Which is what pretty much the above job supposed to do), I am getting the changelog:

I have even tried manual curl command as shown below:

PS C:\Users\petrikas.lu\Desktop\WORK\gitchangelog-test\my_project> curl --header "PRIVATE-TOKEN:xxxxxxxxxxxxxxxxxxx-H" "https://xxx-gitlab.xxxxxxx.lt/api/v4/projects/121/repository/changelog?version=2.3.0"
{"notes":"## 2.3.0 (2023-12-18)\n\n### added (2 changes)\n\n- [another new feature for 2.3.0](embeded-programmers/gitchangelog-test@e78e3550505b7eeabf5483c7613ff92c9ad3bd5c)\n- [new feature for 2.3.0](embeded-programmers/gitchangelog-test@5b0b425b92c8cea49b34ad3fbd8f9a8572e47d6e)\n"}

And as you can see from above that seems to return some kind of changelog.

I would appreciate if someone could help me understand why the CI/CD job failed to generate the changelog. Thanks in advance.

dnsmichi · December 20, 2023, 9:36pm

Seems that the curl request does not succeed. I’d suggest splitting up the commands to debug further, i.e. adding a new line just with curl, but removed | jq block.

    - curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG"

    - 'curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG" | jq -r .notes > release_notes.md'

I suspect either a permission issue with the API token, or a different return format without the notes key. Maybe the surrounding single quotes also have an impact.

Krupis · December 21, 2023, 12:30pm

Thanks for response. I believe the issue was due to the following setting:

After I unchecked Protect variable I was able to get the changelog via CI/CD pipeline but I am not certain why.

Perhaps because the gitlab-runner is not running the jobs on the protected branch (main), instead it is running on its own local branch?

dnsmichi · December 21, 2023, 5:09pm

Glad you could figure it out yourself.

If the changelog script is executed on a “normal” unprotected branch, for example feature-backend-1234, the protected variable will be empty. It will only be filled with its actual value on a protected branch, which by default is only the default branch (main or $CI_DEFAULT_BRANCH if you need to use a variable in CI/CD rules).

So when the release job is running in a MR, or release branch which is not protected, the variable is empty and as such, the curl call will not work.

Krupis · December 21, 2023, 7:47pm

Thanks for your clarifications. I have been further investigating this but I am very confused.

I am initiating a CI/CD pipeline using main branch that is protected :

and trying to execute this pipeline:

# List of stages for jobs, and their order of execution
stages:
  - prepare
  - release


prepare_job:
  stage: prepare
  image: alpine:latest
  rules:
  - if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
  script:
    - apk add curl jq
    - 'curl -H "PRIVATE-TOKEN: $CI_API_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/repository/changelog?version=$CI_COMMIT_TAG" | jq -r .notes > release_notes.md'
    - cat release_notes.md
  after_script:
    - touch JOB_ID.txt
    - echo GE_JOB_ID=$CI_JOB_ID >> JOB_ID.txt
    - cat JOB_ID.txt
  artifacts:
    paths:
      - release_notes.md
    reports:
      # To ensure we've access to this file in the next stage
      dotenv: JOB_ID.txt

release_job:
  stage: release
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  needs:
    - job: prepare_job
      artifacts: true
  rules:
  - if: '$CI_COMMIT_TAG =~ /^v?\d+\.\d+\.\d+$/'
  script:
    - echo "Creating release"
  release:
    name: 'Release $CI_COMMIT_TAG'
    description: release_notes.md
    tag_name: '$CI_COMMIT_TAG'
    ref: '$CI_COMMIT_SHA'
    assets:
      links:
        - name: 'Container Image $CI_COMMIT_TAG'
          url: "https://$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:$CI_COMMIT_SHA"
        - name: 'Changelog'
          url: '$CI_PROJECT_URL/-/jobs/${GE_JOB_ID}/artifacts/file/release_notes.md'

When this is checked :

It will generate null changelog:

However, Disabling the Protect Variable setting and running the same pipeline again:

As you can see it is able to generate changelog. Perhaps this is worth investigating further? Is there any other reasons why it would fail? I would expect this to work since I am on my main protected branch.

dnsmichi · December 21, 2023, 9:19pm

The release job is only triggered by a Git tag. My guess is that there is no relation to a protected branch here. But I am on my ipad, cannot test this now.

Krupis · December 22, 2023, 5:37am

Well the way I trigger the pipeline currently is very simple.

I am on my main branch and I do the following:

Make some changes in the code and commit them :

git add .
git commit -m "I made some changes"; --trailer "Changelog: changed"
git push

Create a tag

git tag 3.9.0

Push the tag

git push origin --tags

The above will create a tag and then I will push it to trigger the pipeline. Do I need to add some additional information in the CI/CD to ensure it works while the protect variable setting is checked?

From what I understand I can trigger the pipeline from any branch. But the changelog generation job should only go through and generate a changelog when pipeline is initiated from the main branch (Which is what I am doing) if the protect variable is checked but that is not the case.

fleXible · February 7, 2024, 4:26pm

I think you have to make your tags protected for the CI/CD protected variable to be usable in that case.
Check and configure Settings -> Repository -> Protected tags

tdlukas · May 16, 2025, 8:01am

Hi, I wonder where do you find this setting to protect and unprotect CI_API_TOKEN?

Topic		Replies	Views
Generate changelog in branch before creating a tag GitLab CI/CD api	3	2991	April 17, 2024
Generating changelog with gitlab API on a repo that implements git flow strategy Infrastructure as Code & Cloud Native api	0	497	August 25, 2022
Get changelog via API GitLab CI/CD api , ci	1	3466	May 24, 2022
Gitlab changelogs API error: Failed to generate the changelog: denied by custom hooks How to Use GitLab api , webhooks , pipelines	0	2997	January 26, 2023
Changelog api raised error when don't mentioning the "from" and "to" How to Use GitLab api , ci , pipelines	4	915	February 29, 2024

Gitlab Changelog API not able to generate changelog via CI/CD pipeline

Related topics