Gitlab sha256sum different everytime we download a release

We are currently on gitlab version 13.12.15. Someone reported that every time he downloads the release without changes, the checksums are different.

I tested it and it is changing everytime. Is this normal?

HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘testprojectafterupgradev13.9.5-test_tag_allan.zip’

testprojectafterupgradev13.9.5-t     [   <=>                                                    ]  53.32K   111KB/s    in 0.5s

2022-06-06 18:46:46 (111 KB/s) - ‘testprojectafterupgradev13.9.5-test_tag_allan.zip’ saved [54595]

$ sha256sum testprojectafterupgradev13.9.5-test_tag_allan.zip
632096aef433f1e8287bf70441c2418a26afbe92c787d9a40f2eb1396a3a6b9c  testprojectafterupgradev13.9.5-test_tag_allan.zip


HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘testprojectafterupgradev13.9.5-test_tag_allan.zip’

testprojectafterupgradev13.9.5-tes     [     <=>                                                         ]  53.32K  56.2KB/s    in 0.9s

2022-06-06 18:45:07 (56.2 KB/s) - ‘testprojectafterupgradev13.9.5-test_tag_allan.zip’ saved [54595]

$ sha256sum testprojectafterupgradev13.9.5-test_tag_allan.zip
bcbf52550736321c380267d7f381a334b832ec8a50a295b5a913f6e632a5ac2f  testprojectafterupgradev13.9.5-test_tag_allan.zip

Also, is there an integrity checksum for gitlab repository release on assets that are archived?

Can you share the download command and URL endpoint? It looks like the command is downloading a text file / website content (Length: text/html) instead of the zip file.

Hi @dnsmichi, thank you for looking into this. As mine was taken directly from the terminal, I was not able to save them. However, this was from my colleague who reported the issue:

fqu@dh-res11:~/codes/test/temp/trash$ wget http://xxx.dtl/fqu/test_release/-/archive/v0.0.1/test_release-v0.0.1.tar.gz && sha256sum test_release-v0.0.1.tar.gz
--2022-06-06 17:42:08--  http://xxx.dtl/fqu/test_release/-/archive/v0.0.1/test_release-v0.0.1.tar.gz
Resolving xxx.dtl (xxx.dtl)... 10.202.208.92
Connecting to xxx.dtl (xxx.dtl)|10.202.208.92|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://xxx.dtl/users/sign_in [following]
--2022-06-06 17:42:08--  http://xxx.dtl/users/sign_in
Reusing existing connection to xxx.dtl:80.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘test_release-v0.0.1.tar.gz’

test_release-v0.0.1.tar.gz                [   <=>                                                                  ]  53.32K   111KB/s    in 0.5s

2022-06-06 17:42:09 (111 KB/s) - ‘test_release-v0.0.1.tar.gz’ saved [54595]

f07b66a46f6b61a3fd35fb1baade12a9f9b4ab88e1df107473fd0d8282b517f2  test_release-v0.0.1.tar.gz
fqu@dh-res11:~/codes/test/temp/trash$ rm test_release-v0.0.1.tar.gz
fqu@dh-res11:~/codes/test/temp/trash$ wget http://xxx.dtl/fqu/test_release/-/archive/v0.0.1/test_release-v0.0.1.tar.gz && sha256sum test_release-v0.0.1.tar.gz
--2022-06-06 17:42:16--  http://xxx.dtl/fqu/test_release/-/archive/v0.0.1/test_release-v0.0.1.tar.gz
Resolving xxx.dtl (xxx.dtl)... 10.202.208.92
Connecting to xxx.dtl (xxx.dtl)|10.202.208.92|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://xxx.dtl/users/sign_in [following]
--2022-06-06 17:42:17--  http://xxx.dtl/users/sign_in
Reusing existing connection to xxx.dtl:80.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘test_release-v0.0.1.tar.gz’

test_release-v0.0.1.tar.gz                [   <=>                                                                  ]  53.32K   111KB/s    in 0.5s

2022-06-06 17:42:17 (111 KB/s) - ‘test_release-v0.0.1.tar.gz’ saved [54595]

e83a4ee42e3d1c8c83c0eaba0566604d5a7c1da3029da8c19bbc276e48d4e7b8  test_release-v0.0.1.tar.gz

Thanks!

Tried to reproduce the problem on GitLab.com SaaS (which is 15.1 pre, way more ahead than 13.12.15).

  1. Created a new Git Tag in v0.0.1-kubeconeu2022 · Tags · everyonecancontribute / observability / cpp-dns-leaker · GitLab (a demo for my KubeCon EU talk)
  2. Inspected the download button to extract the URL for the CLI: https://gitlab.com/everyonecancontribute/observability/cpp-dns-leaker/-/archive/v0.0.1-kubeconeu2022/cpp-dns-leaker-v0.0.1-kubeconeu2022.zip
$ wget -4 https://gitlab.com/everyonecancontribute/observability/cpp-dns-leaker/-/archive/v0.0.1-kubeconeu2022/cpp-dns-leaker-v0.0.1-kubeconeu2022.zip ; sha256sum *.zip; rm *.zip

Generates the same checksum every time, as expected.

I found some discussions which point to longer-term caching refreshments but this feels odd, given that your download commands are fired immediately one after the other.

Maybe there is a bug in 13.x - I’d recommend verifying the changelog and upgrading your GitLab instance.

Sorry, I missed replying to your second question.

Git itself only provides git tag and git archive which do not provide a checksum verification functionality nor do tags provide the possibility to upload more assets. This is something GitLab builds on top with Releases linked to Git tags, providing more features and fields. Releases need to be created based on a Git Tag (or in one go from the Git tag creation page), allow you to add a description, upload more assets, and also provide so-called “Release Evidence”.

Release binaries, checksums, etc. can be stored in the generic package repository, and later be linked as release page assets. You can also automate release creation from CI/CD pipelines, which for example generate the release tarball, and checksums and upload them to GitLab using the API. Both files can then be downloaded by clients, and then verified to match.

Hi @dnsmichi, thank you for checking. I am also trying to upgrade the omnibus server that we have but I am met with this error with any version 14.0.X: Failed Upgrade from v13.12.15 to v14

I’ll suggest using tar -tzvf test_release-v0.0.1.tar.gz as the [text/html] suggests some other, changing content is being saved to the tar.gz file. I’ve done this myself so I’m sensitive to the possibility. After 58+ years, I say “assume nothing” more than I would have predicted when much younger.
Hope you can resolve this.

1 Like

Based on

I’m wondering if you’re really downloading the compressed tarball and not some login page.

Another thing to check, if you really get compressed tarballs, are any timestamps embedded. Download two tarballs, compute their checksums, compare the tar tvaf outputs and finally run a diff -rq on the extracted contents to see if there are any differences in content.

1 Like

Sorry for the delay on this. But going back to @dnsmichi , and I tried his approach and it was already giving out the same checksum. Thank you very much to everyone for your suggestions.

1 Like