Hi everyone,
I encountered a problem that for the same archive: https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 , Gitlab may return different files.
root@vultr:~# curl -L https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 | sha256sum
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 9222k 0 9222k 0 0 22.0M 0 --:--:-- --:--:-- --:--:-- 22.1M
2ddef549e1eaeecc1fc48f0d8332ea3545809e46509db69beb3a0a4bf19ef906 -
# wait for some time...
root@vultr:~# curl -L https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 | sha256sum
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 9218k 0 9218k 0 0 1786k 0 --:--:-- 0:00:05 --:--:-- 1474k
0e988582f315fe76c909accf5e7f81b975c5bd2b850ee760d8e9fac297f70b5d -
I noticed Gitlab used different directory names when making the tarballs, SVT-AV1-v1.4.1 and SVT-AV1-018276d714ce65d9b586f6205ee016cbd8d5425d.
Some communities like homebrew ensure the safety of archives by checksum, but would certainly fail due to this problem: svt-av1: fix sha256 checksum by qianbinbin · Pull Request #127055 · Homebrew/homebrew-core · GitHub
Thanks.
You can see from the first download it was 9222k and for the second download 9218k - since this value is not the same, then that means the download hasn’t downloaded fully and this is why the sha256sum doesn’t match.
Checksums are supposed to be used when the file has been downloaded, and usually the checksums are provided in a text file for the downloads that are available.
This isn’t a Gitlab problem, but rather your download has failed.
Nope. They are just different files:
SVT-AV1-v1.4.1 and SVT-AV1-018276d714ce65d9b586f6205ee016cbd8d5425d.
❯ ls # the directory names are totally different!!
SVT-AV1-018276d714ce65d9b586f6205ee016cbd8d5425d
SVT-AV1-v1.4.1
❯ diff SVT-AV1-v1.4.1 SVT-AV1-018276d714ce65d9b586f6205ee016cbd8d5425d -r
❯ echo $? # but the contents are exactly the same!!
0
I can reproduce in macOS and 4 different Linux servers, in China and the US, amd64/arm64/armv7. It’s not possible all the computers have the same network issue.
It’s weird that I cannot reproduce for now, I’ll try later.
Luckily I saved the two different files with same contents mentioned above, download from GitHub: https://github.com/Homebrew/homebrew-core/files/11114134/old-and-new.zip
I don’t seem to be able to replicate your problem.
[ian@elise tmp]$ curl -s -L https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 | sha256sum
0e988582f315fe76c909accf5e7f81b975c5bd2b850ee760d8e9fac297f70b5d -
[ian@elise tmp]$ curl -s -L https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 | sha256sum
0e988582f315fe76c909accf5e7f81b975c5bd2b850ee760d8e9fac297f70b5d -
[ian@elise tmp]$ curl -s -L https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 | sha256sum
0e988582f315fe76c909accf5e7f81b975c5bd2b850ee760d8e9fac297f70b5d -
[ian@elise tmp]$ curl -s -L https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 | sha256sum
0e988582f315fe76c909accf5e7f81b975c5bd2b850ee760d8e9fac297f70b5d -
[ian@elise tmp]$ curl -s -L https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 | sha256sum
0e988582f315fe76c909accf5e7f81b975c5bd2b850ee760d8e9fac297f70b5d -
[ian@elise tmp]$ curl -s -L https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 | sha256sum
0e988582f315fe76c909accf5e7f81b975c5bd2b850ee760d8e9fac297f70b5d -
[ian@elise tmp]$ curl -s -L https://gitlab.com/AOMediaCodec/SVT-AV1/-/archive/v1.4.1/SVT-AV1-v1.4.1.tar.bz2 | sha256sum
0e988582f315fe76c909accf5e7f81b975c5bd2b850ee760d8e9fac297f70b5d -
provides the same result every time.
Thanks for your reply. I can always reproduce this several hours ago, I’ll check later.
Anyway, it’s not a network problem for sure, the contents are exactly the same and all files are not corrupted, the only difference is the directory name.
You can check the files I downloaded: https://github.com/Homebrew/homebrew-core/files/11114134/old-and-new.zip
Link may be useful: Gitlab release tarball stability - #2 by diegorondini
Gitlab did return different files to me, but for now I can’t reproduce. I suppose it could be related to cache mechanism of Gitlab.
Follow up: Gitlab returned different files for the same archive URL (#403597) · Issues · GitLab.org / GitLab · GitLab
I seem to be seeing a similar issue with the Frotz 2.54 release - see discussion and investigation here. The checksum seems to have changed at some point between Nov 2022 and today.
@qianbinbin I wonder if the file was regenerated recently, and you’re occasionally getting served an old and new version from different caches.
1 Like
@jcgraybill Thank you so much!
it’s really exhausting for the lack of interest while I stuck for hours, and some even thought it’s my configuration problem.
1 Like
@qianbinbin It’s difficult to say where the problem is, for example when you posted you had an issue, and then later you also couldn’t replicate it. I also couldn’t replicate your problem, so it’s not easy to help. Luckily someone else did have an issue, so means that obviously some downloads have been regenerated and cache is the issue.
Also remember, you come here for help, and people help for free. So respect that. You aren’t guaranteed help and you aren’t guaranteed a solution. If you want guarantees then pay for a subscription.
Calm down. I didn’t mean you, I mean some maintainer of homebrew.
And I was’t asking for help, I did report an issue of Gitlab which should be considered the severity of high level.
1 Like
I am calm 
but thanks for the clarification that it was the maintainer.
1 Like
Sorry, I got a bad temper for this problem. Thanks for your help.
1 Like
I wonder if the file was regenerated recently, and you’re occasionally getting served an old and new version from different caches..
.