Hello,
Today I’ve decided to update my gitlab instance, unfortunately I ran into problems, check this out:
root@behemoth.home.lan:~# apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
gitlab-ce
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 972 MB of archives.
After this operation, 23.4 MB disk space will be freed.
Do you want to continue? [Y/n] y
Get:1 https://packages.gitlab.com/gitlab/gitlab-ce/ubuntu focal/main amd64 gitlab-ce amd64 14.5.0-ce.0 [972 MB]
Fetched 972 MB in 1min 28s (11.1 MB/s)
dpkg-deb (subprocess): decompressing archive member: internal gzip read error: '<fd:4>: invalid block type'
dpkg-deb: error: <decompress> subprocess returned error exit status 2
dpkg-deb (subprocess): cannot copy archive member from '/var/cache/apt/archives/gitlab-ce_14.5.0-ce.0_amd64.deb' to decompressor pipe: failed to write (Broken pipe)
Reading changelogs... Done
(Reading database ... 249936 files and directories currently installed.)
Preparing to unpack .../gitlab-ce_14.5.0-ce.0_amd64.deb ...
gitlab preinstall: Checking for unmigrated data on legacy storage
gitlab preinstall: Automatically backing up only the GitLab SQL database (excluding everything else!)
2021-11-23 13:45:06 +0200 -- Dumping database ...
Dumping PostgreSQL database gitlabhq_production ... [DONE]
2021-11-23 13:45:09 +0200 -- done
2021-11-23 13:45:09 +0200 -- Dumping repositories ...
2021-11-23 13:45:09 +0200 -- [SKIPPED]
2021-11-23 13:45:09 +0200 -- Dumping uploads ...
2021-11-23 13:45:09 +0200 -- [SKIPPED]
2021-11-23 13:45:09 +0200 -- Dumping builds ...
2021-11-23 13:45:09 +0200 -- [SKIPPED]
2021-11-23 13:45:09 +0200 -- Dumping artifacts ...
2021-11-23 13:45:09 +0200 -- [SKIPPED]
2021-11-23 13:45:09 +0200 -- Dumping pages ...
2021-11-23 13:45:09 +0200 -- [SKIPPED]
2021-11-23 13:45:09 +0200 -- Dumping lfs objects ...
2021-11-23 13:45:09 +0200 -- [SKIPPED]
2021-11-23 13:45:09 +0200 -- Dumping container registry images ...
2021-11-23 13:45:09 +0200 -- [DISABLED]
Creating backup archive: 1637667909_2021_11_23_14.4.2_gitlab_backup.tar ... done
Uploading backup archive to remote storage ... skipped
Deleting tmp directories ... done
done
Deleting old backups ... done. (0 removed)
Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
Backup task is done.
gitlab preinstall: Automatically backing up /etc/gitlab
Running configuration backup
Creating configuration backup archive: gitlab_config_1637667910_2021_11_23.tar
/etc/gitlab/
/etc/gitlab/gitlab.rb
/etc/gitlab/gitlab.rb.orign
/etc/gitlab/gitlab-secrets.json
/etc/gitlab/trusted-certs/
/etc/gitlab/ssl/
/etc/gitlab/ssl/wildcard.home.lan.key
/etc/gitlab/ssl/CA.pem
/etc/gitlab/ssl/wildcard.home.lan.crt
Configuration backup archive complete: /etc/gitlab/config_backup/gitlab_config_1637667910_2021_11_23.tar
Keeping all older configuration backups
Unpacking gitlab-ce (14.5.0-ce.0) over (14.4.2-ce.0) ...
dpkg-deb (subprocess): decompressing archive member: internal gzip read error: '<fd:4>: invalid block type'
dpkg-deb: error: <decompress> subprocess returned error exit status 2
dpkg: error processing archive /var/cache/apt/archives/gitlab-ce_14.5.0-ce.0_amd64.deb (--unpack):
cannot copy extracted data for './opt/gitlab/embedded/bin/gitlab-workhorse' to '/opt/gitlab/embedded/bin/gitlab-workhorse.dpkg-new': unexpected end of file or stream
Errors were encountered while processing:
/var/cache/apt/archives/gitlab-ce_14.5.0-ce.0_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@behemoth.home.lan:~#```
Fortunately my Gitlab instance is not broken. but still.
Can you give me some advice how to fix it up?
Already tried apt-get clean, but it didn't help.
So I downloaded the file again from my Windows box, then compared the sum again, and it matched the sum shown in the website. Then I have uploaded this file into the Linux box and tried to install - the same issue. So the issue is not in the transaction.
The odd is that I really don’t know why the Linux box shows different md5sum.
I just tested downloading directly to my Linux machine with wget, and md5sum matches. Potentially you have a problem copying between machines. Maybe download directly to the server you are wanting to update Gitlab on using wget like I did?
Yeah, thanks @iwalker Seems that, the issue is only local.
I tried to download the deb package using wget on another machine on the same network, then calculated the md5sum and it matched. Then transferred the file to the gitlab server, again calculated the sum and it doesn’t match anymore…
Most probably issue with the storage, not in the transfer, because as I said the box I downloaded the file is in the same network segment as the gitlab server.
Could be, if you can boot to recovery and then run filesystem checks with fsck just in case. Usually should show something though in dmesg or /var/log/syslog if it was having problems with blocks, etc. Either that or the network card is playing up.
I saw some errors on the message log related to IO, it seems like Proxmox issue, because the physical servers are fine, and the other machines are fine as well.