Issue mirroring Gitlab repos from upstream

Hello,

We are currently running GitLab-ce14.7.2 and we are trying to upgrade to the latest by syncing the local mirror from this patch deb https://packages.gitlab.com/gitlab/gitlab-ce/ubuntu bionic main
It has always worked until now we are getting the following errors after syncing and while trying to run apt-get update on the client

Err:7 http://gitlabaptreposerver/gitlab bionic/main amd64 Packages
File has unexpected size (66916 != 66630). Mirror sync in progress? [IP: 10.10.228.126 80]
Hashes of expected file:

  • Filesize:66630 [weak]
  • SHA256:1e513eecdcc18711d1895edf5541094f414cd99a2e242e5295585b0196b64d69
  • SHA1:5fdf9de37ebad1c17c607412389df0d90f21fc94 [weak]
  • MD5Sum:487b19148e848a3d81ed2df30e10c8de [weak]
    Release file created at: Fri, 25 Feb 2022 23:47:41 +0000
    Fetched 275 kB in 1s (383 kB/s)
    Reading package lists… Done
    E: Failed to fetch http://gitlabaptreposerver/gitlab/dists/bionic/main/binary-amd64/Packages.bz2 File has unexpected size (66916 != 66630). Mirror sync in progress? [IP: 10.50.221.126 80]
    Hashes of expected file:
    - Filesize:66630 [weak]
    - SHA256:1e513eecdcc18711d1895edf5541094f414cd99a2e242e5295585b0196b64d69
    - SHA1:5fdf9de37ebad1c17c607412389df0d90f21fc94 [weak]
    - MD5Sum:487b19148e848a3d81ed2df30e10c8de [weak]
    Release file created at: Fri, 25 Feb 2022 23:47:41 +0000
    E: Some index files failed to download. They have been ignored, or old ones used instead.

We have confirmed the sync to be successful on the gitlabreposerver with the new packages and binaries. We also tried to remove all the packages on the repo and attempted to resync from the upstream server and tried running apt-get update but still get the above error message.

Just wondering if you have any suggestions as to what else we could try. Thank you so much in advance!!

You can try doing something like this:

mkdir /var/lib/apt/old
mv /var/lib/apt/* /var/lib/apt/old
apt-get update

and see if it updates without any errors.

Unfortunately it didn’t work.

Check your local mirror to make sure it’s finished syncing correctly? I have no problems installing or getting packages from the Gitlab one, so looks like you have a problem locally with your mirror not being completely up-to-date. The logs for the mirroring process should hint at issues if syncs are failing because SSL certs/GPG keys were updated recently.

You could verify that as well by disabling your local mirror, and using the Gitlab ones instead - obviously if you can change your network policies/firewall rules to allow that temporarily.

We updated GitLab GPG key, deleted all gitlab packages from repo paths /srv/apt-mirror/skel and /srv/apt-mirror/mirror, and then re-synced repo from upstream repos. We were able to install updates after that.

Thank you!!