Apt-get update failing when running CI

Hello,

A few hours ago, my CI was working just fine and now suddenly it fails while running an apt-get update, this is what I see on the console:

Again, no changes happened between the first build I worked on earlier today, and then started getting this error.

Any help will be greatly appreciated.

Hello,

We have the same issue, trying started CI/CD pipeline on various images like node:18, gcc and similar.

Was you able to fix this issue? We have found that could be caused by old version of Docker, so now we have Docker 24.0.7 (server/client) but still the same issue. Moreover sometimes work sometimes not.

Any info would be appreciated.

Hi all,

Going to write answer by myself :wink:

Finally we have this matter solved. Having following versions of “tools” related to Gitlab

Gitlab itself - 16.6.1
Gitlab runner - 16.5.0
Docker version 24.0.7

And it is also necessary to un-install or un-register the older versions of Gitlab runner, having there more versions of Gitlab runner the trouble with apt update can occur “randomly”, especially if the older version of runner is 15.x.x (I don’t remember the version precisely :-()

Maybe this information could be useful :wink:

You can upgrade the runner, it doesn’t need to be removed. Each time I upgrade my Gitlab install, I also upgrade the runner :slight_smile:

Maybe my way of Gitlab runner upgrade wasn’t fully correct and so I had there two versions available and Gitlab used them “randomly”, using old apt update didn’t work, using new apt update worked.

Because of that it was necessary to remove old version.

Unfortunately I don’t know too much details about whole system functionality and so this is just result of our long debugging and searching.

Maybe there is way to just “upgrade” Gitlab runner to destroy old one and replace it by new one.

Can you show us the command used to “upgrade” Gitlab runner? To do this matter correctly next time?

Mine is installed on a Debian, system as an apt package:

root@gitlab-runner:~# dpkg -l | grep -i gitlab
ii  gitlab-runner                   16.7.0                         amd64        GitLab Runner

so I just do:

apt-get upgrade

as I have a repo file installed on the system to get updates:

root@gitlab-runner:~# cat /etc/apt/sources.list.d/runner_gitlab-runner.list
# this file was generated by packages.gitlab.com for
# the repository at https://packages.gitlab.com/runner/gitlab-runner

deb https://packages.gitlab.com/runner/gitlab-runner/debian/ bookworm main
deb-src https://packages.gitlab.com/runner/gitlab-runner/debian/ bookworm main

We are using Ubuntu 22.04, not sure if the same way could be used also there.

Doing experiments…

Yes the apt registry also works for Ubuntu. If one doesn’t exist for 22.04, then the 20.04 focal one can be used. I don’t use Ubuntu, but in the past when I installed Gitlab on Debian 12, and packages existed for Debian 11 but not yet for Debian 12, I installed the Debian 11 ones. Later when Debian 12 packages came out, then I would update the repo files and use the ones released for Debian 12.

Yes, finally fixed, list for gitlab-runner are activated now and using them it is re-installed. Previously it was installed “locally”, strange. Now fixed.

Thank you very much for all hints.

1 Like