Hi all,
Our Gitlab installation was running without issues (and we do have a backup), but upon updating recently it failed and started throwing the error on the subject.
This is the update info:
Transaction ID : 9
Begin time : Fri 14 Aug 2020 01:20:17 PM WEST
Begin rpmdb : 459:da6d8421630b70c4092183da2b42bcc869066d8b
End time : Thu 01 Jan 1970 01:00:00 AM CET (-1597407617 seconds)
End rpmdb :
User : root <root>
Return-Code : Failure: 1
Releasever :
Command Line : update -y
Packages Altered:
** Install python3-unbound-1.7.3-11.el8_2.x86_64 @AppStream
** Install unbound-libs-1.7.3-11.el8_2.x86_64 @AppStream
** Upgrade nspr-4.25.0-2.el8_2.x86_64 @AppStream
** Upgraded nspr-4.21.0-2.el8_0.x86_64 @@System
** Upgrade nss-3.53.1-11.el8_2.x86_64 @AppStream
** Upgraded nss-3.44.0-15.el8.x86_64 @@System
** Upgrade nss-softokn-3.53.1-11.el8_2.x86_64 @AppStream
** Upgraded nss-softokn-3.44.0-15.el8.x86_64 @@System
** Upgrade nss-softokn-freebl-3.53.1-11.el8_2.x86_64 @AppStream
** Upgraded nss-softokn-freebl-3.44.0-15.el8.x86_64 @@System
** Upgrade nss-sysinit-3.53.1-11.el8_2.x86_64 @AppStream
** Upgraded nss-sysinit-3.44.0-15.el8.x86_64 @@System
** Upgrade nss-tools-3.53.1-11.el8_2.x86_64 @AppStream
** Upgraded nss-tools-3.44.0-15.el8.x86_64 @@System
** Upgrade nss-util-3.53.1-11.el8_2.x86_64 @AppStream
** Upgraded nss-util-3.44.0-15.el8.x86_64 @@System
** Upgrade NetworkManager-1:1.22.8-5.el8_2.x86_64 @BaseOS
** Upgraded NetworkManager-1:1.22.8-4.el8.x86_64 @@System
** Upgrade NetworkManager-libnm-1:1.22.8-5.el8_2.x86_64 @BaseOS
** Upgraded NetworkManager-libnm-1:1.22.8-4.el8.x86_64 @@System
** Upgrade NetworkManager-team-1:1.22.8-5.el8_2.x86_64 @BaseOS
** Upgraded NetworkManager-team-1:1.22.8-4.el8.x86_64 @@System
** Upgrade NetworkManager-tui-1:1.22.8-5.el8_2.x86_64 @BaseOS
** Upgraded NetworkManager-tui-1:1.22.8-4.el8.x86_64 @@System
** Upgrade ca-certificates-2020.2.41-80.0.el8_2.noarch @BaseOS
** Upgraded ca-certificates-2019.2.32-80.0.el8_1.noarch @@System
** Upgrade dbus-1:1.12.8-10.el8_2.x86_64 @BaseOS
** Upgraded dbus-1:1.12.8-9.el8.x86_64 @@System
** Upgrade dbus-common-1:1.12.8-10.el8_2.noarch @BaseOS
** Upgraded dbus-common-1:1.12.8-9.el8.noarch @@System
** Upgrade dbus-daemon-1:1.12.8-10.el8_2.x86_64 @BaseOS
** Upgraded dbus-daemon-1:1.12.8-9.el8.x86_64 @@System
** Upgrade dbus-libs-1:1.12.8-10.el8_2.x86_64 @BaseOS
** Upgraded dbus-libs-1:1.12.8-9.el8.x86_64 @@System
** Upgrade dbus-tools-1:1.12.8-10.el8_2.x86_64 @BaseOS
** Upgraded dbus-tools-1:1.12.8-9.el8.x86_64 @@System
** Upgrade dnf-4.2.17-7.el8_2.noarch @BaseOS
** Upgraded dnf-4.2.17-6.el8.noarch @@System
** Upgrade dnf-data-4.2.17-7.el8_2.noarch @BaseOS
** Upgraded dnf-data-4.2.17-6.el8.noarch @@System
** Upgrade iptables-1.8.4-10.el8_2.1.x86_64 @BaseOS
** Upgraded iptables-1.8.4-10.el8.x86_64 @@System
** Upgrade iptables-ebtables-1.8.4-10.el8_2.1.x86_64 @BaseOS
** Upgraded iptables-ebtables-1.8.4-10.el8.x86_64 @@System
** Upgrade iptables-libs-1.8.4-10.el8_2.1.x86_64 @BaseOS
** Upgraded iptables-libs-1.8.4-10.el8.x86_64 @@System
** Upgrade libdnf-0.39.1-6.el8_2.x86_64 @BaseOS
** Upgraded libdnf-0.39.1-5.el8.x86_64 @@System
** Upgrade python3-dnf-4.2.17-7.el8_2.noarch @BaseOS
** Upgraded python3-dnf-4.2.17-6.el8.noarch @@System
** Upgrade python3-hawkey-0.39.1-6.el8_2.x86_64 @BaseOS
** Upgraded python3-hawkey-0.39.1-5.el8.x86_64 @@System
** Upgrade python3-libdnf-0.39.1-6.el8_2.x86_64 @BaseOS
** Upgraded python3-libdnf-0.39.1-5.el8.x86_64 @@System
** Upgrade selinux-policy-3.14.3-41.el8_2.5.noarch @BaseOS
** Upgraded selinux-policy-3.14.3-41.el8_2.4.noarch @@System
** Upgrade selinux-policy-targeted-3.14.3-41.el8_2.5.noarch @BaseOS
** Upgraded selinux-policy-targeted-3.14.3-41.el8_2.4.noarch @@System
** Upgrade systemd-239-31.el8_2.2.x86_64 @BaseOS
** Upgraded systemd-239-30.el8_2.x86_64 @@System
** Upgrade systemd-libs-239-31.el8_2.2.x86_64 @BaseOS
** Upgraded systemd-libs-239-30.el8_2.x86_64 @@System
** Upgrade systemd-pam-239-31.el8_2.2.x86_64 @BaseOS
** Upgraded systemd-pam-239-30.el8_2.x86_64 @@System
** Upgrade systemd-udev-239-31.el8_2.2.x86_64 @BaseOS
** Upgraded systemd-udev-239-30.el8_2.x86_64 @@System
** Upgrade yum-4.2.17-7.el8_2.noarch @BaseOS
** Upgraded yum-4.2.17-6.el8.noarch @@System
** Upgrade gitlab-ee-13.2.4-ee.0.el8.x86_64 @gitlab_gitlab-ee
** Upgraded gitlab-ee-13.2.3-ee.0.el8.x86_64 @@System
But gitlab-ee-13.2.4-ee.0.el8.x86_64 fails with:
Traceback (most recent call last):
<internal:gem_prelude>:4:in `<internal:gem_prelude>': undefined method `gem' for main:Object (NoMethodError)
1: from <internal:gem_prelude>:3:in `<internal:gem_prelude>'
<internal:gem_prelude>:6:in `rescue in <internal:gem_prelude>': uninitialized constant Gem::LoadError (NameError)
Now whenever I do gitlab-ctl always throws that error, and gitlab is not running.
[root@code1 ~]# ruby -v
ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
[root@code1 ~]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="8 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="8"
Using Omnibus Install.
gitlab.rb file is up to date, the diff has been performed against the latest template available on gitlab repo:
[root@code1 gitlab]# diff gitlab.rb gitlab.rb.template
32c32
< external_url "https://gitlab.domain.io"
---
> external_url 'GENERATED_EXTERNAL_URL'
393,394c393,394
< gitlab_rails['ldap_enabled'] = true
< gitlab_rails['prevent_ldap_sign_in'] = YAML.load_file('/path/to/ldap/config.yml')
---
> # gitlab_rails['ldap_enabled'] = false
> # gitlab_rails['prevent_ldap_sign_in'] = false
2007c2007
< letsencrypt['enable'] = true
---
> # letsencrypt['enable'] = nil