Prometheus upgrade fail on version check

Observability
1

I’m using gitlab-ce package on Centos 7. Updated version today and was warned that prometheus v 1.X is being deprecated, and to run gitlab-ctl prometheus-upgrade to go to version 2.

Deprecations:
== Prometheus ==
Detected Prometheus version 1.x. Version 1.x has been deprecated and support will be removed in GitLab version 12.0.
To upgrade to Promtheus 2.x, use `gitlab-ctl prometheus-upgrade` command.
Running this command will migrate all your existing data to format supported by Prometheus 2.x.
This can be a time consuming operation. To skip migrating the data, and instead remove and start fresh, run `gitlab-ctl prometheus-upgrade --skip-data-migration`.
Check https://docs.gitlab.com/omnibus/update/gitlab_11_changes.html#11-4 for details.
---

gitlab Reconfigured!

Upon running the command I get an error about a undefined method doing a version check, see below. Any help is appreciated.

[user@server]$ sudo gitlab-ctl prometheus-upgrade
/opt/gitlab/embedded/service/omnibus-ctl/lib/gitlab_ctl/prometheus_upgrade.rb:31:in `version_string_check': undefined method `[]' for nil:NilClass (NoMethodError)
        from /opt/gitlab/embedded/service/omnibus-ctl/lib/gitlab_ctl/prometheus_upgrade.rb:22:in `is_version_2?'
        from /opt/gitlab/embedded/service/omnibus-ctl/prometheus-upgrade.rb:35:in `block in load_file'
        from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/omnibus-ctl-0.6.0/lib/omnibus-ctl.rb:197:in `block in add_command'
        from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/omnibus-ctl-0.6.0/lib/omnibus-ctl.rb:746:in `run'
        from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/omnibus-ctl-0.6.0/bin/omnibus-ctl:31:in `<top (required)>'
        from /opt/gitlab/embedded/bin/omnibus-ctl:23:in `load'
        from /opt/gitlab/embedded/bin/omnibus-ctl:23:in `<main>'
2

  1. Have you tried to run sudo gitlab-ctl prometheus-upgrade --skip-data-migration?

  2. I think that if you installed GitLab in a Docker container, you wouldn’t have the upgrade problem.

3

What are the risks if I don’t migrate the data? Do I lose the git repos? I’m no expert on gitlab, just running it at home to keep my own scripts and work tracked.

4

As far as I understand from Redirecting..., it looks as if --skip-data-migration relates to Prometheus data only. Note this sentence:

With GitLab 12.0 Prometheus will be upgraded to 2.x automatically, Prometheus 1.0 data will not be migrated.

I would recommend you either to ignore the deprecation warning and wait until GitLab 12.0 or to make first a backup of all GitLab data and try --skip-data-migration.

5

Check if Prometheus is enabled in /etc/gitlab/gitlab.rb
prometheus_monitoring[‘enable’] = true

6

Prometheus isn’t enabled, so it’s not an issue. I did try to run the gitlab-ctl prometheus-upgrade --skip-data-migration and got the same error. I feel like this is a bug that the devs need to know about at this point.

7

That’s the problem.

  1. Enable Prometheus in /etc/gitlab/gitlab.rb
    prometheus_monitoring[‘enable’] = true

  2. Reconfigure gitlab.
    sudo gitlab-ctl reconfigure

  3. Try update.
    sudo gitlab-ctl prometheus-upgrade

  4. Disable Prometheus in /etc/gitlab/gitlab.rb
    prometheus_monitoring[‘enable’] = false

  5. Reconfigure gitlab.
    sudo gitlab-ctl reconfigure

8

Hello, I have this same issue. I followed the steps listed above and got this error:

“ERROR: Encountered unsupported config key ‘‘enable’’ in /etc/gitlab/gitlab.rb.”

This came up during the first reconfigure step.