Gitlab CE Upgrade from 12.5.4 to 12.10.14 has Failed

Hi Team,

I attempted to upgrade our gitlab CE server from version 12.5.4 to 12.10.14 but has failed. This is what I am seeing after the upgrade:

~#  gitlab-rake gitlab:env:info

System information
System:         Ubuntu 16.04
Current User:   git
Using RVM:      no
Ruby Version:   2.6.6p146
Gem Version:    2.7.10
Bundler Version:1.17.3
Rake Version:   12.3.3
Redis Version:  5.0.7
Git Version:    2.26.2
Sidekiq Version:5.2.7
Go Version:     unknown
rake aborted!
NoMethodError: undefined method `forking_access_level=' for #<ProjectFeature:0x00007fc1ad8188c0>
Did you mean?  string_access_level
               wiki_access_level=
               wiki_access_level
/opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/project_features_compatibility.rb:86:in `write_feature_attribute_raw'
/opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/project_features_compatibility.rb:77:in `write_feature_attribute_boolean'
/opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/project_features_compatibility.rb:29:in `issues_enabled='
/opt/gitlab/embedded/service/gitlab-rails/app/models/project.rb:696:in `initialize'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/info.rake:48:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:env:info
(See full trace by running task with --trace)
  • ruby_block[warn pending redis restart] action run

    ================================================================================
    Error executing action run on resource ‘ruby_block[warn pending redis restart]’

    RuntimeError

    Execution of the command /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket INFO failed with a non-zero exit code (1)
    stdout:
    stderr: Could not connect to Redis at /var/opt/gitlab/redis/redis.socket: No such file or directory

    Cookbook Trace:

    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/helpers/version_helper.rb:16:in version' /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/libraries/redis_helper.rb:63:in running_version’
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/definitions/redis_service.rb:92:in `block (3 levels) in from_file’

    Resource Declaration:

    In /opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/definitions/redis_service.rb

    82: ruby_block ‘warn pending redis restart’ do
    83: block do
    84: message = <<~MESSAGE
    85: The version of the running redis service is different than what is installed.
    86: Please restart redis to start the new version.
    87:
    88: sudo gitlab-ctl restart redis
    89: MESSAGE
    90: LoggingHelper.warning(message)
    91: end
    92: only_if { redis_helper.running_version != redis_helper.installed_version }
    93: end
    94: end

    Compiled Resource:

    Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/definitions/redis_service.rb:82:in `block in from_file’

    ruby_block(“warn pending redis restart”) do
    params {:socket_group=>“git”, :name=>nil}
    action [:run]
    default_guard_interpreter :default
    declared_type :ruby_block
    cookbook_name “redis”
    recipe_name “enable”
    block #Proc:0x0000000004745c80@/opt/gitlab/embedded/cookbooks/cache/cookbooks/redis/definitions/redis_service.rb:83
    block_name “warn pending redis restart”
    only_if { #code block }
    end

    System Info:

    chef_version=14.14.29
    platform=ubuntu
    platform_version=16.04
    ruby=ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client

Recipe: gitlab::unicorn

  • runit_service[unicorn] action restart (up to date)
  • runit_service[unicorn] action restart (up to date)
    Recipe: gitlab::gitlab-rails
  • execute[clear the gitlab-rails cache] action run (skipped due to not_if)

Running handlers:
There was an error running gitlab-ctl reconfigure:

ruby_block[warn pending redis restart] (redis::enable line 82) had an error: RuntimeError: Execution of the command /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket INFO failed with a non-zero exit code (1)
stdout:
stderr: Could not connect to Redis at /var/opt/gitlab/redis/redis.socket: No such file or directory

Running handlers complete
Chef Client failed. 18 resources updated in 16 seconds
Running reconfigure: NOT OK
== Fatal error ==
Something went wrong during final reconfiguration, please check the output
== Reverting ==
ok: down: postgresql: 0s, normally up
Symlink correct version of binaries: OK
ok: run: postgresql: (pid 19255) 0s
== Reverted ==
== Reverted to 10.12. Please check output for what went wrong ==
Toggling deploy page:rm -f /opt/gitlab/embedded/service/gitlab-rails/public/index.html
Toggling deploy page: OK
Toggling services:ok: run: alertmanager: (pid 19273) 0s
ok: run: gitaly: (pid 19275) 0s
ok: run: gitlab-exporter: (pid 19277) 0s
ok: run: gitlab-pages: (pid 19284) 1s
ok: run: grafana: (pid 19307) 0s
ok: run: logrotate: (pid 19327) 0s
ok: run: node-exporter: (pid 19335) 1s
ok: run: postgres-exporter: (pid 19344) 0s
ok: run: prometheus: (pid 19353) 1s
ok: run: redis-exporter: (pid 19363) 0s
ok: run: registry: (pid 19401) 0s
ok: run: sidekiq: (pid 19409) 0s
Toggling services: OK
Ensuring PostgreSQL is updated: NOT OK
Error ensuring PostgreSQL is updated. Please check the logs
dpkg: error processing package gitlab-ce (–install):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
gitlab-ce
:~# gitlab-ctl status
run: alertmanager: (pid 19273) 101s; run: log: (pid 1910) 0s
run: gitaly: (pid 19275) 101s; run: log: (pid 1895) 0s
run: gitlab-exporter: (pid 19277) 101s; run: log: (pid 1896) 0s
run: gitlab-pages: (pid 19284) 101s; run: log: (pid 1871) 0s
run: gitlab-workhorse: (pid 1869) 0s; run: log: (pid 1861) 0s
run: grafana: (pid 19307) 100s; run: log: (pid 1919) 0s
run: logrotate: (pid 19327) 100s; run: log: (pid 1872) 0s
run: nginx: (pid 1877) 0s; run: log: (pid 1873) 0s
run: node-exporter: (pid 19335) 100s; run: log: (pid 1904) 0s
run: postgres-exporter: (pid 19344) 99s; run: log: (pid 1884) 0s
run: postgresql: (pid 19255) 103s; run: log: (pid 1874) 0s
run: prometheus: (pid 19353) 99s; run: log: (pid 1913) 0s
run: redis: (pid 19215) 104s; run: log: (pid 1853) 0s
run: redis-exporter: (pid 19363) 98s; run: log: (pid 1881) 0s
run: registry: (pid 20404) 0s; run: log: (pid 1860) 0s
run: sidekiq: (pid 19409) 95s; run: log: (pid 1859) 0s
run: unicorn: (pid 1867) 0s; run: log: (pid 1854) 0s

Does anyone had ran into similar issue and was able to figure out how to properly upgrade and fix this please. Appreciate your help.

Best!