Bash[migrate gitlab-rails database] error when i do "gitlab-ctl reconfigure" - Gitlab

Hi,
I have a problem with my Gitlab and i cannot resolve it :frowning:
I tried to backup with sudo gitlab-rake gitlab:backup:create STRATEGY=copy, i did that many times and everything was fine, but the last time there was some error.

Error executing action run on resource 'bash[migrate gitlab-rails database]'
================================================================================

Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of "bash"  "/tmp/chef-script20191024-2422-c0yhb3" ----
STDOUT: rake aborted!
TypeError: Internal is not a module
/opt/gitlab/embedded/service/gitlab-rails/lib/api/internal/base.rb:5:in `<module:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/internal/base.rb:3:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:121:in `<class:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:4:in `<module:API>'
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:3:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/rack_attack.rb:12:in `<top (required)>'
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:6:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:configure => environment
(See full trace by running task with --trace)
STDERR:
---- End output of "bash"  "/tmp/chef-script20191024-2422-c0yhb3" ----
Ran "bash"  "/tmp/chef-script20191024-2422-c0yhb3" returned 1

Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb

 54: bash "migrate gitlab-rails database" do
 55:   code <<-EOH
 56:     set -e
 57:     log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
 58:     umask 077
 59:     /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
 60:     STATUS=${PIPESTATUS[0]}
 61:     chown #{account_helper.gitlab_user}:#{account_helper.gitlab_group} ${log_file}
 62:     echo $STATUS > #{db_migrate_status_file}
 63:     exit $STATUS
 64:   EOH
 65:   environment env_variables unless env_variables.empty?
 66:   notifies :run, "execute[clear the gitlab-rails cache]", :immediately
 67:   dependent_services.each do |svc|
 68:     notifies :restart, svc, :immediately
 69:   end
 70:   not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
 71:   only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
 72: end
 73:

Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:54:in `from_file'

bash("migrate gitlab-rails database") do
  action [:run]
  default_guard_interpreter :default
  command nil
  backup 5
  interpreter "bash"
  declared_type :bash
  cookbook_name "gitlab"
  recipe_name "database_migrations"
  code "    set -e\n    log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n    umask 077\n    /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n    STATUS=${PIPESTATUS[0]}\n    chown git:git ${log_file}\n    echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-2417d5becc7\n    exit $STATUS\n"
  domain nil
  user nil
  not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-2417d5becc7) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-2417d5becc7 | grep -Fx 0)"
  only_if { #code block }
end

System Info:
------------
chef_version=14.13.11
platform=centos
platform_version=6.10
ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client

Running handlers:
There was an error running gitlab-ctl reconfigure:

bash[migrate gitlab-rails database] (gitlab::database_migrations line 54) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘1’
---- Begin output of “bash” “/tmp/chef-script20191024-2422-c0yhb3” ----
STDOUT: rake aborted!
TypeError: Internal is not a module
/opt/gitlab/embedded/service/gitlab-rails/lib/api/internal/base.rb:5:in <module:API>' /opt/gitlab/embedded/service/gitlab-rails/lib/api/internal/base.rb:3:in <top (required)>’
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:121:in <class:API>' /opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:4:in module:API
/opt/gitlab/embedded/service/gitlab-rails/lib/api/api.rb:3:in <top (required)>' /opt/gitlab/embedded/service/gitlab-rails/config/initializers/rack_attack.rb:12:in <top (required)>’
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:6:in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in load’
/opt/gitlab/embedded/bin/bundle:23:in `’
Tasks: TOP => gitlab:db:configure => environment
(See full trace by running task with --trace)
STDERR:
---- End output of “bash” “/tmp/chef-script20191024-2422-c0yhb3” ----
Ran “bash” “/tmp/chef-script20191024-2422-c0yhb3” returned 1

Running handlers complete
Chef Client failed. 5 resources updated in 01 minutes 53 seconds


This is the whole error when i was tried to do “gitlab-ctl reconfigure”. Now my git server doesn’t work and i don’t know how to fix this error.
I have 8 GB RAM, so i guess that it’s not a problem with memory.
I cannot do command like gitlab:rake info etc.
All services in gitlab-ctl status are working (RUN)
Tried to restart gitlab-ctl but unsuccess.
Also, cannot yum install gitlab-ce newer version before i fix gitlab-ctl reconfigure :frowning:

STDOUT: rake aborted!
TypeError: Internal is not a module

What does mean this error, Internal is not a module, i didn’t find any Topis whit this error :frowning: I will be gratefull if someone help me with this. Thanks a lot!*

I resolve the problem.
I did yum update -y, so it replaced gitlab-ce with gitlab-ee, and now it’s working :slight_smile: