Fail upgrade from 14.4.0 to 14.10.5

How to Use GitLab Upgrade
1

Hi
I’m trying to upgrade a Gitlab. I successfully upgraded to 14.6.5, then 14.6.5 to 14.9.5 is upgrade failed.

This is an error log.

    end
    
    System Info:
    ------------
    chef_version=15.17.4
    platform=centos
    platform_version=7.6.1810
    ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client
    

Running handlers:
There was an error running gitlab-ctl reconfigure:

rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash"  "/tmp/chef-script20220704-25285-148pq3y" ----
STDOUT: rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

PG::InvalidSchemaName: ERROR:  schema "gitlab_partitions_static" does not exist
LINE 1: ...pplication:web,db_config_name:main*/ CREATE TABLE gitlab_par...
                                                             ^
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:133:in `block (2 levels) in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:127:in `block in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:126:in `create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211007090229_create_issue_search_table.rb:23:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:309:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:308:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:82:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'

Caused by:
ActiveRecord::StatementInvalid: PG::InvalidSchemaName: ERROR:  schema "gitlab_partitions_static" does not exist
LINE 1: ...pplication:web,db_config_name:main*/ CREATE TABLE gitlab_par...
                                                             ^
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:133:in `block (2 levels) in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:127:in `block in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:126:in `create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211007090229_create_issue_search_table.rb:23:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:309:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:308:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:82:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'

Caused by:
PG::InvalidSchemaName: ERROR:  schema "gitlab_partitions_static" does not exist
LINE 1: ...pplication:web,db_config_name:main*/ CREATE TABLE gitlab_par...
                                                             ^
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:133:in `block (2 levels) in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:127:in `block in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:126:in `create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211007090229_create_issue_search_table.rb:23:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:309:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:308:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:82:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
== 20211007090229 CreateIssueSearchTable: migrating ===========================
-- execute("CREATE TABLE issue_search_data (\n  project_id bigint NOT NULL REFERENCES projects(id) ON DELETE CASCADE,\n  issue_id bigint NOT NULL REFERENCES issues(id) ON DELETE CASCADE,\n  created_at timestamp with time zone DEFAULT NOW() NOT NULL,\n  updated_at timestamp with time zone DEFAULT NOW() NOT NULL,\n  search_vector tsvector,\n  PRIMARY KEY (project_id, issue_id)\n) PARTITION BY HASH (project_id)\n")
   -> 0.0050s
-- add_index(:issue_search_data, :issue_id)
   -> 0.0008s
-- add_index(:issue_search_data, :search_vector, {:using=>:gin, :name=>"index_issue_search_data_on_search_vector"})
   -> 0.0005s
-- transaction()
-- execute("CREATE TABLE gitlab_partitions_static.issue_search_data_00\nPARTITION OF issue_search_data\nFOR VALUES WITH (MODULUS 64, REMAINDER 0);\n")
STDERR: 
---- End output of "bash"  "/tmp/chef-script20220704-25285-148pq3y" ----
Ran "bash"  "/tmp/chef-script20220704-25285-148pq3y" returned 1

[root@localhost test]# gitlab-rake db:migrate
== 20211007090229 CreateIssueSearchTable: migrating ===========================
-- execute("CREATE TABLE issue_search_data (\n  project_id bigint NOT NULL REFERENCES projects(id) ON DELETE CASCADE,\n  issue_id bigint NOT NULL REFERENCES issues(id) ON DELETE CASCADE,\n  created_at timestamp with time zone DEFAULT NOW() NOT NULL,\n  updated_at timestamp with time zone DEFAULT NOW() NOT NULL,\n  search_vector tsvector,\n  PRIMARY KEY (project_id, issue_id)\n) PARTITION BY HASH (project_id)\n")
   -> 0.0386s
-- add_index(:issue_search_data, :issue_id)
   -> 0.0008s
-- add_index(:issue_search_data, :search_vector, {:using=>:gin, :name=>"index_issue_search_data_on_search_vector"})
   -> 0.0006s
-- transaction()
-- execute("CREATE TABLE gitlab_partitions_static.issue_search_data_00\nPARTITION OF issue_search_data\nFOR VALUES WITH (MODULUS 64, REMAINDER 0);\n")
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

PG::InvalidSchemaName: ERROR:  schema "gitlab_partitions_static" does not exist
LINE 1: ...pplication:web,db_config_name:main*/ CREATE TABLE gitlab_par...
                                                             ^
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:133:in `block (2 levels) in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:127:in `block in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:126:in `create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211007090229_create_issue_search_table.rb:23:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:309:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:308:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'

Caused by:
ActiveRecord::StatementInvalid: PG::InvalidSchemaName: ERROR:  schema "gitlab_partitions_static" does not exist
LINE 1: ...pplication:web,db_config_name:main*/ CREATE TABLE gitlab_par...
                                                             ^
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:133:in `block (2 levels) in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:127:in `block in create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/partitioning_migration_helpers/table_management_helpers.rb:126:in `create_hash_partitions'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211007090229_create_issue_search_table.rb:23:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:309:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:308:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'


gitlabhq_production=# \dt+ gitlab_partitions*
Did not find any relation named "gitlab_partitions*".

Any help is apreciated !
Thanks.

2

Hi there,

You seem to be missing the schema gitlab_paritions_static from your DB.
That particular one was introduced in 20200623170000 (DB migration).

You can check the status of that particular migration:

sudo gitlab-rake db:migrate:status | grep 20200623170000

Or eventually try rerunning it:

sudo gitlab-rake db:migrate:redo VERSION=20200623170000

More about this can be found under GitLab - Migration wrangling

2 Likes