Manual DB Migration - PG::UnableToSend: no connection to the server

Hello Team,

I’m trying to upgrade the gitlab-ce from 14.9.5 to 14.10.5 . While upgrading , I’m disabled the auto DB migration and upgraded the gitlab. And after the gitlab upgrade , I’m tried to do the DB migration but its failed with below error. Please help me out from this issue.

gitlab-rake db:migrate

== 20211012134316 CleanUpMigrateMergeRequestDiffCommitUsers: migrating ========

– 15 MigrateMergeRequestDiffCommitUsers are still pending

rake aborted!

StandardError: An error has occurred, this and all later migrations canceled:

PG::UnableToSend: no connection to the server

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Caused by:

ActiveRecord::StatementInvalid: PG::UnableToSend: no connection to the server

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Caused by:

PG::UnableToSend: no connection to the server

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Caused by:

ActiveRecord::StatementInvalid: PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly

This probably means the server terminated abnormally

before or while processing the request.

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:99:in `block in read_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:98:in `read_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:47:in `select_all’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:65:in `block in each_row_to_migrate’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/pagination/keyset/iterator.rb:25:in `block in each_batch’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/pagination/keyset/iterator.rb:19:in `loop’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/pagination/keyset/iterator.rb:19:in `each_batch’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:65:in `each_row_to_migrate’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:119:in `get_data_to_update’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:87:in `perform’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:29:in `block in up’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:26:in `each’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:26:in `up’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Caused by:

PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly

This probably means the server terminated abnormally

before or while processing the request.

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:99:in `block in read_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:98:in `read_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:47:in `select_all’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:65:in `block in each_row_to_migrate’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/pagination/keyset/iterator.rb:25:in `block in each_batch’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/pagination/keyset/iterator.rb:19:in `loop’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/pagination/keyset/iterator.rb:19:in `each_batch’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:65:in `each_row_to_migrate’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:119:in `get_data_to_update’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:87:in `perform’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:29:in `block in up’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:26:in `each’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:26:in `up’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Tasks: TOP => db:migrate

(See full trace by running task with --trace)

root@restored-ixed-ops:~# gitlab-rake db:migrate

== 20211012134316 CleanUpMigrateMergeRequestDiffCommitUsers: migrating ========

– 15 MigrateMergeRequestDiffCommitUsers are still pending

rake aborted!

StandardError: An error has occurred, this and all later migrations canceled:

PG::UnableToSend: no connection to the server

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Caused by:

ActiveRecord::StatementInvalid: PG::UnableToSend: no connection to the server

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Caused by:

PG::UnableToSend: no connection to the server

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Caused by:

ActiveRecord::StatementInvalid: PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly

This probably means the server terminated abnormally

before or while processing the request.

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:88:in `method_missing’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:267:in `bulk_update_commit_rows’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:194:in `block in update_commit_rows’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:184:in `each’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:184:in `each_slice’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:184:in `update_commit_rows’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:91:in `perform’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:29:in `block in up’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:26:in `each’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:26:in `up’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Caused by:

PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly

This probably means the server terminated abnormally

before or while processing the request.

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:88:in `method_missing’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:267:in `bulk_update_commit_rows’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:194:in `block in update_commit_rows’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:184:in `each’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:184:in `each_slice’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:184:in `update_commit_rows’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/migrate_merge_request_diff_commit_users.rb:91:in `perform’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:29:in `block in up’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:26:in `each’

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211012134316_clean_up_migrate_merge_request_diff_commit_users.rb:26:in `up’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:305:in `block in transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:304:in `transaction’

/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:in `ddl_transaction’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Tasks: TOP => db:migrate

(See full trace by running task with --trace)

Hi,

When posting, please use the formatting tools to make your posts easier to read by the people you want to help you: Community, first steps: Code, config, log block formatting in topics and replies

But from what I see it looks like postgres database isn’t running. You should check your services:

gitlab-ctl status

and try:

gitlab-ctl reconfigure
gitlab-ctl restart

Hello Ian,

Thanks for the response. I have tried your recommendation and am getting the same error. It’s throwing this error after 1 hour of executing that migration command, not immediately. And I’m checked that psql is up and runningwhile doing the migration.

Hello Iwalker,

And, while doing the gitlab-ctl reconfigure it throws below error.

There was an error running gitlab-ctl reconfigure:

rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::CommandTimeout: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::CommandTimeout: Command timed out after 3600s:
Command exceeded allowed execution time, process terminated
---- Begin output of “bash” “/tmp/chef-script20230824-748306-1j12bf6” ----
STDOUT:
STDERR:
---- End output of “bash” “/tmp/chef-script20230824-748306-1j12bf6” ----
Ran “bash” “/tmp/chef-script20230824-748306-1j12bf6” returned