Upgrade failed 17.8.2 to 17.10.3

Community
1

I just tried to upgrade gitlab-ce from 17.8.2 to 17.10.3 without success. I get this error:

[2025-04-03T17:02:20+02:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2025-04-03T17:02:20+02:00] FATAL: ---------------------------------------------------------------------------------------
[2025-04-03T17:02:20+02:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2025-04-03T17:02:20+02:00] FATAL: ---------------------------------------------------------------------------------------
[2025-04-03T17:02:20+02:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash"  ----
STDOUT: rake aborted!
StandardError: An error has occurred, all later migrations canceled:

uninitialized constant Gitlab::BackgroundMigration::BackfillCiRunnerMachinesPartitionedTable
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration.rb:228:in `job_class'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration.rb:157:in `cursor?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration.rb:214:in `next_min_value'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:103:in `find_next_batch_range'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:94:in `find_or_create_next_batched_job'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:29:in `run_migration_job'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:146:in `run_migration_while'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:80:in `finalize'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:10:in `finalize'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:160:in `block (2 levels) in finalize_batched_background_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzers/base.rb:52:in `with_suppressed'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:159:in `block in finalize_batched_background_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/reestablished_connection_stack.rb:21:in `with_restored_connection_stack'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:158:in `finalize_batched_background_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:249:in `ensure_batched_background_migration_is_finished'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20250113163026_finalize_backfill_ci_runner_machines_partitioned_table.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:in `block in exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:83:in `within'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:21:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:54:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/runner_backoff/active_record_mixin.rb:21:in `execute_migration_in_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:28:in `block in with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:25:in `with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:145:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:112:in `configure_pg_databases'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:99:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'

Caused by:
NameError: uninitialized constant Gitlab::BackgroundMigration::BackfillCiRunnerMachinesPartitionedTable
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration.rb:228:in `job_class'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration.rb:157:in `cursor?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration.rb:214:in `next_min_value'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:103:in `find_next_batch_range'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:94:in `find_or_create_next_batched_job'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:29:in `run_migration_job'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:146:in `run_migration_while'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:80:in `finalize'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:10:in `finalize'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:160:in `block (2 levels) in finalize_batched_background_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzers/base.rb:52:in `with_suppressed'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:159:in `block in finalize_batched_background_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/reestablished_connection_stack.rb:21:in `with_restored_connection_stack'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:158:in `finalize_batched_background_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:249:in `ensure_batched_background_migration_is_finished'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20250113163026_finalize_backfill_ci_runner_machines_partitioned_table.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:in `block in exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:83:in `within'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:21:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:54:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/runner_backoff/active_record_mixin.rb:21:in `execute_migration_in_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:28:in `block in with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:25:in `with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:145:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:112:in `configure_pg_databases'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:99:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
Running db:migrate rake task
main: == [advisory_lock_connection] object_id: 46740, pg_backend_pid: 2699917
main: == 20250113163026 FinalizeBackfillCiRunnerMachinesPartitionedTable: migrating =
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: == [advisory_lock_connection] object_id: 46740, pg_backend_pid: 2699917
STDERR: 
---- End output of "bash"  ----
Ran "bash"  returned 1

I tried to follow workaround steps mentioned here. but that doesn’t work.

I also opened an issue

Question2: I tried to install a backup, but I need to downgrade and launch gitlabctl-reconfigure, witch fails so I can’t install a backup. Any idea on the way to go back to previsou version / backup?

1 Like
2

same issue here

3

Also facing the same issue on the same upgrade version

4

According to the upgrade path tool, you are supposed to upgrade to 17.8.6 first. See: Upgrade Path

Downgrades are almost always impossible. Restore system, or clean install Gitlab 17.8.2 then restore backup, and then attempt the upgrade again correctly by following the upgrade path.

5

same issue here - upgraded from 17.8.6

6

Issue has been opened by someone, you can follow it there since Gitlab Devs don’t read forum posts: upgrade to 17.10.3 fails : uninitialized constant Gitlab::BackgroundMigration::BackfillCiRunnerMachinesPartitionedTable (#534087) · Issues · GitLab.org / GitLab · GitLab