Update gitlab-ce 17.2.1 to 17.11.0 error in 17.5.5

How to Use GitLab Upgrade
1

Hello everyone, I now need to upgrade GitLab CE from version 17.2.1 to 17.11.0. I followed the update path for the upgrade process. Everything went smoothly when upgrading to 17.3.7, but I encountered an error while upgrading to 17.5.5.

ENV

  • Self-managed
  • host is a virtual machine.
  • Operating system is Rocky Linux 9.5.

error log

[2025-04-18T13:48:25+08:00] ERROR: Running exception handlers
There was an error running gitlab-ctl reconfigure:

rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash"  ----
STDOUT: rake aborted!
StandardError: An error has occurred, all later migrations canceled:

invalid date
/opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:163:in `block in redis'
/opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:160:in `redis'
/opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq.rb:73:in `redis'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:23:in `block (2 levels) in up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_sharding/validator.rb:31:in `allow_unrouted_sidekiq_calls'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:22:in `block in up'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:21:in `each'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:21:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:in `block in exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:83:in `within'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:21:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:54:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/runner_backoff/active_record_mixin.rb:21:in `execute_migration_in_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:28:in `block in with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:25:in `with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:138:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:107:in `configure_pg_databases'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:94:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'

Caused by:
Date::Error: invalid date
/opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:163:in `block in redis'
/opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:160:in `redis'
/opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq.rb:73:in `redis'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:23:in `block (2 levels) in up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_sharding/validator.rb:31:in `allow_unrouted_sidekiq_calls'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:22:in `block in up'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:21:in `each'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:21:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:in `block in exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:83:in `within'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:21:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:54:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/runner_backoff/active_record_mixin.rb:21:in `execute_migration_in_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:28:in `block in with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:25:in `with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:138:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:107:in `configure_pg_databases'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:94:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'

Caused by:
Date::Error: invalid date
/opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:163:in `block in redis'
/opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:160:in `redis'
/opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq.rb:73:in `redis'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:23:in `block (2 levels) in up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_sharding/validator.rb:31:in `allow_unrouted_sidekiq_calls'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:22:in `block in up'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:21:in `each'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:21:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:in `block in exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:83:in `within'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:21:in `exec_migration'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:54:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/runner_backoff/active_record_mixin.rb:21:in `execute_migration_in_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:28:in `block in with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:25:in `with_advisory_lock_connection'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:138:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:107:in `configure_pg_databases'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:94:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
Running db:migrate rake task
main: == [advisory_lock_connection] object_id: 49460, pg_backend_pid: 5599
main: == 20240920072729 RemoveDanglingCronWorker: migrating =========================
main: == [advisory_lock_connection] object_id: 49460, pg_backend_pid: 5599
STDERR:
---- End output of "bash"  ----
Ran "bash"  returned 1

I can run gitlab-ctl stop and gitlab-ctl start, but I am unable to complete gitlab-ctl reconfigure. The system appears to be functioning normally at the moment, but I am concerned that other issues might arise. As a precaution, I have reverted to version 17.3.7 using a snapshot.

How should I fix it and continue with the upgrade?

2

I have try

 # gitlab-rails console
--------------------------------------------------------------------------------
 Ruby:         ruby 3.2.5 (2024-07-26 revision 31d0f1a2e7) [x86_64-linux]
 GitLab:       17.5.5 (c8c89100f5b) FOSS
 GitLab Shell: 14.39.0
 PostgreSQL:   14.11
------------------------------------------------------------[ booted in 44.12s ]
Loading production environment (Rails 7.0.8.4)
irb(main):001:0> require './db/post_migrate/20240920072729_remove_dangling_cron_worker.rb'
irb(main):002:0> RemoveDanglingCronWorker.new.up
/opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:631:in `parse': invalid date (Date::Error)

        DateTime.parse(timestamp).to_time.utc
                       ^^^^^^^^^
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:631:in `rescue in parse_enqueue_time'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:628:in `parse_enqueue_time'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:290:in `initialize'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:247:in `new'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:247:in `block in find'
        from /opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:163:in `block in redis'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:110:in `block (2 levels) in with'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:109:in `handle_interrupt'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:109:in `block in with'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:106:in `handle_interrupt'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:106:in `with'
        from /opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:160:in `redis'
        from /opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq.rb:73:in `redis'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:246:in `find'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:262:in `destroy'
        from /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:23:in `block (2 levels) in up'
        ... 17 levels...
/opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:629:in `strptime': invalid date (Date::Error)

        DateTime.strptime(timestamp, LAST_ENQUEUE_TIME_FORMAT).to_time.utc
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:629:in `parse_enqueue_time'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:290:in `initialize'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:247:in `new'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:247:in `block in find'
        from /opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:163:in `block in redis'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:110:in `block (2 levels) in with'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:109:in `handle_interrupt'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:109:in `block in with'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:106:in `handle_interrupt'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/connection_pool-2.4.1/lib/connection_pool.rb:106:in `with'
        from /opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq/config.rb:160:in `redis'
        from /opt/gitlab/embedded/service/gitlab-rails/vendor/gems/sidekiq-7.2.4/lib/sidekiq.rb:73:in `redis'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:246:in `find'
        from /opt/gitlab/embedded/lib/ruby/gems/3.2.0/gems/sidekiq-cron-1.12.0/lib/sidekiq/cron/job.rb:262:in `destroy'
        from /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240920072729_remove_dangling_cron_worker.rb:23:in `block (2 levels) in up'
        from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_sharding/validator.rb:31:in `allow_unrouted_sidekiq_calls'
        ... 16 levels...
irb(main):003:0>