Using the correct command rails console command helped:
sudo -u git -H bundle exec rails console -e production
Then executing:
scheduled_queue = Sidekiq::ScheduledSet.new
pending_job_classes = scheduled_queue.select { |job| job["class"] == "BackgroundMigrationWorker" }.map { |job| job["args"].first }.uniq
pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) }
Appears to successfully finalize the background migrations as running
sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::BackgroundMigration.remaining'
results in 0.
However, I still get the same error when attempting to finish the migration. Seems I still need to run the equivalent of this in the rails console:
gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_build_needs,id,'[["build_id"]\, ["build_id_convert_to_bigint"]]']
Not sure how to do this. Anyone have a suggestion? Beginning to think I need to downgrade to 14.1