Upgrade Problems Debian 14.2.4

Using the correct command rails console command helped:

sudo -u git -H bundle exec rails console -e production

Then executing:

scheduled_queue = Sidekiq::ScheduledSet.new
pending_job_classes = scheduled_queue.select { |job| job["class"] == "BackgroundMigrationWorker" }.map { |job| job["args"].first }.uniq
pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) }

Appears to successfully finalize the background migrations as running

sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::BackgroundMigration.remaining'

results in 0.

However, I still get the same error when attempting to finish the migration. Seems I still need to run the equivalent of this in the rails console:

gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_build_needs,id,'[["build_id"]\, ["build_id_convert_to_bigint"]]']

Not sure how to do this. Anyone have a suggestion? Beginning to think I need to downgrade to 14.1

Okay, seems like my issue is impatience. After 6 hours the background migrations are still running as seen in the GitLab Admin Monitoring area. Just not showing on the command line.

Hi,

I’ve just ran into the same problem as described above. (I’m on CentOS 7, using the yum repo). I was coming from 13.x, so first try the yum error told me to upgrade to 14.0.x (latest), I did, then I just ran the next without specifying the version, so it jumped to 14.5.x, but had this error, the whole instance didn’t start. After like 2hrs of tries, while it kept asking for the same thing all the time:

gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']

I could finally could run the below:

gitlab-rake db:migrate
gitlab-ctl reconfigure

I’m not too sure what fixed it (maybe gitlab-rake gitlab:check SANITIZE=true, but I ran a lot of commands, had 0 in the queue), but it looks ok now. Can you please add a check to the RPMs? So it won’t let us upgrade from 14.0.x → 14.5, but should prompt for the correct journey, 14.1.x…

Thanks!

Any update on this please? Should I report in somewhere as a bug?

You didn’t follow the upgrade path, you jumped. You will have to go back to 14.0.x, restore the backup and then start the upgrade properly following the upgrade path outlined in the documentation as well as earlier in this post: Upgrade Problems Debian 14.2.4 - #7 by iwalker

此问题我已经解决，并且我已经成功升级到14.5.2
我的升级路线为
10.7->10.8.7 → 11.11.8 → 12.0.12 → 12.1.17 → 12.10.14 → 13.0.14 → 13.1.11 → 13.8.8
→ 13.9.2-> 13.12.12 → 14.0.11 → 14.1.6->14.1.7->14.2.1->14.2.5->14.2.6->14.2.7-> 14.3.4->14.4.3->14.5.2

14.2.X 升级到14.3.X 的时候需要更新PG 的数据，但是需要升级到14.2.7 或者14.2.6 进入容器
此时应该运行14.2.7 docker exec -ti gitlab
gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,push_event_payloads,event_id,’[[“event_id”], [“event_id_convert_to_bigint”]]’]
gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_stages,id,’[[“id”], [“id_convert_to_bigint”]]’]

gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,’[[“id”], [“id_convert_to_bigint”]]’]

时间会比较慢，但是需要耐心等待，在14.2.7 上执行成功之后，在升级14.3 或者14.5 版本就可以了，希望帮助到大家

Hello, I ran into the same problem. my migration path was:
12.10.14 → 13.0.14 → 13.1.11 → 13.8.8 → 13.12.15 → 14.0.11 → 14.1.8 → 14.2.6
like recommended on Upgrading GitLab | GitLab

During the update from 14.1.8 to 14.2.6 i got the Problem
I am running gitlab in docker.

Is there any update for this issue?

I solved the problem for me. I started gitlab with version 14.1.8 and used the following commands from Batched background migrations | GitLab
$ gitlab-rake db:migrate
$ gitlab-ctl reconfigure
inside my docker container.
After that i changed the version to 14.2.6, it took about a minute and everything seems fine

I’m facing the same issue. I created a forum post a while back describing my situation but I have yet to receive any help.

I’m having the same issue with an Omnibus installation. I’ve attempted the smallest upgrade from Docker image gitlab/gitlab-ce:14.1.8-ce.0 to gitlab/gitlab-ce:14.2.0-ce.0, I’ve also tried a few other releases in the 14.2.x range, however the result is the same.

The Docker container restarts every 30 seconds or so (before the database migrations can complete). The logs indicate the following:

n error has occurred, all later migrations canceled:

Expected batched background migration for the given configuration to be marked as ‘finished’, but it is ‘finalizing’: {:job_class_name=>“CopyColumnUsingBackgroundMigrationJob”, :table_name=>“ci_job_artifacts”, :column_name=>“id”, :job_arguments=>[[“id”, “job_id”], [“id_convert_to_bigint”, “job_id_convert_to_bigint”]]}

Finalize it manualy by running

sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_job_artifacts,id,‘[[“id”, “job_id”], [“id_convert_to_bigint”, “job_id_convert_to_bigint”]]’]

For more information, check the documentation

Background migrations and upgrades | GitLab

However 30 seconds is insufficient to execute the command.

Any advice on how to get past this?

I was stucked with14.1.8 to 14.2.x, reading above, I came with the commands which finally works for me.

Run with 14.1.8, inside the docker and run the commands before upgrade to 14.2.1

gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']
gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining'
gitlab-rake gitlab:check SANITIZE=true
gitlab-rake db:migrate
gitlab-ctl reconfigure

My upgrade path

14.1.8 (run with above commands) → 14.2.1 → 14.2.5 → 14.3.1 → 14.3.6 → 14.6.2 →
14.9.5 → 14.10.5 → 15.0.2 → 15.1.0 (latest major version at the moment)

Thank you @weisunding , I confirm that the step from 14.1.8 and the upgrade path works. You are the best