Of the Background migrations, only RequireDeleteOrphannedPartitionedCiRunnerMachineRecords: ci_runner_machines_687967fa8a does not complete and does not progress from 0.0%.
The postgresql log keeps outputting the following logs.
postgresql_1 | 2025-03-21 10:35:04.214 UTC [115] ERROR: relation “ci_runner_machines_687967fa8a” does not exist at character 148
postgresql_1 | 2025-03-21 10:35:04.214 UTC [115] STATEMENT: SELECT a.attname
postgresql_1 | FROM (
postgresql_1 | SELECT indrelid, indkey, generate_subscripts(indkey, 1) idx
postgresql_1 | FROM pg_index
postgresql_1 | WHERE indrelid = ‘“ci_runner_machines_687967fa8a”’::regclass
postgresql_1 | AND indisprimary
postgresql_1 | ) i
postgresql_1 | JOIN pg_attribute a
postgresql_1 | ON a.attrelid = i.indrelid
postgresql_1 | AND a.attnum = i.indkey[i.idx]
postgresql_1 | ORDER BY i.idx
I tried to revert to 17.8.0 but got an error that I could not downgrade, so I would like to know what the solution is.
Depending on how large your Gitlab installation and data is, migrations can take a long time. Usually you should just leave it for at least 24 hours before attempting to revert or anything like that since you can make things even worse.
I also upgraded 3 of my servers last night, one test which is empty practically and so migrations happened quick. The other production servers took longer. It varies on the amount of data on how long these will take.
Hello! We upgraded our self-hosted Gitlab (docker) from 17.8.5 to 17.10.0 and now it’s already more than 24 hours for use and migration still keeps at 0.00% in Active state.
Is there any way to check if migration actually running, maybe it just stalled? Thanks.
Hello everyone. We also ran into the same problem. Upgrade from 17.8.5 to 17.10.0. RequeueDeleteOrphanedPartitionedCiRunnerMachineRecords: ci_runner_machines_687967fa8a is Active but is always at 0.00%. In the DB and sidekiq logs, we can see the same error:
2025-03-23 12:50:03.669 UTC [6217] ERROR: relation "ci_runner_machines_687967fa8a" does not exist at character 148
2025-03-23 12:50:03.669 UTC [6217] STATEMENT: SELECT a.attname
FROM (
SELECT indrelid, indkey, generate_subscripts(indkey, 1) idx
FROM pg_index
WHERE indrelid = '"ci_runner_machines_687967fa8a"'::regclass
AND indisprimary
) i
JOIN pg_attribute a
ON a.attrelid = i.indrelid
AND a.attnum = i.indkey[i.idx]
ORDER BY i.idx
EDIT: We wanted to add that we did use this workaround from GitLab’s support website so that a migration that was failing when we upgraded to 17.8.5, BackfillCiRunnerMachinesPartitionedTable, could finish in the UI.
EDIT 2: We ended up downgrading to 17.8.5 (by restoring a backup we had). We then updated to 17.9.2 which added the RequeueDeleteOrphanedPartitionedCiRunnerMachineRecords: ci_runner_machines_687967fa8a migration, but it actually completed this time. No errors on the DB logs either!
We also upgraded a self-hosted GitLab from 17.8.2 to 17.10.0 and have the same Background Migration sitting at 0.0% and it has been a bit over 24 hours.
As janisii mentioned, is there a way to check if it is actually running?
Thanks.
Same here, Backgroud Job is stuck and (as the original poster pointed out) seems to be related to the issue that the relation ‘ci_runner_machines_687967fa8a’ cannot be found (which indeed does not seem to be in our database).
Maybe a missing check if that relation really exists?
Looks to me as if there is nothing to delete so the migration is redundant…?
Is it possible to mark the Migration as done manually?
New forum users need a few more posts before links start working - due to the spammers they ruined all that which required more restrictions putting in place.