Hello everyone,
I have the following problem
Gitlab instance - self-hosted, Ubuntu, Community Edition
After updating to 17.7.1 (17.7.0), 12 background migration jobs are stuck. Status - active, progress 0%. Nothing happens (for more than a day).
When I try to finalise the job using ‘gitlab-rake’, I get a ‘Done’. I interpret that as OK.
sudo gitlab-rake gitlab:background_migrations:finalize[BackfillSeatAssignmentsTable,members,id,‘[[]]’]
Done.
However, nothing changes in the overview of the background migrations in the UI.
The database query with ‘gitlab-psql’ still lists this job as not completed.
What else can I do? Any ideas?
Thank you,
Thomas
Addendum.
I checked the status of the migration jobs with the command ‘gitlab-rake db:migrate:status’. All are marked as ‘up’.
Then I checked each job that is marked as ‘active’ and ‘0%’ progress in the Gitlab UI (i.e. ‘not yet finished’). To do this, I looked at the respective job code and checked the status list to see if this job had run successfully. They are all OK.
In principle, I would still have to check whether the respective changes were made to the database. With 12 jobs, this is unfortunately quite time-consuming.
I wonder if I couldn’t just mark these 12 jobs as completed manually now…
Addendum II
Fortunately, the problem was not a bug but a carelessness on our part.
We recently migrated the Gitlab instance (basically the entire dataset) to a new system (Ubuntu). Beforehand, we had deactivated all pipelines and jobs on the target system, including the system’s background jobs.
After the migration, we initially only activated some of the jobs to see how the new (migrated) system behaves. And that’s what we did.
I came to this point when I analysed why scheduled pipelines were not starting. After I had activated the necessary system jobs, the ‘hanging’ background migration jobs ran in no time at all. Just as the scheduled pipelines started at the scheduled time.
The question is thus closed.