[MIGRATION EE] Database migration error between 17.8.7 and 17.11.3

How to Use GitLab Upgrade
, , ,
1

Hello GitLab Community ! I have encounter a problem when migrating from GitLab EE 17.8.7 to any EE 17.9.x ou 17.11.x versions (following GitLab Upgrades plan).

I’ve tried to to the upgrade multiple times (with rollback to starting point) with no luck. Every time i try to do the upgrade to 17.11.3, the error occurs.

The error seems to be linked to a bad database migration and then keeps stuck with a no idempotent function inside the migration process.

Problem to solve

First stage

When trying to upgrade to new version, in the PGSQL database migration, an error appear on the logs:

main: == 20250130142309 AddInsertLfkDeletedRecordPartition: migrating ============
main: == [Advisory_lock_connection] object_id: 51680, pg_backend_pid: 545
STDERR:
---- End output of "bash" ----
Ran "bash" returend 1

No more information are displayed in the logs. Then, the GitLab docker restart himself, without letting time to debug the issue.

Second stage (after restart)

When the automatic restart occurs, the task migrate gitlab-rails database crash with a big log trace (i can send it if needed, but too much for init the thread IMO). the most pertinent element of it is:

PG::DuplicateFunction: ERROR: function "insert_into_loose_foreign_keys_deleted_records_override_table" already exists with the same argument types

Then, the instance loop restart itself with the same error.

Steps to reproduce

  • Follow GitLab Upgrade plan for EE, going from 16.10.9 to 18.0.1 (with and without zero downtime)
  • Tried with more or less time for background migrations (or between updates)
  • Tried upgrading to versions in-between 17.8.7 and 17.9.x
  • Tried debugging with no luck (container keeps restarting automatically)

Configuration

  • No UI can be shown in this state (NGINX service not started when bug occurs)
  • No major configuration changes in gitlab.rb (apart for some personalization on the UI)

Versions

  • Self-managed
  • GitLab.com SaaS
  • Dedicated

Versions

  • GitLab: 17.11.3-ee
  • DB Version: 14.17

I can give more logs or try some commands if needed. The instance i try to upgrade is our staging instance and can stay in this bad shape until a solution is found.

The problem still quite critical tho, because we can’t upgrade our Production GitLab to the latest version without expecting this bug to occurs…

Thanks in advance for your help !

2

So I also ran into this problem. Hopefully you have a full backup of 17.8.7 around. My solution was to move /opt/gitlab and /var/opt/gitlab into some temp folder. Reinstall 17.8.7, then do a restore from that full backup. Now that I’m happy the upgrade was clean, just getting ready to delete my backup of /opt/gitlab and /var/opt/gitlab

3

Hi,
we encountered the same issue on migrating fro 17.7.7 to 18.1.0
finally we execute the following DDL:

DROP FUNCTION insert_into_loose_foreign_keys_deleted_records_override_table

then we continued with the database import with positive outcome

regards,
massimiliano brenna

4

I ran into the identical issue restoring Gitlab 17.9 from a snapshot, no upgrade. Thank you!