Failed Upgrade from v13.12.15 to v14

Hi,

I am trying to upgrade from version 13.12.15 to version 14. According to the upgrade path, I can upgrade to 14.0.12. But the upgrade fails due to the iterations. I tried to revert and do 14.0.0 and 14.0.5 but still the upgrade fails with the same issue. I already have already migrated all my projects to hashed on version 13.12.15 as well as my postgresql was version 12.6.

This was the error that I experienced:

StandardError: An error has occurred, all later migrations canceled:

PG::UndefinedTable: ERROR:  relation "postgres_indexes" does not exist
LINE 2:         FROM postgres_indexes
                     ^
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20210611100359_rebuild_index_for_cadence_iterations_automation.rb:31:in `index_exists_and_is_valid?'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20210611100359_rebuild_index_for_cadence_iterations_automation.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'

Caused by:
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "postgres_indexes" does not exist
LINE 2:         FROM postgres_indexes
                     ^
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20210611100359_rebuild_index_for_cadence_iterations_automation.rb:31:in `index_exists_and_is_valid?'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20210611100359_rebuild_index_for_cadence_iterations_automation.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'

Caused by:
PG::UndefinedTable: ERROR:  relation "postgres_indexes" does not exist
LINE 2:         FROM postgres_indexes
                     ^
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20210611100359_rebuild_index_for_cadence_iterations_automation.rb:31:in `index_exists_and_is_valid?'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20210611100359_rebuild_index_for_cadence_iterations_automation.rb:11:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)

-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- add_index(:ci_pending_builds, :id, {:where=>"protected = true", :name=>"index_ci_pending_builds_id_on_protected_partial", :algorithm=>:concurrently})
   -> 0.0015s
-- execute("RESET ALL")
   -> 0.0001s
== 20210610113229 AddIndexToProtectedPendingBuilds: migrated (0.0037s) ========

== 20210610141711 DisableExpirationPoliciesLinkedToNoContainerImages: migrating
-- execute("        WITH batched_relation AS MATERIALIZED (SELECT \"container_expiration_policies\".* FROM \"container_expiration_policies\" WHERE \"container_expiration_policies\".\"enabled\" = TRUE AND \"container_expiration_policies\".\"project_id\" >= 33645 LIMIT 1000)\n        UPDATE container_expiration_policies\n        SET enabled = FALSE\n        FROM batched_relation\n        WHERE container_expiration_policies.project_id = batched_relation.project_id\n        AND NOT EXISTS (SELECT 1 FROM \"container_repositories\" WHERE container_repositories.project_id = container_expiration_policies.project_id)\n")
   -> 0.0118s
== 20210610141711 DisableExpirationPoliciesLinkedToNoContainerImages: migrated (0.0204s)

== 20210611080951 FixMissingTraversalIds: migrating ===========================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:namespaces, :id, {:where=>"parent_id IS NULL AND traversal_ids = '{}'", :name=>"tmp_index_namespaces_empty_traversal_ids_with_root_namespaces", :algorithm=>:concurrently})
   -> 0.0063s
-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- add_index(:namespaces, :id, {:where=>"parent_id IS NULL AND traversal_ids = '{}'", :name=>"tmp_index_namespaces_empty_traversal_ids_with_root_namespaces", :algorithm=>:concurrently})
   -> 0.0022s
-- execute("RESET ALL")
   -> 0.0001s
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:namespaces, :id, {:where=>"parent_id IS NOT NULL AND traversal_ids = '{}'", :name=>"tmp_index_namespaces_empty_traversal_ids_with_child_namespaces", :algorithm=>:concurrently})
   -> 0.0059s
-- execute("SET statement_timeout TO 0")
   -> 0.0001s
-- add_index(:namespaces, :id, {:where=>"parent_id IS NOT NULL AND traversal_ids = '{}'", :name=>"tmp_index_namespaces_empty_traversal_ids_with_child_namespaces", :algorithm=>:concurrently})
   -> 0.0018s
-- execute("RESET ALL")
   -> 0.0001s
-- Scheduled 0 BackfillNamespaceTraversalIdsRoots jobs with a maximum of 1000 records per batch and an interval of 120 seconds.

The migration is expected to take at least 0 seconds. Expect all jobs to have completed after 2022-06-05 06:49:48 +0800."
-- Scheduled 0 BackfillNamespaceTraversalIdsChildren jobs with a maximum of 1000 records per batch and an interval of 120 seconds.

The migration is expected to take at least 120 seconds. Expect all jobs to have completed after 2022-06-05 06:51:48 +0800."
== 20210611080951 FixMissingTraversalIds: migrated (0.0703s) ==================

== 20210611100359 RebuildIndexForCadenceIterationsAutomation: migrating =======
-- execute("        SELECT identifier\n        FROM postgres_indexes\n        WHERE identifier LIKE '%cadence_create_iterations_automation' AND valid_index=TRUE\n")
STDERR:
---- End output of "bash"  "/tmp/chef-script20220604-11035-a2uvah" ----
Ran "bash"  "/tmp/chef-script20220604-11035-a2uvah" returned 1

As shown here, postgresql version is now 12.6 and no legacy projects or attachments are existing:

* Found 0 projects using Legacy Storage
root@xxx:~# gitlab-rake gitlab:storage:list_legacy_attachments
* Found 0 attachments using Legacy Storage
root@xxx:~# sudo gitlab-rake gitlab:env:info

System information
System:
Current User:	git
Using RVM:	no
Ruby Version:	2.7.2p137
Gem Version:	3.1.4
Bundler Version:2.1.4
Rake Version:	13.0.3
Redis Version:	6.0.14
Git Version:	2.31.1
Sidekiq Version:5.2.9
Go Version:	unknown

GitLab information
Version:	13.12.15
Revision:	6614e1562c6
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	12.6
URL:		http://xxx.xx
HTTP Clone URL:	http://xxx.xx/some-group/some-project.git
SSH Clone URL:	git@xxx.xx:some-group/some-project.git
Using LDAP:	yes
Using Omniauth:	yes
Omniauth Providers:

GitLab Shell
Version:	13.18.1
Repository storage paths:
- default: 	/var/opt/gitlab/git-data/repositories
GitLab Shell path:		/opt/gitlab/embedded/service/gitlab-shell
Git:		/opt/gitlab/embedded/bin/git

I got pass this error by manually creating the postgres_indexes while referring to the ruby script on:

/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20200922093004_add_postgres_index_view.rb

1 Like