Replace this template with your information
Describe your question in as much detail as possible:
I checked for background migrations, checked GUI and services were running and captured rake check output which were all fine but I hit the following error during a 14.0.12 > 14.3.6 upgrade
System Info:
------------
chef_version=15.17.4
platform=centos
platform_version=7.8.2003
ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client
Recipe: gitlab::selinux
- bash[Set proper security context on ssh files for selinux] action run
- execute “bash” “/tmp/chef-script20220824-67414-ltyr7i”
Recipe: gitlab::gitlab-rails
- execute “bash” “/tmp/chef-script20220824-67414-ltyr7i”
- execute[clear the gitlab-rails cache] action run
- execute /opt/gitlab/bin/gitlab-rake cache:clear
Recipe: gitaly::enable
- execute /opt/gitlab/bin/gitlab-rake cache:clear
- runit_service[gitaly] action restart (up to date)
- runit_service[gitaly] action hup
- send hup to runit_service[gitaly]
Running handlers:
There was an error running gitlab-ctl reconfigure:
rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘1’
---- Begin output of “bash” “/tmp/chef-script20220824-67414-19dm62j” ----
STDOUT: == 20210317210338 AddValidRunnerRegistrars: migrating =========================
– add_column(:application_settings, :valid_runner_registrars, :string, {:array=>true, :default=>[“project”, “group”]})
→ 0.0056s
== 20210317210338 AddValidRunnerRegistrars: migrated (0.0056s) ================
== 20210601132134 RemovePartialIndexForHashedStorageMigration: migrating ======
– transaction_open?()
→ 0.0000s
– index_exists?(:projects, :id, {:name=>“index_on_id_partial_with_legacy_storage”, :algorithm=>:concurrently})
→ 0.0290s
– execute(“SET statement_timeout TO 0”)
→ 0.0003s
– remove_index(:projects, {:name=>“index_on_id_partial_with_legacy_storage”, :algorithm=>:concurrently, :column=>:id})
→ 0.0516s
– execute(“RESET statement_timeout”)
→ 0.0004s
== 20210601132134 RemovePartialIndexForHashedStorageMigration: migrated (0.0826s)
== 20210602155056 AddMergeRequestDiffCommitUsers: migrating ===================
– create_table(:merge_request_diff_commit_users, {:id=>:bigint})
– quote_column_name(:name)
→ 0.0000s
– quote_column_name(:email)
→ 0.0000s
→ 0.0139s
– quote_table_name(“check_147358fc42”)
→ 0.0000s
– quote_table_name(“check_f5fa206cf7”)
→ 0.0000s
– quote_table_name(:merge_request_diff_commit_users)
→ 0.0000s
– execute(“ALTER TABLE "merge_request_diff_commit_users"\nADD CONSTRAINT "check_147358fc42" CHECK (char_length("name") <= 512),\nADD CONSTRAINT "check_f5fa206cf7" CHECK (char_length("email") <= 512)\n”)
→ 0.0007s
– transaction_open?()
→ 0.0000s
– current_schema()
→ 0.0002s
– execute(“ALTER TABLE merge_request_diff_commit_users\nADD CONSTRAINT merge_request_diff_commit_users_name_or_email_existence\nCHECK ( (COALESCE(name, ‘’) != ‘’) OR (COALESCE(email, ‘’) != ‘’) )\nNOT VALID;\n”)
→ 0.0005s
– current_schema()
→ 0.0002s
– execute(“SET statement_timeout TO 0”)
→ 0.0002s
– execute(“ALTER TABLE merge_request_diff_commit_users VALIDATE CONSTRAINT merge_request_diff_commit_users_name_or_email_existence;”)
→ 0.0008s
– execute(“RESET statement_timeout”)
→ 0.0002s
== 20210602155056 AddMergeRequestDiffCommitUsers: migrated (0.0460s) ==========
== 20210602155110 AddMergeRequestDiffCommitUserColumns: migrating =============
– add_column(:merge_request_diff_commits, :commit_author_id, :bigint)
→ 0.0008s
– add_column(:merge_request_diff_commits, :committer_id, :bigint)
→ 0.0004s
== 20210602155110 AddMergeRequestDiffCommitUserColumns: migrated (0.0013s) ====
== 20210602164044 ScheduleLatestPipelineIdPopulation: migrating ===============
== 20210602164044 ScheduleLatestPipelineIdPopulation: migrated (0.0000s) ======
== 20210604032738 CreateDastSiteProfilesBuilds: migrating =====================
– create_table(:dast_site_profiles_builds, {:primary_key=>[:dast_site_profile_id, :ci_build_id], :comment=>“{"owner":"group::dynamic analysis","description":"Join table between DAST Site Profiles and CI Builds"}”})
→ 0.0083s
== 20210604032738 CreateDastSiteProfilesBuilds: migrated (0.0085s) ============
== 20210604034158 AddCiBuildIdFkToDastSiteProfilesBuilds: migrating ===========
– transaction_open?()
→ 0.0000s
– foreign_keys(:dast_site_profiles_builds)
→ 0.0066s
– execute(“ALTER TABLE dast_site_profiles_builds\nADD CONSTRAINT fk_a325505e99\nFOREIGN KEY (ci_build_id)\nREFERENCES ci_builds (id)\nON DELETE CASCADE\nNOT VALID;\n”)
→ 0.0015s
– execute(“SET statement_timeout TO 0”)
→ 0.0002s
– execute(“ALTER TABLE dast_site_profiles_builds VALIDATE CONSTRAINT fk_a325505e99;”)
→ 0.0030s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210604034158 AddCiBuildIdFkToDastSiteProfilesBuilds: migrated (0.0165s) ==
== 20210604034354 AddDastSiteProfileIdFkToDastSiteProfilesBuilds: migrating ===
– transaction_open?()
→ 0.0000s
– foreign_keys(:dast_site_profiles_builds)
→ 0.0059s
– execute(“ALTER TABLE dast_site_profiles_builds\nADD CONSTRAINT fk_94e80df60e\nFOREIGN KEY (dast_site_profile_id)\nREFERENCES dast_site_profiles (id)\nON DELETE CASCADE\nNOT VALID;\n”)
→ 0.0010s
– execute(“SET statement_timeout TO 0”)
→ 0.0002s
– execute(“ALTER TABLE dast_site_profiles_builds VALIDATE CONSTRAINT fk_94e80df60e;”)
→ 0.0035s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210604034354 AddDastSiteProfileIdFkToDastSiteProfilesBuilds: migrated (0.0157s)
== 20210604051330 CreateDastScannerProfilesBuilds: migrating ==================
– create_table(:dast_scanner_profiles_builds, {:primary_key=>[:dast_scanner_profile_id, :ci_build_id], :comment=>“{"owner":"group::dynamic analysis","description":"Join table between DAST Scanner Profiles and CI Builds"}”})
→ 0.0054s
== 20210604051330 CreateDastScannerProfilesBuilds: migrated (0.0055s) =========
== 20210604051742 AddCiBuildIdFkToDastScannerProfilesBuilds: migrating ========
– transaction_open?()
→ 0.0000s
– foreign_keys(:dast_scanner_profiles_builds)
→ 0.0055s
– execute(“ALTER TABLE dast_scanner_profiles_builds\nADD CONSTRAINT fk_e4c49200f8\nFOREIGN KEY (ci_build_id)\nREFERENCES ci_builds (id)\nON DELETE CASCADE\nNOT VALID;\n”)
→ 0.0009s
– execute(“SET statement_timeout TO 0”)
→ 0.0003s
– execute(“ALTER TABLE dast_scanner_profiles_builds VALIDATE CONSTRAINT fk_e4c49200f8;”)
→ 0.0023s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210604051742 AddCiBuildIdFkToDastScannerProfilesBuilds: migrated (0.0136s)
== 20210604051917 AddDastScannerProfileIdFkToDastScannerProfilesBuilds: migrating
– transaction_open?()
→ 0.0000s
– foreign_keys(:dast_scanner_profiles_builds)
→ 0.0053s
– execute(“ALTER TABLE dast_scanner_profiles_builds\nADD CONSTRAINT fk_5d46286ad3\nFOREIGN KEY (dast_scanner_profile_id)\nREFERENCES dast_scanner_profiles (id)\nON DELETE CASCADE\nNOT VALID;\n”)
→ 0.0008s
– execute(“SET statement_timeout TO 0”)
→ 0.0003s
– execute(“ALTER TABLE dast_scanner_profiles_builds VALIDATE CONSTRAINT fk_5d46286ad3;”)
→ 0.0035s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210604051917 AddDastScannerProfileIdFkToDastScannerProfilesBuilds: migrated (0.0145s)
== 20210604133651 ScheduleMergeRequestDiffUsersBackgroundMigration: migrating =
== 20210604133651 ScheduleMergeRequestDiffUsersBackgroundMigration: migrated (0.0000s)
== 20210609202501 ScheduleBackfillDraftStatusOnMergeRequests: migrating =======
== 20210609202501 ScheduleBackfillDraftStatusOnMergeRequests: migrated (0.0000s)
== 20210610042700 RemoveClustersApplicationsFluentdTable: migrating ===========
– drop_table(:clusters_applications_fluentd)
→ 0.0160s
== 20210610042700 RemoveClustersApplicationsFluentdTable: migrated (0.0161s) ==
== 20210610153556 DeleteLegacyOperationsFeatureFlags: migrating ===============
– execute(“DELETE FROM operations_feature_flags WHERE version = 1”)
→ 0.0047s
== 20210610153556 DeleteLegacyOperationsFeatureFlags: migrated (0.0048s) ======
== 20210611082822 AddPagesFileEntriesToPlanLimits: migrating ==================
– add_column(:plan_limits, :pages_file_entries, :integer, {:default=>200000, :null=>false})
→ 0.0019s
== 20210611082822 AddPagesFileEntriesToPlanLimits: migrated (0.0020s) =========
== 20210611101034 AddDevopsAdoptionSastDast: migrating ========================
– add_column(:analytics_devops_adoption_snapshots, :sast_enabled_count, :integer)
→ 0.0009s
– add_column(:analytics_devops_adoption_snapshots, :dast_enabled_count, :integer)
→ 0.0005s
== 20210611101034 AddDevopsAdoptionSastDast: migrated (0.0014s) ===============
== 20210614124111 AddDevopsAdoptionSastDastIndexes: migrating =================
– transaction_open?()
→ 0.0000s
– index_exists?(:ci_job_artifacts, [:project_id, :created_at], {:where=>“file_type = 5”, :name=>“index_ci_job_artifacts_sast_for_devops_adoption”, :algorithm=>:concurrently})
→ 0.0055s
– execute(“SET statement_timeout TO 0”)
→ 0.0003s
– add_index(:ci_job_artifacts, [:project_id, :created_at], {:where=>“file_type = 5”, :name=>“index_ci_job_artifacts_sast_for_devops_adoption”, :algorithm=>:concurrently})
→ 0.0167s
– execute(“RESET statement_timeout”)
→ 0.0003s
– transaction_open?()
→ 0.0000s
– index_exists?(:ci_job_artifacts, [:project_id, :created_at], {:where=>“file_type = 8”, :name=>“index_ci_job_artifacts_dast_for_devops_adoption”, :algorithm=>:concurrently})
→ 0.0050s
– execute(“SET statement_timeout TO 0”)
→ 0.0003s
– add_index(:ci_job_artifacts, [:project_id, :created_at], {:where=>“file_type = 8”, :name=>“index_ci_job_artifacts_dast_for_devops_adoption”, :algorithm=>:concurrently})
→ 0.0126s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210614124111 AddDevopsAdoptionSastDastIndexes: migrated (0.0440s) ========
== 20210614142311 AddRunningContainerScanningMaxSizeToPlanLimits: migrating ===
– add_column(:plan_limits, :ci_max_artifact_size_running_container_scanning, :integer, {:null=>false, :default=>0})
→ 0.0016s
== 20210614142311 AddRunningContainerScanningMaxSizeToPlanLimits: migrated (0.0017s)
== 20210615064342 AddIssueIdToRequirement: migrating ==========================
– add_column(:requirements, :issue_id, :bigint, {:null=>true})
→ 0.0007s
== 20210615064342 AddIssueIdToRequirement: migrated (0.0041s) =================
== 20210615234935 FixBatchedMigrationsOldFormatJobArguments: migrating ========
== 20210615234935 FixBatchedMigrationsOldFormatJobArguments: migrated (0.0270s)
== 20210616110748 AddIssueIndexToRequirement: migrating =======================
– transaction_open?()
→ 0.0000s
– index_exists?(:requirements, :issue_id, {:name=>“index_requirements_on_issue_id”, :unique=>true, :algorithm=>:concurrently})
→ 0.0051s
– execute(“SET statement_timeout TO 0”)
→ 0.0002s
– add_index(:requirements, :issue_id, {:name=>“index_requirements_on_issue_id”, :unique=>true, :algorithm=>:concurrently})
→ 0.0038s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210616110748 AddIssueIndexToRequirement: migrated (0.0105s) ==============
== 20210616111311 AddIssueRequirementForeignKey: migrating ====================
– transaction_open?()
→ 0.0000s
– foreign_keys(:requirements)
→ 0.0057s
– execute(“ALTER TABLE requirements\nADD CONSTRAINT fk_85044baef0\nFOREIGN KEY (issue_id)\nREFERENCES issues (id)\nON DELETE CASCADE\nNOT VALID;\n”)
→ 0.0012s
– execute(“SET statement_timeout TO 0”)
→ 0.0002s
– execute(“ALTER TABLE requirements VALIDATE CONSTRAINT fk_85044baef0;”)
→ 0.0071s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210616111311 AddIssueRequirementForeignKey: migrated (0.0186s) ===========
== 20210616134905 AddTimestampToSchemaMigration: migrating ====================
– add_column(:schema_migrations, :finished_at, :timestamptz)
→ 0.0022s
– change_column_default(:schema_migrations, :finished_at, #<Proc:0x00007fe972f93ce8 /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20210616134905_add_timestamp_to_schema_migration.rb:9 (lambda)>)
→ 0.0017s
== 20210616134905 AddTimestampToSchemaMigration: migrated (0.0040s) ===========
== 20210616145254 AddPartialIndexForCiBuildsToken: migrating ==================
– transaction_open?()
→ 0.0000s
– index_exists?(:ci_builds, :token, {:unique=>true, :where=>“token IS NOT NULL”, :name=>“index_ci_builds_on_token_partial”, :algorithm=>:concurrently})
→ 0.0174s
– execute(“SET statement_timeout TO 0”)
→ 0.0003s
– add_index(:ci_builds, :token, {:unique=>true, :where=>“token IS NOT NULL”, :name=>“index_ci_builds_on_token_partial”, :algorithm=>:concurrently})
→ 0.0531s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210616145254 AddPartialIndexForCiBuildsToken: migrated (0.0730s) =========
== 20210616154808 RemoveCiBuildProtectedIndex: migrating ======================
– transaction_open?()
→ 0.0000s
– index_exists?(:ci_builds, :protected, {:name=>“index_ci_builds_on_protected”, :algorithm=>:concurrently})
→ 0.0163s
– execute(“SET statement_timeout TO 0”)
→ 0.0003s
– remove_index(:ci_builds, {:name=>“index_ci_builds_on_protected”, :algorithm=>:concurrently, :column=>:protected})
→ 0.0244s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210616154808 RemoveCiBuildProtectedIndex: migrated (0.0424s) =============
== 20210616185947 AddMailgunSettingsToApplicationSetting: migrating ===========
– add_column(:application_settings, :encrypted_mailgun_signing_key, :binary)
→ 0.0019s
– add_column(:application_settings, :encrypted_mailgun_signing_key_iv, :binary)
→ 0.0008s
– add_column(:application_settings, :mailgun_events_enabled, :boolean, {:default=>false, :null=>false})
→ 0.0016s
== 20210616185947 AddMailgunSettingsToApplicationSetting: migrated (0.0044s) ==
== 20210617022324 CreateIncidentManagementPendingAlertEscalations: migrating ==
– execute(“\nCREATE TABLE incident_management_pending_alert_escalations (\n id bigserial NOT NULL,\n rule_id bigint,\n alert_id bigint NOT NULL,\n schedule_id bigint NOT NULL,\n process_at timestamp with time zone NOT NULL,\n created_at timestamp with time zone NOT NULL,\n updated_at timestamp with time zone NOT NULL,\n status smallint NOT NULL,\n PRIMARY KEY (id, process_at)\n) PARTITION BY RANGE (process_at);\n\nCREATE INDEX index_incident_management_pending_alert_escalations_on_alert_id\n ON incident_management_pending_alert_escalations USING btree (alert_id);\n\nCREATE INDEX index_incident_management_pending_alert_escalations_on_rule_id\n ON incident_management_pending_alert_escalations USING btree (rule_id);\n\nCREATE INDEX index_incident_management_pending_alert_escalations_on_schedule_id\n ON incident_management_pending_alert_escalations USING btree (schedule_id);\n\nALTER TABLE incident_management_pending_alert_escalations ADD CONSTRAINT fk_rails_fcbfd9338b\n FOREIGN KEY (schedule_id) REFERENCES incident_management_oncall_schedules(id) ON DELETE CASCADE;\n\nALTER TABLE incident_management_pending_alert_escalations ADD CONSTRAINT fk_rails_057c1e3d87\n FOREIGN KEY (rule_id) REFERENCES incident_management_escalation_rules(id) ON DELETE SET NULL;\n\nALTER TABLE incident_management_pending_alert_escalations ADD CONSTRAINT fk_rails_8d8de95da9\n FOREIGN KEY (alert_id) REFERENCES alert_management_alerts(id) ON DELETE CASCADE;\n”)
→ 0.0048s
== 20210617022324 CreateIncidentManagementPendingAlertEscalations: migrated (0.0071s)
== 20210617161348 CascadeDeleteFreezePeriods: migrating =======================
– transaction_open?()
→ 0.0000s
– foreign_keys(:ci_freeze_periods)
→ 0.0062s
– execute(“ALTER TABLE ci_freeze_periods\nADD CONSTRAINT fk_2e02bbd1a6\nFOREIGN KEY (project_id)\nREFERENCES projects (id)\nON DELETE CASCADE\nNOT VALID;\n”)
→ 0.0016s
– execute(“SET statement_timeout TO 0”)
→ 0.0003s
– execute(“ALTER TABLE ci_freeze_periods VALIDATE CONSTRAINT fk_2e02bbd1a6;”)
→ 0.0065s
– execute(“RESET statement_timeout”)
→ 0.0002s
– foreign_keys(:ci_freeze_periods)
→ 0.0055s
– remove_foreign_key(:ci_freeze_periods, :projects, {:column=>:project_id, :name=>“fk_rails_2e02bbd1a6”})
→ 0.0125s
== 20210617161348 CascadeDeleteFreezePeriods: migrated (0.0393s) ==============
== 20210617180131 MigrateUsagePingSidekiqQueue: migrating =====================
== 20210617180131 MigrateUsagePingSidekiqQueue: migrated (0.0012s) ============
== 20210621043337 RenameServicesToIntegrations: migrating =====================
– execute(“LOCK services IN ACCESS EXCLUSIVE MODE”)
→ 0.0003s
– execute(“DROP TRIGGER IF EXISTS trigger_has_external_wiki_on_insert ON services”)
→ 0.0004s
– execute(“DROP TRIGGER IF EXISTS trigger_has_external_wiki_on_update ON services”)
→ 0.0004s
– execute(“DROP TRIGGER IF EXISTS trigger_has_external_wiki_on_delete ON services”)
→ 0.0003s
– execute(“DROP TRIGGER IF EXISTS trigger_has_external_issue_tracker_on_insert ON services”)
→ 0.0004s
– execute(“DROP TRIGGER IF EXISTS trigger_has_external_issue_tracker_on_update ON services”)
→ 0.0004s
– execute(“DROP TRIGGER IF EXISTS trigger_has_external_issue_tracker_on_delete ON services”)
→ 0.0005s
– transaction()
– rename_table(:services, :integrations)
→ 0.0144s
– execute(“CREATE VIEW services AS SELECT * FROM integrations”)
→ 0.0259s
→ 0.0404s
– execute(“CREATE TRIGGER trigger_has_external_wiki_on_insert\nAFTER INSERT ON integrations\nFOR EACH ROW\nWHEN (NEW.active = TRUE AND NEW.type = ‘ExternalWikiService’ AND NEW.project_id IS NOT NULL)\nEXECUTE FUNCTION set_has_external_wiki();\n”)rake aborted!
StandardError: An error has occurred, all later migrations canceled:
Expected batched background migration for the given configuration to be marked as ‘finished’, but it is ‘active’{:job_class_name=>“CopyColumnUsingBackgroundMigrationJob”, :table_name=>“events”, :column_name=>“id”, :job_arguments=>[[“id”], [“id_convert_to_bigint”]]}
Finalize it manualy by running
sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']
For more information, check the documentation
https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1109:in ensure_batched_background_migration_is_finished' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:11:in up’
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in ddl_transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in block (3 levels) in <top (required)>’
/opt/gitlab/embedded/bin/bundle:23:in load' /opt/gitlab/embedded/bin/bundle:23:in ’
Caused by:
Expected batched background migration for the given configuration to be marked as ‘finished’, but it is ‘active’{:job_class_name=>“CopyColumnUsingBackgroundMigrationJob”, :table_name=>“events”, :column_name=>“id”, :job_arguments=>[[“id”], [“id_convert_to_bigint”]]}
Finalize it manualy by running
sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']
For more information, check the documentation
https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1109:in ensure_batched_background_migration_is_finished' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:11:in up’
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in ddl_transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in block (3 levels) in <top (required)>’
/opt/gitlab/embedded/bin/bundle:23:in load' /opt/gitlab/embedded/bin/bundle:23:in ’
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
→ 0.0007s
– execute(“CREATE TRIGGER trigger_has_external_wiki_on_update\nAFTER UPDATE ON integrations\nFOR EACH ROW\nWHEN (NEW.type = ‘ExternalWikiService’ AND OLD.active != NEW.active AND NEW.project_id IS NOT NULL)\nEXECUTE FUNCTION set_has_external_wiki();\n”)
→ 0.0005s
– execute(“CREATE TRIGGER trigger_has_external_wiki_on_delete\nAFTER DELETE ON integrations\nFOR EACH ROW\nWHEN (OLD.type = ‘ExternalWikiService’ AND OLD.project_id IS NOT NULL)\nEXECUTE FUNCTION set_has_external_wiki();\n”)
→ 0.0005s
– execute(“CREATE OR REPLACE FUNCTION set_has_external_issue_tracker()\nRETURNS TRIGGER AS\n$$\nBEGIN\nUPDATE projects SET has_external_issue_tracker = (\n EXISTS\n (\n SELECT 1\n FROM integrations\n WHERE project_id = COALESCE(NEW.project_id, OLD.project_id)\n AND active = TRUE\n AND category = ‘issue_tracker’\n )\n)\nWHERE projects.id = COALESCE(NEW.project_id, OLD.project_id);\nRETURN NULL;\n\nEND\n$$ LANGUAGE PLPGSQL\n”)
→ 0.0027s
– execute(“CREATE TRIGGER trigger_has_external_issue_tracker_on_insert\nAFTER INSERT ON integrations\nFOR EACH ROW\nWHEN (NEW.category = ‘issue_tracker’ AND NEW.active = TRUE AND NEW.project_id IS NOT NULL)\nEXECUTE FUNCTION set_has_external_issue_tracker();\n”)
→ 0.0007s
– execute(“CREATE TRIGGER trigger_has_external_issue_tracker_on_update\nAFTER UPDATE ON integrations\nFOR EACH ROW\nWHEN (NEW.category = ‘issue_tracker’ AND OLD.active != NEW.active AND NEW.project_id IS NOT NULL)\nEXECUTE FUNCTION set_has_external_issue_tracker();\n”)
→ 0.0005s
– execute(“CREATE TRIGGER trigger_has_external_issue_tracker_on_delete\nAFTER DELETE ON integrations\nFOR EACH ROW\nWHEN (OLD.category = ‘issue_tracker’ AND OLD.active = TRUE AND OLD.project_id IS NOT NULL)\nEXECUTE FUNCTION set_has_external_issue_tracker();\n”)
→ 0.0005s
== 20210621043337 RenameServicesToIntegrations: migrated (0.0498s) ============
== 20210621044000 RenameServicesIndexesToIntegrations: migrating ==============
– execute(“ALTER INDEX IF EXISTS "index_services_on_project_and_type_where_inherit_null" RENAME TO "index_integrations_on_project_and_type_where_inherit_null"\n”)
→ 0.0007s
– execute(“ALTER INDEX IF EXISTS "index_services_on_project_id_and_type_unique" RENAME TO "index_integrations_on_project_id_and_type_unique"\n”)
→ 0.0022s
– execute(“ALTER INDEX IF EXISTS "index_services_on_template" RENAME TO "index_integrations_on_template"\n”)
→ 0.0003s
– execute(“ALTER INDEX IF EXISTS "index_services_on_type" RENAME TO "index_integrations_on_type"\n”)
→ 0.0002s
– execute(“ALTER INDEX IF EXISTS "index_services_on_type_and_instance_partial" RENAME TO "index_integrations_on_type_and_instance_partial"\n”)
→ 0.0003s
– execute(“ALTER INDEX IF EXISTS "index_services_on_type_and_template_partial" RENAME TO "index_integrations_on_type_and_template_partial"\n”)
→ 0.0003s
– execute(“ALTER INDEX IF EXISTS "index_services_on_type_id_when_active_and_project_id_not_null" RENAME TO "index_integrations_on_type_id_when_active_and_project_id_not_null"\n”)
→ 0.0007s
– execute(“ALTER INDEX IF EXISTS "index_services_on_unique_group_id_and_type" RENAME TO "index_integrations_on_unique_group_id_and_type"\n”)
→ 0.0004s
== 20210621044000 RenameServicesIndexesToIntegrations: migrated (0.0054s) =====
== 20210621084632 AddSummaryToTimelogs: migrating =============================
– add_column(:timelogs, :summary, :text)
→ 0.0076s
== 20210621084632 AddSummaryToTimelogs: migrated (0.0077s) ====================
== 20210621090030 AddTextLimitToTimelogsSummary: migrating ====================
– transaction_open?()
→ 0.0000s
– current_schema()
→ 0.0003s
– execute(“ALTER TABLE timelogs\nADD CONSTRAINT check_271d321699\nCHECK ( char_length(summary) <= 255 )\nNOT VALID;\n”)
→ 0.0006s
– current_schema()
→ 0.0002s
– execute(“SET statement_timeout TO 0”)
→ 0.0002s
– execute(“ALTER TABLE timelogs VALIDATE CONSTRAINT check_271d321699;”)
→ 0.0008s
– execute(“RESET statement_timeout”)
→ 0.0002s
== 20210621090030 AddTextLimitToTimelogsSummary: migrated (0.0067s) ===========
== 20210621091830 AddDevopsAdoptionSnapshotDependencyScanning: migrating ======
– add_column(:analytics_devops_adoption_snapshots, :dependency_scanning_enabled_count, :integer)
→ 0.0009s
== 20210621091830 AddDevopsAdoptionSnapshotDependencyScanning: migrated (0.0010s)
== 20210621111747 AddCiArtifactsDevopsAdoptionIndex: migrating ================
– transaction_open?()
→ 0.0000s
– index_exists?(:ci_job_artifacts, [:file_type, :project_id, :created_at], {:name=>“index_ci_job_artifacts_on_file_type_for_devops_adoption”, :where=>“file_type IN (5,6,8,23)”, :algorithm=>:concurrently})
→ 0.0053s
– execute(“SET statement_timeout TO 0”)
→ 0.0002s
– add_index(:ci_job_artifacts, [:file_type, :project_id, :created_at], {:name=>“index_ci_job_artifacts_on_file_type_for_devops_adoption”, :where=>“file_type IN (5,6,8,23)”, :algorithm=>:concurrently})
→ 0.0147s
– execute(“RESET statement_timeout”)
→ 0.0002s
== 20210621111747 AddCiArtifactsDevopsAdoptionIndex: migrated (0.0217s) =======
== 20210621155328 ReplaceProjectAuthorizationsProjectIdIndex: migrating =======
– transaction_open?()
→ 0.0000s
– index_exists?(:project_authorizations, [:project_id, :user_id], {:name=>“index_project_authorizations_on_project_id_user_id”, :algorithm=>:concurrently})
→ 0.0019s
– execute(“SET statement_timeout TO 0”)
→ 0.0002s
– add_index(:project_authorizations, [:project_id, :user_id], {:name=>“index_project_authorizations_on_project_id_user_id”, :algorithm=>:concurrently})
→ 0.0526s
– execute(“RESET statement_timeout”)
→ 0.0003s
– transaction_open?()
→ 0.0000s
– indexes(:project_authorizations)
→ 0.0024s
– execute(“SET statement_timeout TO 0”)
→ 0.0002s
– remove_index(:project_authorizations, {:algorithm=>:concurrently, :name=>“index_project_authorizations_on_project_id”})
→ 0.0067s
– execute(“RESET statement_timeout”)
→ 0.0003s
== 20210621155328 ReplaceProjectAuthorizationsProjectIdIndex: migrated (0.0668s)
== 20210621164210 DropRemoveOnCloseFromLabels: migrating ======================
– column_exists?(:labels, :remove_on_close)
→ 0.0030s
– remove_column(:labels, :remove_on_close)
→ 0.0007s
== 20210621164210 DropRemoveOnCloseFromLabels: migrated (0.0061s) =============
== 20210621223000 StealBackgroundJobsThatReferenceServices: migrating =========
== 20210621223000 StealBackgroundJobsThatReferenceServices: migrated (0.0076s)
== 20210621223242 FinalizeRenameServicesToIntegrations: migrating =============
– transaction()
– execute(“DROP VIEW IF EXISTS services”)
→ 0.0010s
→ 0.0010s
== 20210621223242 FinalizeRenameServicesToIntegrations: migrated (0.0011s) ====
== 20210622041846 FinalizePushEventPayloadsBigintConversion: migrating ========
== 20210622041846 FinalizePushEventPayloadsBigintConversion: migrated (0.0000s)
== 20210622045705 FinalizeEventsBigintConversion: migrating ===================
STDERR:
---- End output of “bash” “/tmp/chef-script20220824-67414-19dm62j” ----
Ran “bash” “/tmp/chef-script20220824-67414-19dm62j” returned 1
Warnings:
The version of the running redis service is different than what is installed.
Please restart redis to start the new version.
sudo gitlab-ctl restart redis
The version of the running postgresql service is different than what is installed.
Please restart postgresql to start the new version.
sudo gitlab-ctl restart postgresql
Running handlers complete
Chef Infra Client failed. 31 resources updated in 01 minutes 24 seconds
Warnings:
The version of the running redis service is different than what is installed.
Please restart redis to start the new version.
sudo gitlab-ctl restart redis
The version of the running postgresql service is different than what is installed.
Please restart postgresql to start the new version.
sudo gitlab-ctl restart postgresql
===
There was an error running gitlab-ctl reconfigure. Please check the output above for more
details.
warning: %posttrans(gitlab-ce-14.3.6-ce.0.el7.x86_64) scriptlet failed, exit status 1
Non-fatal POSTTRANS scriptlet failure in rpm package gitlab-ce-14.3.6-ce.0.el7.x86_64
Verifying : gitlab-ce-14.3.6-ce.0.el7.x86_64 1/2
Verifying : gitlab-ce-14.0.12-ce.0.el7.x86_64 2/2
Updated:
gitlab-ce.x86_64 0:14.3.6-ce.0.el7
Complete!
The error reportedly happened “…running gitlab-ctl reconfigure:” as per above output.
I would like to establish if I am looking at a known issue?
There are suggestions in the output to do the following to do the following but I have no idea which if any of these should be applied or if I should try gitlab-ctl reconfigure again
sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,‘[[“id”], [“id_convert_to_bigint”]]’]
The version of the running redis service is different than what is installed.
Please restart redis to start the new version.
sudo gitlab-ctl restart redis
The version of the running postgresql service is different than what is installed.
Please restart postgresql to start the new version.
sudo gitlab-ctl restart postgresql
-
Consider including screenshots, error messages, and/or other helpful visuals
-
What version are you on (Hint:
/help) ? and are you using self-managed or gitlab.com?
Self managed CE v14.3.6 (package now installed)
- What troubleshooting steps have you already taken? Can you link to any docs or other resources so we know where you have been?
I took the decision to run gitlab-ctl stop but have gone no further.
It’s a VM so I can roll back to running 14.0.12 snapshot but I need to solve this issue quickly due to the current security issue.
Thanks for taking the time to be thorough in your request, it really helps! 