Gitlab17.5.5 upgrade to 17.6.3 upgrade failure

Problem to solve

We are Gitlab EE 17.5.5. I attempted to upgrade to 17.6.3. Towards the end the upgrade process, I see the following error:

“There was an error running gitlab-ctl reconfigure. Please check the output above for more
details”

If I run “gitlab-ctl reconfigure” I see the following errors in red:

rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘1’
---- Begin output of “bash” ----
STDOUT: [WARNING] Object storage for ci_secure_files must have a bucket specified
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::CheckViolation: ERROR: check constraint “check_9e09ffbd0f” of relation “agent_activity_events” is violated by some row

Caused by:
PG::CheckViolation: ERROR: check constraint “check_9e09ffbd0f” of relation “agent_activity_events” is violated by some row

Steps to reproduce

I reverted back to Gitlab 17.5.5 and tried upgrading to 17.7.2. I ran into the same problem.

gitlab-rake db:migrate produces the same errors.

Configuration

OS: Ubuntu 22.04 64-bit

Versions

Please add an x whether options apply, and add the version information.

  • [ X] Self-managed
  • GitLab.com SaaS
  • Dedicated

Versions

  • 17.5.5-ee

It sounds like you have some invalid entries in the database.

If you run gitlab-psql, what is the output of the following?

SELECT * FROM agent_activity_events WHERE agent_project_id IS NULL;

id | agent_id | user_id | project_id | merge_request_id | agent_token_id | recorded_at | kind | level | sha | detail | agent_project_id
----±---------±--------±-----------±-----------------±---------------±------------------------------±-----±------±----±-------±-----------------
77 | 13 | | | | 13 | 2024-08-21 22:49:03.958891+00 | 2 | 1 | | |
(1 row)

Ran DELETE FROM agent_activity_events WHERE agent_project_id IS NULL; and that fixed the problem. Thanks!