A number of hashed_storage_project_migrate jobs failed / Projects show "The repository for this project does not exist"

Thanks, I had also found that Issue and was keeping it aside as a last resort option. After all, messing with stuff directly through the Rails Console is a bit dangerous. Your message gave me the confidence to try it myself :slight_smile:

For anyone else that encounters this, here is how I solved it. Bear in mind that there may be other causes for this or similar errors (in my case it seemed to really only be that the repository_read_only flag was already set, for whatever reason). Proceed at your own risk.

If you don’t already know which repositories/projects are affected, you should see a failed Sidekiq job for each under Admin Area > Monitoring > Background Jobs > Dead. The Repository storage Rake tasks can also come in handy to confirm the no. and list the projects stuck in legacy storage.

  1. Start a Rails console session
  2. Find the project by name: p = Project.find_by_name("<project-name>")
  3. Confirm that the repository is in fact read-only: p.repository_read_only
  4. Unset the repository_read_only flag: p.update!(repository_read_only:nil)
  5. Retry the corresponding Sidekiq job in the Admin Area (for me it would succeed almost instantly - the dead job count decreased and I never saw a running job)
  6. Rinse and repeat from step 2.
6 Likes