System Hooks gives me error 500

Opening System Hooks page as admin yields error page 500

I’m on Gitlab 15.7.3 (Ubuntu 18) right now and I noticed that I can no longer open the System Hooks page as admin in my staging system. It gives me error 500 right away. (And it was the same on 15.7.2. I merely updated to see if somehow, magically, it goes away. It didn’t…)

The production.log yields:

Started GET "/admin/hooks" for 137.250.20.43 at 2023-01-17 15:24:32 +0100
Processing by Admin::HooksController#index as HTML
  Rendered layout layouts/admin.html.haml (Duration: 9.4ms | Allocations: 3262)
Completed 500 Internal Server Error in 40ms (ActiveRecord: 3.3ms | Elasticsearch: 0.0ms | Allocations: 9013)
  
ActionView::Template::Error ():
     5:   .row
     6:     .col-md-8.col-lg-7
     7:       %strong.light-header
     8:         = hook.url
     9:         - if hook.rate_limited?
    10:           = gl_badge_tag(_('Disabled'), variant: :danger, size: :sm)
    11:         - elsif hook.permanently_disabled?
  
app/views/shared/web_hooks/_hook.html.haml:8
app/views/shared/web_hooks/_index.html.haml:10
app/views/shared/web_hooks/_index.html.haml:9:in `each'
app/views/shared/web_hooks/_index.html.haml:9
app/components/pajamas/card_component.html.haml:7:in `call'
app/views/shared/web_hooks/_index.html.haml:2
app/views/admin/hooks/index.html.haml:12
app/controllers/application_controller.rb:142:in `render'
lib/gitlab/auth/current_user_mode.rb:78:in `with_current_admin'
app/controllers/application_controller.rb:536:in `set_current_admin'
lib/gitlab/session.rb:11:in `with_session'
app/controllers/application_controller.rb:524:in `set_session_storage'
lib/gitlab/i18n.rb:107:in `with_locale'
lib/gitlab/i18n.rb:113:in `with_user_locale'
app/controllers/application_controller.rb:512:in `set_locale'
app/controllers/application_controller.rb:506:in `set_current_context'
lib/gitlab/middleware/memory_report.rb:13:in `call'
lib/gitlab/middleware/speedscope.rb:13:in `call'
lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'
lib/gitlab/jira/middleware.rb:19:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:21:in `call'
lib/gitlab/middleware/query_analyzer.rb:11:in `block in call'
lib/gitlab/database/query_analyzer.rb:37:in `within'
lib/gitlab/middleware/query_analyzer.rb:11:in `call'
lib/gitlab/middleware/multipart.rb:173:in `call'
lib/gitlab/middleware/read_only/controller.rb:50:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
lib/gitlab/middleware/request_context.rb:21:in `call'
lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'
config/initializers/fix_local_cache_middleware.rb:11:in `call'
lib/gitlab/middleware/compressed_json.rb:37:in `call'
lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'
lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:77:in `call'
lib/gitlab/middleware/release_env.rb:13:in `call'

Many thanks for any help on this!

OK, it turns out I had a similar issue to Problem setting up gitlab with external Postgres and forced SSL connection (#4049) · Issues · GitLab.org / omnibus-gitlab · GitLab

When I upgraded recently to 15.8.0, I no longer saw only a template error, but the production log also showed an “OpenSSL::Cipher::CipherError”, which lead me in the right direction: somehow, during a restore I had obtained a wrong db_key_base and my installation could no longer decode any integrations stored in the database.

A simple gitlab-rake gitlab:doctor:secrets would have also shown that, but the error message was too general, in that it merely complained about being unable to render a page. So I never looked in that direction.

Hopefully, this is useful for other people with similar issues.