Upgrading from 13.9.4 to 13.10.0 results in an error 500

aaronleem · March 23, 2021, 12:30pm

I’ve been trying to upgrade from the image gitlab/gitlab-ee:13.9.4-ee.0 to the image gitlab/gitlab-ee:13.10.0-ee.0 and I’ve been having issues with an internal server error happening.

This is the full error text:

gitlab_1   | {"method":"GET","path":"/","format":"html","controller":"RootController","action":"index","status":500,"time":"2021-03-23T12:21:53.586Z","params":[],"remote_ip":"197.94.245.112","user_id":1,"username":"aaron","ua":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4421.0 Safari/537.36 Edg/90.0.810.1","correlation_id":"01F1FGHH7T354M5WG8FZQ8TVQR","meta.caller_id":"RootController#index","meta.remote_ip":"197.94.245.112","meta.feature_category":"projects","meta.client_id":"ip/197.94.245.112","redis_calls":1,"redis_duration_s":0.000288,"redis_read_bytes":396,"redis_write_bytes":702,"redis_shared_state_calls":1,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":396,"redis_shared_state_write_bytes":702,"db_count":3,"db_write_count":0,"db_cached_count":0,"cpu_s":0.051896,"mem_objects":14602,"mem_bytes":996136,"mem_mallocs":2201,"queue_duration_s":0.005026,"exception.class":"NoMethodError","exception.message":"undefined method `markdown_surround_selection=' for #<UserPreference:0x00006afc8a069360>\nDid you mean?  mark_for_destruction","exception.backtrace":["app/models/user.rb:1801:in `user_preference'","app/models/user.rb:283:in `markdown_surround_selection'","lib/gitlab/metrics/instrumentation.rb:160:in `block in markdown_surround_selection'","lib/gitlab/metrics/method_call.rb:27:in `measure'","lib/gitlab/metrics/instrumentation.rb:160:in `markdown_surround_selection'","lib/gitlab/gon_helper.rb:17:in `add_gon_variables'","ee/lib/ee/gitlab/gon_helper.rb:10:in `add_gon_variables'","app/controllers/application_controller.rb:464:in `block in set_current_context'","lib/gitlab/application_context.rb:63:in `block in use'","lib/gitlab/application_context.rb:63:in `use'","lib/gitlab/application_context.rb:24:in `with_context'","app/controllers/application_controller.rb:455:in `set_current_context'","lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'","lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'","lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'","lib/gitlab/metrics/transaction.rb:56:in `run'","lib/gitlab/metrics/rack_middleware.rb:16:in `call'","lib/gitlab/request_profiler/middleware.rb:17:in `call'","lib/gitlab/jira/middleware.rb:19:in `call'","lib/gitlab/middleware/go.rb:20:in `call'","lib/gitlab/etag_caching/middleware.rb:21:in `call'","lib/gitlab/middleware/multipart.rb:172:in `call'","lib/gitlab/middleware/read_only/controller.rb:50:in `call'","lib/gitlab/middleware/read_only.rb:18:in `call'","lib/gitlab/middleware/same_site_cookies.rb:27:in `call'","lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'","lib/gitlab/middleware/basic_health_check.rb:25:in `call'","lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'","lib/gitlab/middleware/request_context.rb:21:in `call'","config/initializers/fix_local_cache_middleware.rb:11:in `call'","lib/gitlab/metrics/requests_rack_middleware.rb:76:in `call'","lib/gitlab/middleware/release_env.rb:12:in `call'"],"db_duration_s":0.00182,"view_duration_s":0.0,"duration_s":0.04174}
gitlab_1   | 
gitlab_1   | ==> /var/log/gitlab/gitlab-rails/production.log <==
gitlab_1   |   
gitlab_1   | NoMethodError (undefined method `markdown_surround_selection=' for #<UserPreference:0x00006afc8a069360>
gitlab_1   | Did you mean?  mark_for_destruction):
gitlab_1   |   
gitlab_1   | app/models/user.rb:1801:in `user_preference'
gitlab_1   | app/models/user.rb:283:in `markdown_surround_selection'
gitlab_1   | lib/gitlab/metrics/instrumentation.rb:160:in `block in markdown_surround_selection'
gitlab_1   | lib/gitlab/metrics/method_call.rb:27:in `measure'
gitlab_1   | lib/gitlab/metrics/instrumentation.rb:160:in `markdown_surround_selection'
gitlab_1   | lib/gitlab/gon_helper.rb:17:in `add_gon_variables'
gitlab_1   | ee/lib/ee/gitlab/gon_helper.rb:10:in `add_gon_variables'
gitlab_1   | app/controllers/application_controller.rb:464:in `block in set_current_context'
gitlab_1   | lib/gitlab/application_context.rb:63:in `block in use'
gitlab_1   | lib/gitlab/application_context.rb:63:in `use'
gitlab_1   | lib/gitlab/application_context.rb:24:in `with_context'
gitlab_1   | app/controllers/application_controller.rb:455:in `set_current_context'
gitlab_1   | lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'
gitlab_1   | lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'
gitlab_1   | lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'
gitlab_1   | lib/gitlab/metrics/transaction.rb:56:in `run'
gitlab_1   | lib/gitlab/metrics/rack_middleware.rb:16:in `call'
gitlab_1   | lib/gitlab/request_profiler/middleware.rb:17:in `call'
gitlab_1   | lib/gitlab/jira/middleware.rb:19:in `call'
gitlab_1   | lib/gitlab/middleware/go.rb:20:in `call'
gitlab_1   | lib/gitlab/etag_caching/middleware.rb:21:in `call'
gitlab_1   | lib/gitlab/middleware/multipart.rb:172:in `call'
gitlab_1   | lib/gitlab/middleware/read_only/controller.rb:50:in `call'
gitlab_1   | lib/gitlab/middleware/read_only.rb:18:in `call'
gitlab_1   | lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
gitlab_1   | lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
gitlab_1   | lib/gitlab/middleware/basic_health_check.rb:25:in `call'
gitlab_1   | lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
gitlab_1   | lib/gitlab/middleware/request_context.rb:21:in `call'
gitlab_1   | config/initializers/fix_local_cache_middleware.rb:11:in `call'
gitlab_1   | lib/gitlab/metrics/requests_rack_middleware.rb:76:in `call'
gitlab_1   | lib/gitlab/middleware/release_env.rb:12:in `call'

and I can see that it’s related to markdown, but I’ve no idea how to fix this.

gitlab-greg · March 23, 2021, 7:16pm

Hi @aaronleem , welcome to the GitLab Community forum!

The error message indicates that there’s no markdown_surround_selection method.

For this to happen, it seems likely that that this database migration that adds the Markdown Surround Selection to User preferences was failed or did not complete successfuly.

gitlab.com

gitlab-org/gitlab/blob/master/db/migrate/20210209232508_add_markdown_surround_selection_to_user_preferences.rb

# frozen_string_literal: true

class AddMarkdownSurroundSelectionToUserPreferences < ActiveRecord::Migration[6.0]
  include Gitlab::Database::MigrationHelpers

  DOWNTIME = false

  disable_ddl_transaction!

  def up
    with_lock_retries do
      add_column :user_preferences, :markdown_surround_selection, :boolean, default: true, null: false
    end
  end

  def down
    with_lock_retries do
      remove_column :user_preferences, :markdown_surround_selection, :boolean
    end
  end

This file has been truncated. show original

Sometimes this can be resolved by manually triggering the migrations. Can you run the following commands and let us know if the problem persists?

sudo gitlab-ctl reconfigure
sudo gitlab-rake db:migrate

Thanks!

pyth1978 · March 29, 2021, 7:41pm

Hi,

I had the same issue and this fixed it for me. Running ubuntu arm64 on raspberry, official packages.
Thanks,

Dirkjan

ariva07 · April 16, 2021, 3:31pm

Thanks a lot!
The same issue, your tip helped in my case as well.