Migrating gitlab installed by helm to omnibus-gitlab，Object Storage is not enabled

Infrastructure as Code & Cloud Native
, , , ,
1

Hi everyone, I know this problem exists in the documentation。
But my problem is more complicated。

I have a gitlab installed by helm，and used object storage by cephfs
gitlab version 11.9.8
This is some configuration in the configuration file

minio:
  persistence:
    storageClass: cephfs
    size: 500Gi
  resources:
    limits:
      cpu: 1000m
      memory: 2Gi
    requests:
      cpu: 50m
      memory: 350Mi

I entered the docker and executed the following command

gitlab-rake gitlab:backup:create

then，restore in the server installed omnibus-gitlab ，and is the same verison。
The difference is that using local storage in omnibus-gitlab
put the backup file into the ‘backup_path’ what edit in /etc/gitlab.rb

gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq
gitlab-rake gitlab:backup:restore BACKUP=1588947812_2020_05_08_11.9.8
gitlab-ctl start
gitlab-rake gitlab:check SANITIZE=true

And it’s worked !
But here is the problem !
when i access ‘system hooks’ or ‘/admin/groups’ it’s return 500.
GITLAB_LOG_HOME/gitlab-rails/production.log

Started GET "/admin/groups" for 127.0.0.1 at 2020-05-11 11:23:42 +0800
Processing by Admin::GroupsController#index as HTML
Completed 500 Internal Server Error in 172ms (ActiveRecord: 14.3ms)
  
ActionView::Template::Error (Object Storage is not enabled):
    23:       = visibility_level_icon(group.visibility_level, fw: false)
    24: 
    25:   .avatar-container.rect-avatar.s40
    26:     = group_icon(group, class: "avatar s40 d-none d-sm-block")
    27:   .title
    28:     = link_to [:admin, group], class: 'group-name' do
    29:       = group.full_name
  
app/uploaders/object_storage.rb:390:in `storage_for'
app/uploaders/object_storage.rb:229:in `object_store='
app/uploaders/object_storage.rb:64:in `upload='
app/uploaders/object_storage.rb:49:in `retrieve_from_store!'
app/models/concerns/avatarable.rb:57:in `uncached_avatar_path'
app/models/concerns/avatarable.rb:53:in `avatar_path'
app/models/concerns/avatarable.rb:24:in `avatar_url'
app/helpers/avatars_helper.rb:114:in `source_icon'
app/helpers/avatars_helper.rb:9:in `group_icon'
app/views/admin/groups/_group.html.haml:26:in `_app_views_admin_groups__group_html_haml___2364090223597980464_70264417591840'
app/views/admin/groups/index.html.haml:18:in `_app_views_admin_groups_index_html_haml___1908539934592158426_70264413756160'
app/controllers/application_controller.rb:116:in `render'
lib/gitlab/i18n.rb:55:in `with_locale'
lib/gitlab/i18n.rb:61:in `with_user_locale'
app/controllers/application_controller.rb:434:in `set_locale'
lib/gitlab/middleware/rails_queue_duration.rb:27:in `call'
lib/gitlab/metrics/rack_middleware.rb:17:in `block in call'
lib/gitlab/metrics/transaction.rb:55:in `run'
lib/gitlab/metrics/rack_middleware.rb:17:in `call'
lib/gitlab/middleware/multipart.rb:103:in `call'
lib/gitlab/request_profiler/middleware.rb:16:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:13:in `call'
lib/gitlab/middleware/correlation_id.rb:16:in `block in call'
lib/gitlab/correlation_id.rb:15:in `use_id'
lib/gitlab/middleware/correlation_id.rb:15:in `call'
lib/gitlab/middleware/read_only/controller.rb:40:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/request_context.rb:26:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:29:in `call'
lib/gitlab/middleware/release_env.rb:13:in `call'

I know there has instructions in the documentation

Migrating from object storage to local storage

In Omnibus installations:

In order to migrate back to local storage:

  1. Set both direct_upload and background_upload to false in gitlab.rb , under the artifacts object storage settings.
  2. Reconfigure GitLab.
  3. Run gitlab-rake gitlab:artifacts:migrate_to_local .
  4. Disable object_storage for artifacts in gitlab.rb :
  • Set gitlab_rails['artifacts_object_store_enabled'] = false .
  • Comment out all other artifacts_object_store settings, including the entire artifacts_object_store_connection section, including the closing } .
  1. [Reconfigure GitLab]
    (How to restart GitLab | GitLab)

But When i execute this command

# gitlab-rake gitlab:artifacts:migrate_to_local    
rake aborted!
    Don't know how to build task 'gitlab:artifacts:migrate_to_local' (See the list of available tasks with `rake --tasks`)
    Did you mean?  gitlab:artifacts:migrate
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'
    (See full trace by running task with --trace)

So， what can i do 。help me please，thanks

2

Hi, I have the exact same problem. Did you get it resolved?

3

For me the problem happened when I added LDAP.

apparently gitlab backup of helm-based install does not include (or restore does not restore on omnibus) user uploads (such as avatars).

Some users had their avatars on minio and after restore gitlab started throwing error 500 on these, with cryptic error “Object Storage is not enabled for AvatarUploader”.

For me the workaround was to delete the broken Uploads records in database.

root@omnibus15:/# gitlab-rake gitlab:uploads:check
Checking integrity of Uploads
- 1..21: Failures: 14
Done!

To delete mentions of uploads:

`gitlab-rails dbconsole --database main`
`truncate uploads cascade;`