Invalid byte sequence in UTF-8 on Gitlab installation

How to Use GitLab
#1

Hi, I’m trying to install Gitlab via Helm chart but my gitlab-unicorn, sidekiq and runner pods are always crashing.

Setup
Kubernetes version: 1.15.6
Helm version: 3.0.1
Storage: rook-ceph with rook-ceph-block StorageClass

Helm values:

certmanager:
  install: false
gitlab-runner:
  runners:
    privileged: true
global:
  email:
    display_name: ****
    from: ****
    reply_to: ****
  hosts:
    domain: ****
    https: false
  ingress:
    annotations:
      kubernetes:
        io/tls-acme: true
    tls:
      enabled: false
  smtp:
    address: smtp.gmail.com
    authentication: login
    domain: ********
    enabled: true
    force_ssl: false
    openssl_verify_mode: peer
    password:
      secret: smtp-gmail
    port: 587
    starttls_auto: true
    tls: false
    user_name: ******

Relevant logs
Unicorn

bundler: failed to load command: unicorn (/srv/gitlab/vendor/bundle/ruby/2.6.0/bin/unicorn)
ArgumentError: invalid byte sequence in UTF-8
/srv/gitlab/config/initializers/smtp_settings.rb:12:in `strip'
/srv/gitlab/config/initializers/smtp_settings.rb:12:in `<top (required)>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:285:in `load'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:285:in `block in load'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:257:in `load_dependency'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:285:in `load'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:657:in `block in load_config_initializer'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:170:in `instrument'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:656:in `load_config_initializer'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:614:in `block (2 levels) in <class:Engine>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:613:in `each'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:613:in `block in <class:Engine>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:32:in `instance_exec'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:32:in `run'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:61:in `block in run_initializers'
/usr/lib/ruby/2.6.0/tsort.rb:228:in `block in tsort_each'
/usr/lib/ruby/2.6.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
/usr/lib/ruby/2.6.0/tsort.rb:422:in `block (2 levels) in each_strongly_connected_component_from'
/usr/lib/ruby/2.6.0/tsort.rb:431:in `each_strongly_connected_component_from'
/usr/lib/ruby/2.6.0/tsort.rb:421:in `block in each_strongly_connected_component_from'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:50:in `each'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:50:in `tsort_each_child'
/usr/lib/ruby/2.6.0/tsort.rb:415:in `call'
/usr/lib/ruby/2.6.0/tsort.rb:415:in `each_strongly_connected_component_from'
/usr/lib/ruby/2.6.0/tsort.rb:349:in `block in each_strongly_connected_component'
/usr/lib/ruby/2.6.0/tsort.rb:347:in `each'
/usr/lib/ruby/2.6.0/tsort.rb:347:in `call'
/usr/lib/ruby/2.6.0/tsort.rb:347:in `each_strongly_connected_component'
/usr/lib/ruby/2.6.0/tsort.rb:226:in `tsort_each'
/usr/lib/ruby/2.6.0/tsort.rb:205:in `tsort_each'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:60:in `run_initializers'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/application.rb:361:in `initialize!'
/srv/gitlab/config/environment.rb:6:in `<top (required)>'
/srv/gitlab/config.ru:19:in `require'
/srv/gitlab/config.ru:19:in `block in <main>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/builder.rb:55:in `instance_eval'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/rack-2.0.7/lib/rack/builder.rb:55:in `initialize'
/srv/gitlab/config.ru:1:in `new'
/srv/gitlab/config.ru:1:in `<main>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/unicorn-5.4.1/lib/unicorn.rb:56:in `eval'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/unicorn-5.4.1/lib/unicorn.rb:56:in `block in builder'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/unicorn-5.4.1/lib/unicorn/http_server.rb:795:in `build_app!'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/unicorn-5.4.1/lib/unicorn/http_server.rb:139:in `start'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/unicorn-5.4.1/bin/unicorn:126:in `<top (required)>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `load'
/srv/gitlab/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `<top (required)>'

Sidekiq pod

invalid byte sequence in UTF-8
/srv/gitlab/config/initializers/smtp_settings.rb:12:in `strip'
/srv/gitlab/config/initializers/smtp_settings.rb:12:in `<top (required)>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:285:in `load'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:285:in `block in load'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:257:in `load_dependency'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies.rb:285:in `load'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:657:in `block in load_config_initializer'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications.rb:170:in `instrument'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:656:in `load_config_initializer'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:614:in `block (2 levels) in <class:Engine>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:613:in `each'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/engine.rb:613:in `block in <class:Engine>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:32:in `instance_exec'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:32:in `run'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:61:in `block in run_initializers'
/usr/lib/ruby/2.6.0/tsort.rb:228:in `block in tsort_each'
/usr/lib/ruby/2.6.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
/usr/lib/ruby/2.6.0/tsort.rb:422:in `block (2 levels) in each_strongly_connected_component_from'
/usr/lib/ruby/2.6.0/tsort.rb:431:in `each_strongly_connected_component_from'
/usr/lib/ruby/2.6.0/tsort.rb:421:in `block in each_strongly_connected_component_from'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:50:in `each'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:50:in `tsort_each_child'
/usr/lib/ruby/2.6.0/tsort.rb:415:in `call'
/usr/lib/ruby/2.6.0/tsort.rb:415:in `each_strongly_connected_component_from'
/usr/lib/ruby/2.6.0/tsort.rb:349:in `block in each_strongly_connected_component'
/usr/lib/ruby/2.6.0/tsort.rb:347:in `each'
/usr/lib/ruby/2.6.0/tsort.rb:347:in `call'
/usr/lib/ruby/2.6.0/tsort.rb:347:in `each_strongly_connected_component'
/usr/lib/ruby/2.6.0/tsort.rb:226:in `tsort_each'
/usr/lib/ruby/2.6.0/tsort.rb:205:in `tsort_each'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/initializable.rb:60:in `run_initializers'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/railties-5.2.3/lib/rails/application.rb:361:in `initialize!'
/srv/gitlab/config/environment.rb:6:in `<top (required)>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/cli.rb:288:in `require'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/cli.rb:288:in `boot_system'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/cli.rb:46:in `run'
/srv/gitlab/vendor/bundle/ruby/2.6.0/gems/sidekiq-5.2.7/bin/sidekiq:12:in `<top (required)>'
/srv/gitlab/vendor/bundle/ruby/2.6.0/bin/sidekiq:23:in `load'
/srv/gitlab/vendor/bundle/ruby/2.6.0/bin/sidekiq:23:in `<top (required)>'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `load'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:74:in `kernel_load'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli/exec.rb:28:in `run'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:463:in `exec'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:27:in `dispatch'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/cli.rb:18:in `start'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:30:in `block in <top (required)>'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
/usr/lib/ruby/gems/2.6.0/gems/bundler-1.17.3/exe/bundle:22:in `<top (required)>'
/srv/gitlab/bin/bundle:3:in `load'
/srv/gitlab/bin/bundle:3:in `<main>'

Runner

ERROR: Registering runner... failed runner=LM28mwta status=404 Not Found
PANIC: Failed to register this runner. Perhaps you are having network problems

Any help is welcome.

#2

In case someone comes across this issue, I’ve fixed it by reinstalling gitlab without smtp settings.
EDIT: this was due to my smtp password using non standard characters.

#3

Actually, the message means that the special characters in your password were not encoded in UTF-8, but in another encoding, like Latin-1 or WIN1251 (now outdated 8-bit special character sets). Ruby (the interpreter behind a great deal of gitlab’s moving parts) expects them to be UTF-8.

It is highly recommended to switch your entire working environment to UTF-8. If you’re a Linux user, this Stackexchange question has a ton of information about making that switch: https://unix.stackexchange.com/questions/23389/how-can-i-set-vims-default-encoding-to-utf-8

Before making the change though, one bit of advice: temporarily change your password to not use special characters, and set up a test account to make sure that you can login with a utf-8 password.