Update from 13.1.5 to 13.2 permission prb

How to Use GitLab
1

Hello

First time ever I have problem with upgrading dockerized gitlab.
My current version is 13.1.5 and I’ve tried to upgrade to 13.2.0 and latest.
Everytime docker exits with error:

Recipe: gitlab::database_migrations
  * bash[migrate gitlab-rails database] action run
    [execute] rake aborted!
              StandardError: An error has occurred, this and all later migrations canceled:

              PG::InsufficientPrivilege: ERROR:  permission denied for database gitlabhq_production
              /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20200623121135_create_dynamic_partitions_schema.rb:9:in `up'
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:54:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'

              Caused by:
              ActiveRecord::StatementInvalid: PG::InsufficientPrivilege: ERROR:  permission denied for database gitlabhq_production
              /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20200623121135_create_dynamic_partitions_schema.rb:9:in `up'
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:54:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'

              Caused by:
              PG::InsufficientPrivilege: ERROR:  permission denied for database gitlabhq_production
              /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20200623121135_create_dynamic_partitions_schema.rb:9:in `up'
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:54:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'
              Tasks: TOP => db:migrate
              (See full trace by running task with --trace)
              == 20200623121135 CreateDynamicPartitionsSchema: migrating ====================
              -- execute("CREATE SCHEMA gitlab_partitions_dynamic")

    ================================================================================
    Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
    ================================================================================

    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1'
    ---- Begin output of "bash"  "/tmp/chef-script20200726-28-1taiyv8" ----
    STDOUT: rake aborted!
    StandardError: An error has occurred, this and all later migrations canceled:

    PG::InsufficientPrivilege: ERROR:  permission denied for database gitlabhq_production
    /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20200623121135_create_dynamic_partitions_schema.rb:9:in `up'
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:54:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'

    Caused by:
    ActiveRecord::StatementInvalid: PG::InsufficientPrivilege: ERROR:  permission denied for database gitlabhq_production
    /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20200623121135_create_dynamic_partitions_schema.rb:9:in `up'
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:54:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'

    Caused by:
    PG::InsufficientPrivilege: ERROR:  permission denied for database gitlabhq_production
    /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20200623121135_create_dynamic_partitions_schema.rb:9:in `up'
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:54:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'
    Tasks: TOP => db:migrate
    (See full trace by running task with --trace)
    == 20200623121135 CreateDynamicPartitionsSchema: migrating ====================
    -- execute("CREATE SCHEMA gitlab_partitions_dynamic")
    STDERR:
    ---- End output of "bash"  "/tmp/chef-script20200726-28-1taiyv8" ----
    Ran "bash"  "/tmp/chef-script20200726-28-1taiyv8" returned 1

    Resource Declaration:
    ---------------------
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb

     55: bash "migrate gitlab-rails database" do
     56:   code <<-EOH
     57:     set -e
     58:     log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
     59:     umask 077
     60:     /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
     61:     STATUS=${PIPESTATUS[0]}
     62:     chown #{account_helper.gitlab_user}:#{account_helper.gitlab_group} ${log_file}
     63:     echo $STATUS > #{db_migrate_status_file}
     64:     exit $STATUS
     65:   EOH
     66:   environment env_variables unless env_variables.empty?
     67:   notifies :run, "execute[clear the gitlab-rails cache]", :immediately
     68:   dependent_services.each do |svc|
     69:     notifies :restart, svc, :immediately
     70:   end
     71:   not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
     72:   only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
     73: end

    Compiled Resource:
    ------------------
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:55:in `from_file'

    bash("migrate gitlab-rails database") do
      action [:run]
      default_guard_interpreter :default
      command nil
      backup 5
      interpreter "bash"
      declared_type :bash
      cookbook_name "gitlab"
      recipe_name "database_migrations"
      code "    set -e\n    log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n    umask 077\n    /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n    STATUS=${PIPESTATUS[0]}\n    chown git:git ${log_file}\n    echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-b55baf593e6\n    exit $STATUS\n"
      domain nil
      user nil
      not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-b55baf593e6) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-b55baf593e6 | grep -Fx 0)"
      only_if { #code block }
    end

    System Info:
    ------------
    chef_version=15.9.17
    platform=ubuntu
    platform_version=16.04
    ruby=ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client
2

I experienced the same issue. The problem seems to be due to incorrect permissions on the database. I ran this commands to fix:

  1. stop your container
  2. start the container with a bash entrypoint:
# TODO: make sure to adjust paths
docker run -v  '/docker/gitlab/etc:/etc/gitlab' -v '/docker/gitlab/log:/var/log/gitlab' -v '/docker/gitlab/data:/var/opt/gitlab' -v '/docker/gitlab/certs:/certs' -it 'gitlab/gitlab-ce:latest' bash
  1. start postgres, as the gitlab-psql user:
su gitlab-psql -s /bin/bash
/opt/gitlab/embedded/bin/postgres -D ~/data
  1. from another terminal, attach to the running container
docker exec -it <container_name> bash
  1. connect to postgres:
su gitlab-psql -s /bin/bash
/opt/gitlab/embedded/postgresql/11/bin/psql template0
  1. change database ownerhip:
alter database gitlabhq_production owner to gitlab;
  1. stop the database and docker container, and restart the container as you are used to
1 Like
3

sorry, cant edit.
in step 5, use

PGHOST=/var/opt/gitlab/postgresql /opt/gitlab/embedded/postgresql/11/bin/psql template1
1 Like
4

@eraic Thanks so much for supporting here and sharing your solution!

5

Works like a charm. Thanks for help.
p.s i had no time to test it erlier