Repository shows 502 to any interaction

Hello folks, we have a problem accessing repository. Error 502 is displayed. Nothing was changed in configs or network configuration… I would be gratefull for some tips how to solve this issue.


Hi,

First, please don’t post screenshots of console commands - it’s unnecessary and difficult to read. It’s far easy to copy and paste the text from the console.

Also, you should either be using sudo or running the reconfigure command as root user since it needs to make changes under /opt as well as other places as well. That is why it’s not working for you.

Understood my bad. This is the result based on your suggestion:

wi-workstation@WastelandsWorkstation:~$ sudo gitlab-ctl reconfigure
[sudo] password for wi-workstation:
Starting Chef Client, version 14.13.11
resolving cookbooks for run list: [“gitlab-ee”]
Synchronizing Cookbooks:

  • gitlab-ee (0.0.1)
  • package (0.1.0)
  • gitlab (0.0.1)
  • consul (0.1.0)
  • repmgr (0.1.0)
  • runit (4.3.0)
  • postgresql (0.1.0)
  • redis (0.1.0)
  • monitoring (0.1.0)
  • registry (0.1.0)
  • mattermost (0.1.0)
  • gitaly (0.1.0)
  • letsencrypt (0.1.0)
  • nginx (0.1.0)
  • acme (4.0.0)
  • crond (0.1.0)
    Installing Cookbook Gems:
    Compiling Cookbooks…
    Recipe: gitlab::default
  • directory[/etc/gitlab] action create (up to date)
    Converging 285 resources

  • directory[/etc/gitlab] action create (up to date)

  • directory[Create /var/opt/gitlab] action create (up to date)

  • directory[Create /var/log/gitlab] action create (up to date)

  • directory[/opt/gitlab/embedded/etc] action create (up to date)

  • template[/opt/gitlab/embedded/etc/gitconfig] action create (up to date)
    Recipe: gitlab::web-server

  • account[Webserver user and group] action create

    • group[Webserver user and group] action create (up to date)
    • linux_user[Webserver user and group] action create (up to date)
      (up to date)
      Recipe: gitlab::users
  • directory[/var/opt/gitlab] action create (up to date)

  • account[GitLab user and group] action create

    • group[GitLab user and group] action create (up to date)
    • linux_user[GitLab user and group] action create (up to date)
      (up to date)
  • template[/var/opt/gitlab/.gitconfig] action create (up to date)

  • directory[/var/opt/gitlab/.bundle] action create (up to date)
    Recipe: gitlab::gitlab-shell

  • storage_directory[/var/opt/gitlab/.ssh] action create

    • ruby_block[directory resource: /var/opt/gitlab/.ssh] action run (skipped due to not_if)
      (up to date)
  • directory[/var/log/gitlab/gitlab-shell/] action create (up to date)

  • directory[/var/opt/gitlab/gitlab-shell] action create (up to date)

  • templatesymlink[Create a config.yml and create a symlink to Rails root] action create

    • template[/var/opt/gitlab/gitlab-shell/config.yml] action create (up to date)
    • link[Link /opt/gitlab/embedded/service/gitlab-shell/config.yml to /var/opt/gitlab/gitlab-shell/config.yml] action create (up to date)
      (up to date)
  • link[/opt/gitlab/embedded/service/gitlab-shell/.gitlab_shell_secret] action create (up to date)

  • file[/var/opt/gitlab/.ssh/authorized_keys] action create_if_missing (up to date)

  • bash[Set proper security context on ssh files for selinux] action run (skipped due to only_if)
    Recipe: gitlab::gitlab-rails

  • storage_directory[/mnt/wi-raid2/gitlab/] action create

    • ruby_block[directory resource: /mnt/wi-raid2/gitlab/] action run (skipped due to not_if)
      (up to date)
  • storage_directory[/mnt/wi-raid2/gitlab/repositories] action create

    • ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories] action run

      ================================================================================
      Error executing action run on resource ‘ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories]’

      Mixlib::ShellOut::ShellCommandFailed

      Failed asserting that mode permissions on “/mnt/wi-raid2/gitlab/repositories” is 2770
      ---- Begin output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
      STDOUT:
      STDERR: + grep -o …$

      • readlink -f /mnt/wi-raid2/gitlab/repositories
      • stat --printf=%04a /mnt/wi-raid2/gitlab/repositories
      • [ 0770 = 2770 ]
        ---- End output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
        Ran set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] returned 1

      Cookbook Trace:

      /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:125:in validate_command' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:113:in block in validate’
      /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in each_index' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in validate’
      /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:87:in validate!' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:43:in block (3 levels) in class_from_file’

      Resource Declaration:

      In /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb

      34: ruby_block “directory resource: #{new_resource.path}” do
      35: block do
      36: # Ensure the directory exists
      37: storage_helper.ensure_directory_exists(new_resource.path)
      38:
      39: # Ensure the permissions are set
      40: storage_helper.ensure_permissions_set(new_resource.path)
      41:
      42: # Error out if we have not achieved the target permissions
      43: storage_helper.validate!(new_resource.path)
      44: end
      45: not_if { storage_helper.validate(new_resource.path) }
      46: end
      47: end

      Compiled Resource:

      Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:34:in `block in class_from_file’

      ruby_block(“directory resource: /mnt/wi-raid2/gitlab/repositories”) do
      action [:run]
      default_guard_interpreter :default
      declared_type :ruby_block
      cookbook_name “gitlab”
      block #Proc:0x000055d080656c78@/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:35
      block_name “directory resource: /mnt/wi-raid2/gitlab/repositories”
      not_if { #code block }
      end

      System Info:

      chef_version=14.13.11
      platform=ubuntu
      platform_version=18.04
      ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
      program_name=/opt/gitlab/embedded/bin/chef-client
      executable=/opt/gitlab/embedded/bin/chef-client

    ================================================================================
    Error executing action create on resource ‘storage_directory[/mnt/wi-raid2/gitlab/repositories]’

    Mixlib::ShellOut::ShellCommandFailed

    ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Failed asserting that mode permissions on “/mnt/wi-raid2/gitlab/repositories” is 2770
    ---- Begin output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
    STDOUT:
    STDERR: + grep -o …$

    • readlink -f /mnt/wi-raid2/gitlab/repositories
    • stat --printf=%04a /mnt/wi-raid2/gitlab/repositories
    • [ 0770 = 2770 ]
      ---- End output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
      Ran set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] returned 1

    Cookbook Trace:

    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:125:in validate_command' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:113:in block in validate’
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in each_index' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in validate’
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:87:in validate!' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:43:in block (3 levels) in class_from_file’

    Resource Declaration:

    In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/gitlab-rails.rb

    56: storage_directory repositories_storage[‘path’] do
    57: owner gitlab_user
    58: mode “2770”
    59: end
    60: end

    Compiled Resource:

    Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/gitlab-rails.rb:56:in `block in from_file’

    storage_directory(“/mnt/wi-raid2/gitlab/repositories”) do
    action [:create]
    default_guard_interpreter :default
    declared_type :storage_directory
    cookbook_name “gitlab”
    recipe_name “gitlab-rails”
    owner “git”
    mode “2770”
    path “/mnt/wi-raid2/gitlab/repositories”
    end

    System Info:

    chef_version=14.13.11
    platform=ubuntu
    platform_version=18.04
    ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client

Running handlers:
There was an error running gitlab-ctl reconfigure:

storage_directory[/mnt/wi-raid2/gitlab/repositories] (gitlab::gitlab-rails line 56) had an error: Mixlib::ShellOut::ShellCommandFailed: ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Failed asserting that mode permissions on “/mnt/wi-raid2/gitlab/repositories” is 2770
---- Begin output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
STDOUT:
STDERR: + grep -o …$

  • readlink -f /mnt/wi-raid2/gitlab/repositories
  • stat --printf=%04a /mnt/wi-raid2/gitlab/repositories
  • [ 0770 = 2770 ]
    ---- End output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
    Ran set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] returned 1

Running handlers complete
Chef Client failed. 0 resources updated in 08 seconds

What are the permissions for the following directories:

/mnt/wi-raid2
/mnt/wi-raid2/gitlab
/mnt/wi-raid2/gitlab/repositories

Basically, /mnt/wi-raid2 should be owned as root with 755. The same for /mnt/wi-raid2/gitlab. However, for /mnt/wi-raid2/gitlab/repositories, this should be owned by the git user. Assuming this is an empty directory, then:

chown git:git /mnt/wi-raid2/gitlab/repositories
chmod 775 /mnt/wi-raid2/gitlab/repositories
chmod g+s /mnt/wi-raid2/gitlab/repositories

When you are finished, it should more or less look like below:

drwxrws--- 11 git  git  4.0K Oct 17 04:19 repositories

then run reconfigure again.

Thank you for prompt reply. Here is how this looks:

wi-workstation@WastelandsWorkstation:~$ sudo chown git:git /mnt/wi-raid2/gitlab/repositories -R
wi-workstation@WastelandsWorkstation:~$ sudo chmod 775 /mnt/wi-raid2/gitlab/repositories -R
wi-workstation@WastelandsWorkstation:~$ sudo chmod g+s /mnt/wi-raid2/gitlab/repositories -R
wi-workstation@WastelandsWorkstation:~$ sudo gitlab-ctl reconfigure
Starting Chef Client, version 14.13.11
resolving cookbooks for run list: [“gitlab-ee”]
Synchronizing Cookbooks:

  • gitlab-ee (0.0.1)
  • package (0.1.0)
  • gitlab (0.0.1)
  • consul (0.1.0)
  • repmgr (0.1.0)
  • runit (4.3.0)
  • postgresql (0.1.0)
  • redis (0.1.0)
  • monitoring (0.1.0)
  • registry (0.1.0)
  • mattermost (0.1.0)
  • gitaly (0.1.0)
  • letsencrypt (0.1.0)
  • nginx (0.1.0)
  • acme (4.0.0)
  • crond (0.1.0)
    Installing Cookbook Gems:
    Compiling Cookbooks…
    Recipe: gitlab::default
  • directory[/etc/gitlab] action create (up to date)
    Converging 285 resources

  • directory[/etc/gitlab] action create (up to date)

  • directory[Create /var/opt/gitlab] action create (up to date)

  • directory[Create /var/log/gitlab] action create (up to date)

  • directory[/opt/gitlab/embedded/etc] action create (up to date)

  • template[/opt/gitlab/embedded/etc/gitconfig] action create (up to date)
    Recipe: gitlab::web-server

  • account[Webserver user and group] action create

    • group[Webserver user and group] action create (up to date)
    • linux_user[Webserver user and group] action create (up to date)
      (up to date)
      Recipe: gitlab::users
  • directory[/var/opt/gitlab] action create (up to date)

  • account[GitLab user and group] action create

    • group[GitLab user and group] action create (up to date)
    • linux_user[GitLab user and group] action create (up to date)
      (up to date)
  • template[/var/opt/gitlab/.gitconfig] action create (up to date)

  • directory[/var/opt/gitlab/.bundle] action create (up to date)
    Recipe: gitlab::gitlab-shell

  • storage_directory[/var/opt/gitlab/.ssh] action create

    • ruby_block[directory resource: /var/opt/gitlab/.ssh] action run (skipped due to not_if)
      (up to date)
  • directory[/var/log/gitlab/gitlab-shell/] action create (up to date)

  • directory[/var/opt/gitlab/gitlab-shell] action create (up to date)

  • templatesymlink[Create a config.yml and create a symlink to Rails root] action create

    • template[/var/opt/gitlab/gitlab-shell/config.yml] action create (up to date)
    • link[Link /opt/gitlab/embedded/service/gitlab-shell/config.yml to /var/opt/gitlab/gitlab-shell/config.yml] action create (up to date)
      (up to date)
  • link[/opt/gitlab/embedded/service/gitlab-shell/.gitlab_shell_secret] action create (up to date)

  • file[/var/opt/gitlab/.ssh/authorized_keys] action create_if_missing (up to date)

  • bash[Set proper security context on ssh files for selinux] action run (skipped due to only_if)
    Recipe: gitlab::gitlab-rails

  • storage_directory[/mnt/wi-raid2/gitlab/] action create

    • ruby_block[directory resource: /mnt/wi-raid2/gitlab/] action run (skipped due to not_if)
      (up to date)
  • storage_directory[/mnt/wi-raid2/gitlab/repositories] action create

    • ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories] action run

      ================================================================================
      Error executing action run on resource ‘ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories]’

      Mixlib::ShellOut::ShellCommandFailed

      Failed asserting that mode permissions on “/mnt/wi-raid2/gitlab/repositories” is 2770
      ---- Begin output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
      STDOUT:
      STDERR: + grep -o …$

      • readlink -f /mnt/wi-raid2/gitlab/repositories
      • stat --printf=%04a /mnt/wi-raid2/gitlab/repositories
      • [ 0770 = 2770 ]
        ---- End output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
        Ran set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] returned 1

      Cookbook Trace:

      /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:125:in validate_command' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:113:in block in validate’
      /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in each_index' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in validate’
      /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:87:in validate!' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:43:in block (3 levels) in class_from_file’

      Resource Declaration:

      In /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb

      34: ruby_block “directory resource: #{new_resource.path}” do
      35: block do
      36: # Ensure the directory exists
      37: storage_helper.ensure_directory_exists(new_resource.path)
      38:
      39: # Ensure the permissions are set
      40: storage_helper.ensure_permissions_set(new_resource.path)
      41:
      42: # Error out if we have not achieved the target permissions
      43: storage_helper.validate!(new_resource.path)
      44: end
      45: not_if { storage_helper.validate(new_resource.path) }
      46: end
      47: end

      Compiled Resource:

      Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:34:in `block in class_from_file’

      ruby_block(“directory resource: /mnt/wi-raid2/gitlab/repositories”) do
      action [:run]
      default_guard_interpreter :default
      declared_type :ruby_block
      cookbook_name “gitlab”
      block #Proc:0x00005606963b21d8@/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:35
      block_name “directory resource: /mnt/wi-raid2/gitlab/repositories”
      not_if { #code block }
      end

      System Info:

      chef_version=14.13.11
      platform=ubuntu
      platform_version=18.04
      ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
      program_name=/opt/gitlab/embedded/bin/chef-client
      executable=/opt/gitlab/embedded/bin/chef-client

    ================================================================================
    Error executing action create on resource ‘storage_directory[/mnt/wi-raid2/gitlab/repositories]’

    Mixlib::ShellOut::ShellCommandFailed

    ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Failed asserting that mode permissions on “/mnt/wi-raid2/gitlab/repositories” is 2770
    ---- Begin output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
    STDOUT:
    STDERR: + grep -o …$

    • readlink -f /mnt/wi-raid2/gitlab/repositories
    • stat --printf=%04a /mnt/wi-raid2/gitlab/repositories
    • [ 0770 = 2770 ]
      ---- End output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
      Ran set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] returned 1

    Cookbook Trace:

    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:125:in validate_command' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:113:in block in validate’
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in each_index' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in validate’
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:87:in validate!' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:43:in block (3 levels) in class_from_file’

    Resource Declaration:

    In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/gitlab-rails.rb

    56: storage_directory repositories_storage[‘path’] do
    57: owner gitlab_user
    58: mode “2770”
    59: end
    60: end

    Compiled Resource:

    Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/gitlab-rails.rb:56:in `block in from_file’

    storage_directory(“/mnt/wi-raid2/gitlab/repositories”) do
    action [:create]
    default_guard_interpreter :default
    declared_type :storage_directory
    cookbook_name “gitlab”
    recipe_name “gitlab-rails”
    owner “git”
    mode “2770”
    path “/mnt/wi-raid2/gitlab/repositories”
    end

    System Info:

    chef_version=14.13.11
    platform=ubuntu
    platform_version=18.04
    ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client

Running handlers:
There was an error running gitlab-ctl reconfigure:

storage_directory[/mnt/wi-raid2/gitlab/repositories] (gitlab::gitlab-rails line 56) had an error: Mixlib::ShellOut::ShellCommandFailed: ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Failed asserting that mode permissions on “/mnt/wi-raid2/gitlab/repositories” is 2770
---- Begin output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
STDOUT:
STDERR: + grep -o …$

  • readlink -f /mnt/wi-raid2/gitlab/repositories
  • stat --printf=%04a /mnt/wi-raid2/gitlab/repositories
  • [ 0770 = 2770 ]
    ---- End output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
    Ran set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] returned 1

Running handlers complete
Chef Client failed. 0 resources updated in 05 seconds
wi-workstation@WastelandsWorkstation:~$

Also friend suggested to post this:

wi-workstation@WastelandsWorkstation:~$ ls -l /mnt/wi-raid2
total 4
drwx------ 5 git git 4096 cze 8 2021 gitlab
wi-workstation@WastelandsWorkstation:~$

Also tried like this :

wi-workstation@WastelandsWorkstation:~$ sudo chown root:root /mnt/wi-raid2
[sudo] password for wi-workstation:
wi-workstation@WastelandsWorkstation:~$ sudo chmod 755 /mnt/wi-raid2
wi-workstation@WastelandsWorkstation:~$ sudo chown root:root /mnt/wi-raid2/gitlab
wi-workstation@WastelandsWorkstation:~$ sudo chmod 755 /mnt/wi-raid2/gitlab
wi-workstation@WastelandsWorkstation:~$ ^C
wi-workstation@WastelandsWorkstation:~$ sudo gitlab-ctl reconfigure
Starting Chef Client, version 14.13.11
resolving cookbooks for run list: [“gitlab-ee”]
Synchronizing Cookbooks:

  • gitlab-ee (0.0.1)
  • package (0.1.0)
  • gitlab (0.0.1)
  • consul (0.1.0)
  • repmgr (0.1.0)
  • runit (4.3.0)
  • postgresql (0.1.0)
  • redis (0.1.0)
  • monitoring (0.1.0)
  • registry (0.1.0)
  • mattermost (0.1.0)
  • gitaly (0.1.0)
  • letsencrypt (0.1.0)
  • nginx (0.1.0)
  • acme (4.0.0)
  • crond (0.1.0)
    Installing Cookbook Gems:
    Compiling Cookbooks…
    Recipe: gitlab::default
  • directory[/etc/gitlab] action create (up to date)
    Converging 285 resources

  • directory[/etc/gitlab] action create (up to date)

  • directory[Create /var/opt/gitlab] action create (up to date)

  • directory[Create /var/log/gitlab] action create (up to date)

  • directory[/opt/gitlab/embedded/etc] action create (up to date)

  • template[/opt/gitlab/embedded/etc/gitconfig] action create (up to date)
    Recipe: gitlab::web-server

  • account[Webserver user and group] action create

    • group[Webserver user and group] action create (up to date)
    • linux_user[Webserver user and group] action create (up to date)
      (up to date)
      Recipe: gitlab::users
  • directory[/var/opt/gitlab] action create (up to date)

  • account[GitLab user and group] action create

    • group[GitLab user and group] action create (up to date)
    • linux_user[GitLab user and group] action create (up to date)
      (up to date)
  • template[/var/opt/gitlab/.gitconfig] action create (up to date)

  • directory[/var/opt/gitlab/.bundle] action create (up to date)
    Recipe: gitlab::gitlab-shell

  • storage_directory[/var/opt/gitlab/.ssh] action create

    • ruby_block[directory resource: /var/opt/gitlab/.ssh] action run (skipped due to not_if)
      (up to date)
  • directory[/var/log/gitlab/gitlab-shell/] action create (up to date)

  • directory[/var/opt/gitlab/gitlab-shell] action create (up to date)

  • templatesymlink[Create a config.yml and create a symlink to Rails root] action create

    • template[/var/opt/gitlab/gitlab-shell/config.yml] action create (up to date)
    • link[Link /opt/gitlab/embedded/service/gitlab-shell/config.yml to /var/opt/gitlab/gitlab-shell/config.yml] action create (up to date)
      (up to date)
  • link[/opt/gitlab/embedded/service/gitlab-shell/.gitlab_shell_secret] action create (up to date)

  • file[/var/opt/gitlab/.ssh/authorized_keys] action create_if_missing (up to date)

  • bash[Set proper security context on ssh files for selinux] action run (skipped due to only_if)
    Recipe: gitlab::gitlab-rails

  • storage_directory[/mnt/wi-raid2/gitlab/] action create

    • ruby_block[directory resource: /mnt/wi-raid2/gitlab/] action run
      • execute the ruby block directory resource: /mnt/wi-raid2/gitlab/
  • storage_directory[/mnt/wi-raid2/gitlab/repositories] action create

    • ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories] action run

      ================================================================================
      Error executing action run on resource ‘ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories]’

      Mixlib::ShellOut::ShellCommandFailed

      Failed asserting that mode permissions on “/mnt/wi-raid2/gitlab/repositories” is 2770
      ---- Begin output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
      STDOUT:
      STDERR: + grep -o …$

      • readlink -f /mnt/wi-raid2/gitlab/repositories
      • stat --printf=%04a /mnt/wi-raid2/gitlab/repositories
      • [ 0770 = 2770 ]
        ---- End output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
        Ran set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] returned 1

      Cookbook Trace:

      /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:125:in validate_command' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:113:in block in validate’
      /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in each_index' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in validate’
      /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:87:in validate!' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:43:in block (3 levels) in class_from_file’

      Resource Declaration:

      In /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb

      34: ruby_block “directory resource: #{new_resource.path}” do
      35: block do
      36: # Ensure the directory exists
      37: storage_helper.ensure_directory_exists(new_resource.path)
      38:
      39: # Ensure the permissions are set
      40: storage_helper.ensure_permissions_set(new_resource.path)
      41:
      42: # Error out if we have not achieved the target permissions
      43: storage_helper.validate!(new_resource.path)
      44: end
      45: not_if { storage_helper.validate(new_resource.path) }
      46: end
      47: end

      Compiled Resource:

      Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:34:in `block in class_from_file’

      ruby_block(“directory resource: /mnt/wi-raid2/gitlab/repositories”) do
      action [:run]
      default_guard_interpreter :default
      declared_type :ruby_block
      cookbook_name “gitlab”
      block #Proc:0x0000562472310f00@/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:35
      block_name “directory resource: /mnt/wi-raid2/gitlab/repositories”
      not_if { #code block }
      end

      System Info:

      chef_version=14.13.11
      platform=ubuntu
      platform_version=18.04
      ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
      program_name=/opt/gitlab/embedded/bin/chef-client
      executable=/opt/gitlab/embedded/bin/chef-client

    ================================================================================
    Error executing action create on resource ‘storage_directory[/mnt/wi-raid2/gitlab/repositories]’

    Mixlib::ShellOut::ShellCommandFailed

    ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Failed asserting that mode permissions on “/mnt/wi-raid2/gitlab/repositories” is 2770
    ---- Begin output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
    STDOUT:
    STDERR: + grep -o …$

    • readlink -f /mnt/wi-raid2/gitlab/repositories
    • stat --printf=%04a /mnt/wi-raid2/gitlab/repositories
    • [ 0770 = 2770 ]
      ---- End output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
      Ran set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] returned 1

    Cookbook Trace:

    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:125:in validate_command' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:113:in block in validate’
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in each_index' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:112:in validate’
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/libraries/storage_directory_helper.rb:87:in validate!' /opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb:43:in block (3 levels) in class_from_file’

    Resource Declaration:

    In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/gitlab-rails.rb

    56: storage_directory repositories_storage[‘path’] do
    57: owner gitlab_user
    58: mode “2770”
    59: end
    60: end

    Compiled Resource:

    Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/gitlab-rails.rb:56:in `block in from_file’

    storage_directory(“/mnt/wi-raid2/gitlab/repositories”) do
    action [:create]
    default_guard_interpreter :default
    declared_type :storage_directory
    cookbook_name “gitlab”
    recipe_name “gitlab-rails”
    owner “git”
    mode “2770”
    path “/mnt/wi-raid2/gitlab/repositories”
    end

    System Info:

    chef_version=14.13.11
    platform=ubuntu
    platform_version=18.04
    ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client

Running handlers:
There was an error running gitlab-ctl reconfigure:

storage_directory[/mnt/wi-raid2/gitlab/repositories] (gitlab::gitlab-rails line 56) had an error: Mixlib::ShellOut::ShellCommandFailed: ruby_block[directory resource: /mnt/wi-raid2/gitlab/repositories] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/package/resources/storage_directory.rb line 34) had an error: Mixlib::ShellOut::ShellCommandFailed: Failed asserting that mode permissions on “/mnt/wi-raid2/gitlab/repositories” is 2770
---- Begin output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
STDOUT:
STDERR: + grep -o …$

  • readlink -f /mnt/wi-raid2/gitlab/repositories
  • stat --printf=%04a /mnt/wi-raid2/gitlab/repositories
  • [ 0770 = 2770 ]
    ---- End output of set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] ----
    Ran set -x && [ “$(stat --printf=‘%04a’ $(readlink -f /mnt/wi-raid2/gitlab/repositories) | grep -o ‘…$’)” = ‘2770’ ] returned 1

Running handlers complete
Chef Client failed. 2 resources updated in 05 seconds
wi-workstation@WastelandsWorkstation:~$

You didn’t follow the commands properly. You’ve now set 755 on all files and directories which was wrong. I said set 755 to /mnt/wi-raid2, /mnt/wi-raid2/gitlab, and /mnt/wi-raid2/gitlab/repositories. Nowhere did I say use -R parameter. And the g+s parameter also was only for the repositories directory, no other directories.

Yes sorry I tried you way first. Same result. I am consulting your repplies with two programmers one is in Ukraine getting in and out of a bomb shelter the other is in Poland. They read your responses and suggested to add the -R . Regardless we tried both result is as you can see the same above.

You can try using:

sudo -E gitlab-ctl reconfigure

-E passes environment stuff through sudo.

Or you can also switch to root using sudo or su and providing root password, and then run it, so:

sudo su
gitlab-ctl reconfigure

or:

su (give root password when prompted)
gitlab-ctl reconfigure

Also, please tell me what filesystem this mount point uses? You can also try:

chmod 2770 /mnt/wi-raid2/gitlab/repositories

and then see if reconfigure works. But also, if filesystem is not supported Linux filesystem, then there will be problems configuring permissions.

wi-workstation@WastelandsWorkstation:~$ mount | column -t
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=8152920k,nr_inodes=2038230,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=1635688k,mode=755)
/dev/sdb2 on / type ext4 (rw,relatime,errors=remount-ro)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/rdma type cgroup (rw,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=24,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17930)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
mqueue on /dev/mqueue type mqueue (rw,relatime)
configfs on /sys/kernel/config type configfs (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
/var/lib/snapd/snaps/gnome-logs_112.snap on /snap/gnome-logs/112 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-characters_781.snap on /snap/gnome-characters/781 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-system-monitor_181.snap on /snap/gnome-system-monitor/181 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-characters_741.snap on /snap/gnome-characters/741 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-34-1804_72.snap on /snap/gnome-3-34-1804/72 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-28-1804_145.snap on /snap/gnome-3-28-1804/145 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/canonical-livepatch_146.snap on /snap/canonical-livepatch/146 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-28-1804_161.snap on /snap/gnome-3-28-1804/161 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-system-monitor_178.snap on /snap/gnome-system-monitor/178 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-calculator_884.snap on /snap/gnome-calculator/884 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/canonical-livepatch_138.snap on /snap/canonical-livepatch/138 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-38-2004_115.snap on /snap/gnome-3-38-2004/115 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gtk-common-themes_1534.snap on /snap/gtk-common-themes/1534 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-34-1804_77.snap on /snap/gnome-3-34-1804/77 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core20_1623.snap on /snap/core20/1623 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-calculator_920.snap on /snap/gnome-calculator/920 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core18_2566.snap on /snap/core18/2566 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core22_275.snap on /snap/core22/275 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-42-2204_29.snap on /snap/gnome-42-2204/29 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-logs_115.snap on /snap/gnome-logs/115 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core_13741.snap on /snap/core/13741 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core_13886.snap on /snap/core/13886 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/bare_5.snap on /snap/bare/5 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core18_2560.snap on /snap/core18/2560 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core20_1611.snap on /snap/core20/1611 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gtk-common-themes_1535.snap on /snap/gtk-common-themes/1535 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-38-2004_119.snap on /snap/gnome-3-38-2004/119 type squashfs (ro,nodev,relatime,x-gdu.hide)
/dev/md127 on /mnt/wi-raid type ext4 (rw,nosuid,nodev,relatime,x-gvfs-show)
/dev/sdb1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
tmpfs on /run/snapd/ns type tmpfs (rw,nosuid,noexec,relatime,size=1635688k,mode=755)
nsfs on /run/snapd/ns/canonical-livepatch.mnt type nsfs (rw)
tmpfs on /run/user/121 type tmpfs (rw,nosuid,nodev,relatime,size=1635684k,mode=700,uid=121,gid=125)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=1635684k,mode=700,uid=1000,gid=1000)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
/dev/md127 on /mnt/wi-raid2 type ext4 (rw,relatime)
/dev/fuse on /run/user/1000/doc type fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
wi-workstation@WastelandsWorkstation:~$

OK, ext4, so that’s fine. Try:

chmod 2770 /mnt/wi-raid2/gitlab/repositories

and then reconfigure again. Either using the sudo -E that I mentioned in previous post, or su to root, and then reconfigure. That way will rule out other permissions, or lack of environment variables.

I think this WORKED ! Let me verify.

I bow to you good Sir. Thank you very much for your help.

1 Like