Permission denied and 'no file or directory' errors

How to Use GitLab
1

Recently I upgraded from ver 7 to 8 via omnibus and have been having trouble getting pull/push/clone to work. This is on Ubuntu 14.04. The web pages load fine but any interaction via git seems to be messed up.

Doing a git clone via http gives this error in the gitlab_error.log:

2016/03/27 03:58:22 [crit] 2718#0: *6321 connect() to unix://var/opt/gitlab/gitlab-workhorse/socket failed (13: Permission denied) while connecting to upstream, client: 11.11.11.111, server: DOMAIN.COM, request: “GET /AUser/calendarview.git/info/refs?service=git-upload-pack HTTP/1.1”, upstream: “http://unix://var/opt/gitlab/gitlab-workhorse/socket:/AUser/calendarview.git/info/refs?service=git-upload-pack”, host: “DOMAIN.COM”

Doing a git clone via git/ssl gives this error in the log:

2016/03/27 04:44:19 [error] 22129#0: *522 open() “/opt/gitlab/embedded/service/gitlab-rails/public/api/v3/internal/allowed” failed (2: No such file or directory), client: 11.11.11.111, server: DOMAIN.COM, request: “POST /api/v3/internal/allowed HTTP/1.1”, host: “DOMAIN.COM”

I’m using an already existing Nginx installation w/ Passenger.

sudo -u www-data ls -la /var/opt/gitlab/gitlab-workhorse/socket prints
srwxrwxrwx 1 git git 0 Mar 27 03:58 /var/opt/gitlab/gitlab-workhorse/socket just fine

relevant /etc/nginx/nginx.conf parts

user www-data;
worker_processes 4;
pid /run/nginx.pid;

events {
  worker_connections 768;
  # multi_accept on;
}

http {

	# Passenger Settings
	passenger_root /var/lib/gems/1.9.1/gems/passenger-5.0.26;
	passenger_app_env production;
	passenger_friendly_error_pages on;
  
	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# Logging Settings
	##

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	##
	# Gzip Settings
	gzip on;
	gzip_disable "msie6";

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javasc$

	##
	# nginx-naxsi config
	##
	# Uncomment it if you installed nginx-naxsi
	##

	#include /etc/nginx/naxsi_core.rules;

	##
	# nginx-passenger config
	##
	# Uncomment it if you installed nginx-passenger
	##

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	#include /etc/nginx/sites-enabled/*;


  upstream gitlab-workhorse {
    server unix:/var/opt/gitlab/gitlab-workhorse/socket fail_timeout=0;
  }
  
  server {
    listen *:80;
    server_name DOMAIN.COM;
    server_tokens off;
    root /opt/gitlab/embedded/service/gitlab-rails/public;
  
    client_max_body_size 250m;
  
    access_log  /var/log/gitlab/nginx/gitlab_access.log;
    error_log   /var/log/gitlab/nginx/gitlab_error.log;
  
    # Ensure Passenger uses the bundled Ruby version
    passenger_ruby /opt/gitlab/embedded/bin/ruby;
  
    # Correct the $PATH variable to included packaged executables
    passenger_env_var PATH "/opt/gitlab/bin:/opt/gitlab/embedded/bin:/usr/local/bin:/usr/bin:/bin";
  
    # Make sure Passenger runs as the correct user and group to
    # prevent permission issues
    passenger_user git;
    passenger_group git;
  
    # Enable Passenger & keep at least one instance running at all times
    passenger_enabled on;
    passenger_min_instances 1;
  
    location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
      # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
      error_page 418 = @gitlab-workhorse;
      return 418;
    }
  
    location ~ ^/[\w\.-]+/[\w\.-]+/repository/archive {
      # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
      error_page 418 = @gitlab-workhorse;
      return 418;
    }
  
    location ~ ^/api/v3/projects/.*/repository/archive {
      # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
      error_page 418 = @gitlab-workhorse;
      return 418;
    }
  
    # Build artifacts should be submitted to this location
    location ~ ^/[\w\.-]+/[\w\.-]+/builds/download {
        client_max_body_size 0;
        # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
        error_page 418 = @gitlab-workhorse;
        return 418;
    }
  
    # Build artifacts should be submitted to this location
    location ~ /ci/api/v1/builds/[0-9]+/artifacts {
        client_max_body_size 0;
        # 'Error' 418 is a hack to re-use the @gitlab-workhorse block
        error_page 418 = @gitlab-workhorse;
        return 418;
    }
  
    location @gitlab-workhorse {
  
      ## https://github.com/gitlabhq/gitlabhq/issues/694
      ## Some requests take more than 30 seconds.
      proxy_read_timeout      300;
      proxy_connect_timeout   300;
      proxy_redirect          off;
  
      # Do not buffer Git HTTP responses
      proxy_buffering off;
  
      proxy_set_header    Host                $http_host;
      proxy_set_header    X-Real-IP           $remote_addr;
      proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
      proxy_set_header    X-Forwarded-Proto   $scheme;
  
      proxy_pass http://gitlab-workhorse;
  
      ## The following settings only work with NGINX 1.7.11 or newer
      #
      ## Pass chunked request bodies to gitlab-workhorse as-is
      # proxy_request_buffering off;
      # proxy_http_version 1.1;
    }
  
    ## Enable gzip compression as per rails guide:
    ## http://guides.rubyonrails.org/asset_pipeline.html#gzip-compression
    ## WARNING: If you are using relative urls remove the block below
    ## See config/application.rb under "Relative url support" for the list of
    ## other files that need to be changed for relative url support
    location ~ ^/(assets)/ {
      root /opt/gitlab/embedded/service/gitlab-rails/public;
      gzip_static on; # to serve pre-gzipped version
      expires max;
      add_header Cache-Control public;
    }
  
    error_page 502 /502.html;
  }

}

/etc/gitlab/gitlab.rb

# Disable the built-in nginx
nginx['enable'] = false

# Disable the built-in nginx for Gitlab CI
ci_nginx['enable'] = false

# Disable the built-in unicorn
unicorn['enable'] = false

# Set the internal API URL
gitlab_rails['internal_api_url'] = 'http://DOMAIN.COM'

external_url "http://DOMAIN.COM/"
gitlab_rails['gitlab_email_from'] = "gitlab@DOMAIN.COM"
gitlab_rails['gitlab_support_email'] = "gitlab@DOMAIN.COM"
#nginx['redirect_http_to_https'] = false
#nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.crt"
#nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.key"
ci_external_url 'http://ci.DOMAIN.COM'

gitlab_rails['omniauth_providers'] = [
    {
      "name" => "github",
      "app_id" => "APP_ID",
      "app_secret" => "APP_SECRET",
      "url" => "https://github.com/",
      "args" => { "scope" => "user:email" }
    }
  ]

# Tried this too
# web_server['external_users'] = ['nginx','gitlab-www','git']

I have done gitlab-ctl reconfigure with this and it doesn’t fix the error.

If there is something I’m missing, I’d love to know.

2

I ended up just reimaging the server with a fresh Ubuntu image and reinstalling. Originally we installed with a Gitlab image from Digital Ocean which was kind of wonky.

I think the problem was that I had unicorn disabled. I’m not sure if things changed between v7 an v8, but running in v7 we were able to do without the unicorn webserver. I suppose this means we don’t need the passenger stuff now either (we’re not reliant on passenger in our setup).

Edit: To clarify, reimagining and turning unicorn back on and using the nginx conf here fixed it. I had to change the location of the socket in the upstream section since this conf assumes you did a source install and I’m on a omnibus install (omnibus install socket is at /var/opt/gitlab/gitlab-workhorse/socket).

3

Thank you for reporting back.

Just to add a bit though, you’ve mentioned that you did a clean reinstall via OmniBus but you still had to include some additional edits? to get it working? The omnibus installation should work straight out of the box. If you experience some additional issues please open a new thread and link to this conversation :slight_smile: