I got 502 error and I have no idea what is wrong!

inhesion · January 19, 2016, 12:39pm

Can’t deal with centos 7 gitlab omnibus and my preinstalled nginx server

At first I installed and setuped host, so that I had working nginx server with mariadb and php-fpm. Everything was cool and working. Then I installed gitlab via omnibus package, set up external url and reconfigured. Now I only have access to gitlab, and my nginx server hosts are unavailable. That was first problem. I tried to set up gitlab.rb with nginx[‘enable’] = false and web_server[‘external_users’] = [‘nginx’], and my nginx server host came back to access but now I got 502 error when I try to access gitlab server via host set in external_url. My nginx-user is ‘nginx’.

Here is my config for my nginx gitlab.config

user nginx;
#user git root;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 2048;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    #include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*.conf;

    server {Can't deal with centos 7 gitlab omnibus and my preinstalled nginx server
        listen       80 default_server;
        listen       [::]:80 default_server;
        server_name  _;
        root         /usr/share/nginx/html;

        # Load configuration files for the default server block.
        include /etc/nginx/default.d/*.conf;

        location / {
        }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }
}

Here is my gitlab.conf file which is located in /etc/nginx/sites-available/ and synlinked to sites-enabled

upstream gitlab {
  server unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fail_timeout=0;
}

upstream gitlab-workhorse {
  server unix:/var/opt/gitlab/gitlab-workhorse/socket;
}

server {
  listen 0.0.0.0:80;
  listen [::]:80;
  server_name code.nac.in.ua; ## Replace this with something like gitlab.example.com
  server_tokens off; ## Don't show the nginx version number, a security best practice
  root /opt/gitlab/embedded/service/gitlab-rails/public;

  client_max_body_size 20m;

  access_log  /var/log/nginx/gitlab_access.log;
  error_log   /var/log/nginx/gitlab_error.log;

  location / {
    try_files $uri $uri/index.html $uri.html @gitlab;
  }

  location /uploads/ {
    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;
    proxy_set_header    X-Frame-Options     SAMEORIGIN;

    proxy_pass http://localhost:8080;
  }

  location @gitlab {
    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;
    proxy_set_header    X-Frame-Options     SAMEORIGIN;

    proxy_pass http://gitlab;
  }

  location ~ ^/[\w\.-]+/[\w\.-]+/gitlab-lfs/objects {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
  }

  location ~ ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$ {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
  }

  location ~ ^/[\w\.-]+/[\w\.-]+/repository/archive {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
  }

  location ~ ^/api/v3/projects/.*/repository/archive {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
  }

  location ~ ^/[\w\.-]+/[\w\.-]+/builds/download {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
  }

  location ~ /ci/api/v1/builds/[0-9]+/artifacts {
    client_max_body_size 0;
    error_page 418 = @gitlab-workhorse;
    return 418;
  }

  location @gitlab-workhorse {
    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_buffering off;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;

    proxy_pass http://gitlab-workhorse;
  }

  location ~ ^/(assets)/ {
    root /opt/gitlab/embedded/service/gitlab-rails/public;
    gzip_static on; # to serve pre-gzipped version
    expires max;
    add_header Cache-Control public;
  }

  error_page 502 /502.html;
}

Here is tail of /var/log/nginx/gitlab_error.log

2016/01/19 13:59:47 [error] 6984#0: *1 connect() to unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket failed (111: Connection refused) while connecting to upstream, client: 194.187.130.80, server: code.nac.in.ua, request: "GET / HTTP/1.1", upstream: "http://unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket:/", host: "code.nac.in.ua"
2016/01/19 14:01:16 [error] 7276#0: *1 connect() to unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket failed (111: Connection refused) while connecting to upstream, client: 194.187.130.80, server: code.nac.in.ua, request: "GET / HTTP/1.1", upstream: "http://unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket:/", host: "code.nac.in.ua"
2016/01/19 14:01:17 [error] 7276#0: *1 connect() to unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket failed (111: Connection refused) while connecting to upstream, client: 194.187.130.80, server: code.nac.in.ua, request: "GET / HTTP/1.1", upstream: "http://unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket:/", host: "code.nac.in.ua"
/var/log/nginx/gitlab_error.log

Here is tail of /var/log/gitlab/unicorn/unicorn_stderr.log

[2016-01-19T14:35:16.845126 #11189] FATAL -- : error adding listener addr=/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket
/opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/socket_helper.rb:144:in `initialize': Permission denied - connect(2) for /var/opt/gitlab/gitlab-rails/sockets/gitlab.socket (Errno::EACCES)
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/socket_helper.rb:144:in `new'
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/socket_helper.rb:144:in `bind_listen'
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:242:in `listen'
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:809:in `block in bind_new_listeners!'
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:809:in `each'
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:809:in `bind_new_listeners!'
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:138:in `start'
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/unicorn-4.8.3/bin/unicorn:126:in `<top (required)>'
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/unicorn:23:in `load'
        from /opt/gitlab/embedded/service/gem/ruby/2.1.0/bin/unicorn:23:in `<main>'

Here is permissions for gitlab.socket

$ sudo ls -la /var/opt/gitlab/gitlab-rails/sockets/
total 8
drwxr-x---. 2 git   gitlab-www 4096 Jan 19 13:27 .
drwxr-xr-x. 8 git   root       4096 Jan 19 14:28 ..
srw-rw----. 1 nginx gitlab-www    0 Jan 19 13:27 gitlab.socket

Here is output for $ id nginx
uid=995(nginx) gid=993(nginx) groups=993(nginx),992(gitlab-www)

If I comment gitlab.rb nginx=false my gitlab server will work fine, but without my nginx server hosts. Can anybody help me. I’ve googled 2 weeks every day with no changes

newton · February 1, 2016, 4:51pm

Hey inhesion,

As I previously pointed out in #gitlab, your gitlab configs are out of date. Your config should basically mirror https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/gitlab other then making adjustments to paths.

inhesion · February 2, 2016, 7:46am

Okay, but I’m confused. I have no /home/git/ directory in! I installed my gitlab via omnibus package! What yo will suggest now for me?!

axil · February 6, 2016, 2:11pm

@inhesion use the configs from https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/nginx.

Follow the steps in the documentation http://doc.gitlab.com/omnibus/settings/nginx.html#using-a-non-bundled-web-server

Topic		Replies	Views
502 Error Setup Omnibus Gitlab with Unbundled Nginx How to Use GitLab	1	1668	December 25, 2017
On going 502 error no idea why How to Use GitLab	11	19422	April 27, 2016
Git Lab 502 - Cent OS How to Use GitLab	6	4147	July 20, 2016
Getting a 503 error when trying access gitlab How to Use GitLab	0	2100	February 11, 2016
Error 502 with GitLab installation on Ubuntu 16.04 How to Use GitLab	2	2910	December 5, 2018

I got 502 error and I have no idea what is wrong!

Related topics