GitLab is not Responsding 502

Configured GitLab and following are the logs when try to access GitLab after 10 to 15 minutes. For 10-15 minutes, it works very well but soon after that it starts 502 not responding errors. Here are the logs.

==> /var/log/gitlab/unicorn/unicorn_stderr.log <==
E, [2016-02-10T09:31:31.234769 #32221] ERROR – : worker=1 PID:2414 timeout (61s > 60s), killing
E, [2016-02-10T09:31:31.243546 #32221] ERROR – : reaped #<Process::Status: pid 2414 SIGKILL (signal 9)> worker=1
I, [2016-02-10T09:31:31.253130 #2574] INFO – : worker=1 spawned pid=2574
I, [2016-02-10T09:31:31.253959 #2574] INFO – : worker=1 ready

==> /var/log/gitlab/nginx/gitlab_error.log <==
2016/02/10 09:31:31 [error] 32289#0: *226 upstream prematurely closed connection while reading response header from upstream, client: 101.50.105.24, server: 50.244.177.210, request: β€œGET / HTTP/1.1”, upstream: β€œhttp://unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket:/”, host: β€œ50.244.177.210:88”

==> /var/log/gitlab/nginx/gitlab_access.log <==
101.50.105.24 - - [10/Feb/2016:09:31:31 -0500] β€œGET / HTTP/1.1” 502 323 β€œ-” β€œMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36”
101.50.105.24 - - [10/Feb/2016:09:31:31 -0500] β€œGET /favicon.ico HTTP/1.1” 200 32988 β€œhttp://50.244.177.210:88/” β€œMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36”

==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET β€œ/” for 101.50.105.24 at 2016-02-10 09:31:35 -0500
Processing by DashboardController#show as HTML

==> /var/log/gitlab/unicorn/unicorn_stderr.log <==
E, [2016-02-10T09:32:36.398426 #32221] ERROR – : worker=2 PID:2495 timeout (61s > 60s), killing

==> /var/log/gitlab/nginx/gitlab_error.log <==
2016/02/10 09:32:36 [error] 32289#0: *226 upstream prematurely closed connection while reading response header from upstream, client: 101.50.105.24, server: 50.244.177.210, request: β€œGET / HTTP/1.1”, upstream: β€œhttp://unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket:/”, host: β€œ50.244.177.210:88”

==> /var/log/gitlab/nginx/gitlab_access.log <==
101.50.105.24 - - [10/Feb/2016:09:32:36 -0500] β€œGET / HTTP/1.1” 502 323 β€œ-” β€œMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36”

==> /var/log/gitlab/unicorn/unicorn_stderr.log <==
E, [2016-02-10T09:32:36.409427 #32221] ERROR – : reaped #<Process::Status: pid 2495 SIGKILL (signal 9)> worker=2
I, [2016-02-10T09:32:36.505577 #2654] INFO – : worker=2 spawned pid=2654
I, [2016-02-10T09:32:36.507864 #2654] INFO – : worker=2 ready

==> /var/log/gitlab/nginx/gitlab_access.log <==
101.50.105.24 - - [10/Feb/2016:09:32:36 -0500] β€œGET /static.css HTTP/1.1” 304 0 β€œhttp://50.244.177.210:88/” β€œMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36”
101.50.105.24 - - [10/Feb/2016:09:32:37 -0500] β€œGET /favicon.ico HTTP/1.1” 200 32988 β€œhttp://50.244.177.210:88/” β€œMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36”

This is gitlab status:

/var/log/gitlab/unicorn# gitlab-ctl status
run: logrotate: (pid 3947) 206s; run: log: (pid 10738) 17144s
run: nginx: (pid 3952) 206s; run: log: (pid 10727) 17146s
run: postgresql: (pid 3959) 204s; run: log: (pid 10630) 17200s
run: redis: (pid 3962) 204s; run: log: (pid 10539) 17211s
run: sidekiq: (pid 4229) 69s; run: log: (pid 10712) 17148s
run: unicorn: (pid 32296) 3055s, got TERM; run: log: (pid 10690) 17154s

Any help? Thanks

1 Like