Hello.
We are facing the following issue with our self-hosted Gitlab for several weeks now.
All of a sudden (after 2 years of working) Gitlab stopped working, producing 502 page. After running “reconfigure” command, it works for several seconds/minutes and again show 502 page.
Logs show following error:
==> /var/log/gitlab/nginx/gitlab_access.log <==
10.0.0.1 - - [24/Jun/2022:08:37:21 +0000] "GET /admin HTTP/1.0" 502 2940 "" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36" -
==> /var/log/gitlab/nginx/gitlab_error.log <==
2022/06/24 08:37:42 [crit] 567#0: *274 connect() to unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket failed (2: No such file or directory) while connecting to upstream, client: 10.0.0.1, server: gitlab.nervtech.com, request: "GET / HTTP/1.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/", host: "gitlab.nervtech.com"
It seems like after some time, sockets/socket file gets removed by the system and nginx server doesn’t find it anymore. The same goes with config.toml file, both files are located in /var/opt/gitlab/gitlab-workhorse/ folder after reconfigure.
This is driving us crazy, since we cannot determine what causes those files to be removed.
Any help or suggestions would be much appreciated.
Klemen