Hi everybody,
We’ve been running Gitlab inside Docker for a while now, and until a month ago (since feb 1) everything worked fine. We are adding more and more projects to it, and are running into strange behaviour recently.
The container Gitlab is running is becomes spontaneously ‘unhealthy’ in the early morning. The only way to recover is to restart the docker daemon, trying to restart or kill the container is not working at all.
It seems to be related to memory usage on the machine. it has 12GB. We monitor this and see high loads, 100% memory usage (including swap) when the contains becomes unhealthy. The normal behaviour of Linux is to kill processes (even in the container) at random to free up memory which can easily account for the erratic behaviour we’re seeing.
However we’re at a loss how to proceed from here. Does anybody have a suggestion on how to analyse where the huge memory consumption comes from?
Thanks in advance,