Internal Server Error 502 - Postgresql could not fsync file “./base/16385/18631”: Cannot allocate memory
Problem
We started to have a hard disk space problem with the gitlab ee container reaching its maximum capacity.
After freeing up space in the virtual machine where the gitlab ee container is located, it gives us the following problem continuously with the Postgresql database:
2021-07-30_21:51:03.08968 FATAL: the database system is starting up
2021-07-30_21:51:03.27052 FATAL: the database system is starting up
2021-07-30_21:51:03.45126 FATAL: the database system is starting up
2021-07-30_21:51:03.48419 FATAL: the database system is starting up
2021-07-30_21:51:07.49525 LOG: could not fsync file “./base/16385/18631”: Cannot allocate memory
2021-07-30_21:51:12.37920 FATAL: the database system is starting up
2021-07-30_21:51:12.39835 FATAL: the database system is starting up
2021-07-30_21:51:12.41203 FATAL: the database system is starting up
2021-07-30_21:51:12.65906 FATAL: the database system is starting up
2021-07-30_21:51:13.36838 LOG: database system was not properly shut down; automatic recovery in progress
2021-07-30_21:51:13.40212 LOG: redo starts at 1/50D8D290
2021-07-30_21:51:13.40319 LOG: invalid record length at 1/50DD0190: wanted 24, got 0
2021-07-30_21:51:13.40320 LOG: redo done at 1/50DD0168
2021-07-30_21:51:13.40320 LOG: last completed transaction was at log time 2021-07-30 18:59:09.189807+00
2021-07-30_21:51:13.69861 FATAL: could not fsync file “base/16385/18631”: Cannot allocate memory
2021-07-30_21:51:13.70114 LOG: startup process (PID 7959) exited with exit code 1
2021-07-30_21:51:13.70114 LOG: aborting startup due to startup process failure
2021-07-30_21:51:13.70115 LOG: database system is shut down
2021-07-30_21:51:13.92404 LOG: database system shutdown was interrupted; last known up at 2021-07-30 21:51:13 GMT
2021-07-30_21:51:13.96408 WARNING: could not flush dirty data: Cannot allocate memory
2021-07-30_21:51:16.27557 FATAL: the database system is starting up
2021-07-30_21:51:16.48136 FATAL: the database system is starting up
2021-07-30_21:51:16.64223 FATAL: the database system is starting up
2021-07-30_21:51:16.67136 FATAL: the database system is starting up
2021-07-30_21:51:19.97905 LOG: could not fsync file “./base/16385/18631”: Cannot allocate memory
Specifications
Linux SO: Red Hat Enterprise Linux Server - 7.6 (Maipo)
Docker version 18.09.6, build 481bc77156
Image: gitlab/gitlab-ee:11.11.3-ee.0
Postgresql version: 9.6
Notes
- The default database provided by gitlab is being used.
- I have searched in different forums but none makes mention of this type of error.
Question
- How could it be corrected or what are the steps to be followed to correct this problem?