Issues updating to gitlab-ce:15.11.x / Updating PostgreSQL 12 to 13.11

Hello there,

I’m facing issues updating from gitlab-ce:15.10.x to gitlab-ce:15.11.x ,
more accurate Updating PostgreSQL 12 to 13.11

“gitlab-ctl pg-upgrade -V 13” fails with:

There was an error fetching locale and encoding information from the database
Please ensure the database is running and functional before running pg-upgrade
STDERR: psql: error: FATAL: database “postgres” does not exist
== Fatal error ==
Please check error logs
== Reverting ==

gitlab-psql > \l shows:

   Name         |    Owner    | Encoding | Collate |  Ctype  | Access privileges 

gitlabhq_production | gitlab | UTF8 | C.UTF-8 | C.UTF-8 |
template1 | gitlab-psql | UTF8 | C.UTF-8 | C.UTF-8 |

so fair enough the database"postgres" does not exist really doesn’t exisits.

my docker-compose.yml:

version: “3.7”

image: gitlab/gitlab-ce:15.11.8-ce.0
container_name: dms
restart: unless-stopped
hostname: “xxxxxxxxxxx”
- “9700:443”
- “9701:80”
- “9702:22”
- /home/ji/gitlabdata/dms-etc:/etc/gitlab
- /home/ji/gitlabdata/dms-logs:/var/log/gitlab
- /home/ji/gitlabdata/dms-data:/var/opt/gitlab
network_mode: bridge

note: this is a fairly old gitlab installation migrate and upgrated from varios gitlab versions.

I have no Idea to keep on searching for soulutions.
any ideas are highly appreciated.

push :slight_smile:

I’m not sure how you can have a postgres installation without a postgres database. All my Gitlab installs have a postgres database. That said I don’t use docker, but even so, I would expect them to have it as well. The question is, what happened to yours? I don’t believe postgres can function without it (unless I’m mistaken?).

GitLab in Docker is all-in-one container - it does not have any containers next to it :slight_smile:

That being said, normally GitLab has everything it needs and all migrations also in that container - so I think you don’t even need to do this by yourself? Normally you just update the docker tag and within new image should be everything updated (including postgres) and all necessary scripts normally run automatically to make sure everything is fine.

So, I think updating posgres manually in this case is not needed… but I might be wrong as well. I’m myself on GL 15.10, but I will upgrade soon to 15.11 so I’ll get back here to post my experiences.

I’m not sure what this variable “GITLAB_SKIP_PG_UPGRADE=true” does though and why did you put it. Maybe it prevents automatic postgres updates… but why did you do that in the first place?

I’m sorry, perhaps I didn’t describe the problem accurately enough.
We are using the “all-in-one” Docker container/image.

The update from gitlab-ce:15.10.x to gitlab-ce:15.11.x starts the issue.

With this version upgrade, PostgreSQL 13.11 is used.
The existing PostgreSQL 12 version in the container is supposed to be updated during the image switch.
However, this fails with the error code:

“STDERR: psql: error: FATAL: database “postgres” does not exist”

A confusing error code for me.

The PostgreSQL database in my container is named “gitlabhq_production.”
There is no database with the name “postgres.”

I have set the environment variable “GITLAB_SKIP_PG_UPGRADE=true” as per the documentation to ensure the container starts at all and the PostgreSQL Update will be Skipped.
What is blocking my upgrade path to GitLab 16.x is the requirement of PostgreSQL 13, as GitLab 16.x relies on it.

Ah okay… so basically it failed to automatically upgrade postgres?

Hmm. Interesting. Could you check if all background migrations have finished? Perhaps guys have changed the database name recently.

I have updated yesterday to 15.11.9 with no issues, Postgres was also updated to v13. I have no idea what could have happened to your installation (to be more precise - why there seems to be a mismatch between database names in reality and migration script). In any case I’d seek more professional help. Unfortunately, with community edition you don’t have access to customer support portal, so I guess the only thing that is left is try to get someone from gitlab team here or maybe creating an issue on ? But both approaches will probably take some time until you have a solution.