Gitlab pages return 502 error

I have the ce omnibus installation and it is up to date with regards to patches.
This deployment has been around for a long time and things have been working well.
Users are now reporting 502 error when they access the “pages” site of a project.
I have checked for errors using check-config and reconfigure runs without errors. I have not made any config changes, just updates on a monthly basis.
The only error I see is:

{“error”:“Get "http://gue-lin-git.rwdigroup.net/api/v4/internal/pages?host=pages.devdoc.rwdigroup.net\”: dial tcp: lookup gue-lin-git.rwdigroup.net on [::1]:53: read udp [::1]:33647-\u003e[::1]:53: read: connection refused",“level”:“error”,“msg”:“could not fetch domain information from a source”,“time”:“2021-01-19T13:29:08Z”}
{“error”:“Get "http://gue-lin-git.rwdigroup.net/api/v4/internal/pages?host=pages.devdoc.rwdigroup.net\”: dial tcp: lookup gue-lin-git.rwdigroup.net on [::1]:53: read udp [::1]:54970-\u003e[::1]:53: read: connection refused",“level”:“error”,“msg”:“could not fetch domain information from a source”,“time”:“2021-01-19T13:31:04Z”}

I checked/confirmed dns can resolve both gue-lin-git.rwdigroup.net and pages.devdoc.rwdigroup.net.

I am at a loss as to why this is now happening. Any thoughts by all would be appreciated.

Thanks in Advance

Hey,

we have exactly the same troubles for approximately one week. Seems to happen out of nowhere, the site run for month without any troubles.

We use GitLab 13.7.3, do you have the same version?

Greetings
Michael

cc @dnsmichi

@MarcoA We fixed it by making sure that the external_url in the gitlab.rb config file is FQDN url. After reconfigure it worked.

Greetings
Michael

We have the same issue on 13.8.3. It has been working fine for years and then out of the blue today we got the same error. No updates were done to GitLab or the OS, but it just seemed to stop all of a sudden. Checked the external_url, but that is the same as it has always been with a FQDN.
Bit of a weird one. If anyone has any other suggestions on how to fix it, that would be good.
Cheers
Alan.

Hi,

I’m having the same problem here, on omnibus starter edition, 13.8.1. I fixed it once using this trick: https://docs.gitlab.com/ee/administration/pages/#502-error-when-connecting-to-gitlab-pages-proxy-when-server-does-not-listen-over-ipv6 but the problem is back again, after several days (the pages are generated every day and I’ve checked them each time since the first problem occur).

@tonka3000 : Can you provide a little bit more details about your fix? I’m not an expert here and this FQDN thing could be a good fix but I don’t get it fully.

Thanks in advance,
Best Regards,
Julien

I’m running on centos 7 . In my case ipv6 was enabled but not being used. Disabling it cured the problem. I did restart the machine in the process.

Well this keeps happening at what appears a random interval, it works for days at a time and then pages stopped being served, I then do a gitlab-crl restart and the pages come back as expected. Clearly something is killing the pages daemon, but I am unsure where the logs would be to get an idea of what is happening.

This is happening for us as well. We’re running 13.9.3

The error message in the log is below. domains have been replaced with example.com:

{“error”:“Get “https://example.com/api/v4/internal/pages?host=blah.pages.example.com”: dial tcp: lookup example.com on [::1]:53: read udp [::1]:43190-\u003e[::1]:53: read: connection refused”,“level”:“error”,“msg”:“could not fetch domain information from a source”,“time”:“2021-03-30T17:24:41Z”}

I checked that I could resolve the domain with dig and that I could curl that URL from that machine. We have GitLab running on the same server as the GitLab Pages server.

Hi, there may be an answer here: GitLab Pages administration | GitLab

So far it’s working for me, let’s see how it will go the next few days/weeks.

Another solution is to simply call the https://gitlabce.example.localhost/users/sign_in page

Hope this works and no more 502

I was having the 502 error too.

In my case, I upgraded to 14.0, then in a few days I started noticing the error. But I’m not sure the upgrade is related.

I tried lots of things. Including the upgrade to 14.3 as recommended by the Gitlab docs. Nothing worked. Eventually the problem went away, some 3 hours after the upgrade to 14.3. I didn’t do any other meaningful change.

So, try upgrading it, I guess.

Update: I’ve had a slightly different problem, but kinda related (was also resulting in 502), so I’ll post anyway - it may help someone…

It happened to me also. In my case it was due to a configuration conflict, but thing that confuses me is how it could work at all earlier? Anyway, here’s what I’ve found and my solution:

When the problem has arisen, in /var/log/gitlab/gitlab-pages/current I’ve found errors that contain something like:

Get \"https://gitlab.mygitlabdomain.com/api/v4/internal/pages?host=pagename.mypagesdomain.io\": dial tcp 127.0.1.1:443`.

Obviously Gitlab pages daemon was trying to access Gitlab API, but it failed. In my case the reason was that the connection attempt is made on 127.0.1.1 address, and my NginX is not configured to listen there. My NginX config contains:

nginx['listen_addresses'] = ['172.16.2.25']

Why the attempt is made on 127.0.1.1? Well because in my /etc/hosts I have:

127.0.1.1 gitlab.mygitlabdomain.com gitlab

I’ve solved it by setting:

nginx['listen_addresses'] = ['172.16.2.25','127.0.1.1']

Alternative solution would probably be to change the /etc/hosts file so that it contains:

172.16.2.25 gitlab.mygitlabdomain.com gitlab

I haven’t tested the second solution, but chances are that it would also solve the problem.

The key question for me is how it was working earlier with the same config? And it was working…

But anyway, I’ve solved the problem in my case.

I’m running on centos 7 . In my case ipv6 was enabled but not being used..

.