This isn’t related to Gitlab. DNS over HTTPS instead of using normal DNS on UDP port 53 or perhaps TCP port 53, is sending the DNS requests over HTTPS instead.
Therefore, if I have an internal Gitlab instance, configured in my internal DNS, I cannot resolve this in DNS over HTTPS if I’m using for example Cloudflare as the provider in Firefox configuration. I would need to have a public configured DNS entry that Cloudflare would see and be able to resolve for my server to be accessible.
Also, the same is if I decide, well, I want a custom DNS over HTTPS, and I input my internal DNS server instead, it also won’t work, because I don’t have a HTTPS listener configured on my DNS servers for it to resolve the DNS entries.
I would need on my internal DNS servers a HTTPS listener, to redirect the requests to the DNS daemon on my servers, to resolve the entry and send the reply back over HTTPS.
This link will help for custom DNS over HTTPS: Set Up DNS over HTTPS (DoH) Resolver on Debian with DNSdist if I wanted to configure on my own site. That way I can use my own internal DNS servers and provide the resolution over HTTPS. Then it would work fine.
What you can also do is make exceptions for DNS entries that cannot be resolved by DNS over HTTPS: Firefox DNS-over-HTTPS | Firefox Help