I already used gitlab pages and custom domains without any problems. I followed the guide here about adding a custom domain on gitlab pages and I was able to enable HTTPS and A record while I was not enable to setup CNAME record to redirect www.example.com to example.com.
I’m using Publii and a similar problem is described here.
;; ANSWER SECTION:
www.example.com. 10800 IN CNAME example.gitlab.io.
example.gitlab.io. 300 IN A 18.104.22.168