GitLab CE in Azure - Cannot push files

I installed GitLab CE on an Azure VM through the Marketplace. I followed the instructions here and added a custom DNS [project] Everything seems to work well and I can access the web interface. When I created a new project, it gave me a URL of gitlab-ce.[longtoken] which is not resolvable when I try to push the project files from my local machine.

I tried to update gitlab.rb to use the domain, but that breaks the web interface. Not sure how to proceed. Any thoughts would be greatly appreciated.