Problem, workaround and Suggestion : gitlab-runner docker-windows

This is a problem, a workaround I have found and a suggestion for the developers to improve error reporting for gitlabs runner “docker-windows”

My environment;

1. Latest Gitlab self-hosted on a Virtual machine “gitvm” on Suse Linux host.
2. Separate Machine running Windows 10 Pro 21H2 “supersally” and Docker with windows containers
   All latest patches/updates/service packs installed.

What I’m trying to do;
Create a CI/CD job to run a docker image on the Windows machine to build some Visual Basic code.

What works;
I have built a docker image, and can fetch/build the code manually from CLI sat at the windows machine. I added gitvm to the hosts file of the image by manually editing the hosts file, and by using the --add-hostname command.
The docker host and docker image can both ping “gitvm” by name.
I can use git (which I installed in the image) to fetch the code from my repository, then MSBUILD to build it. Yay !
I can push the container to gitvm, and it exists in the contaier repository.

What Doesn’t work
If I register and use gitlab-runner on “supersally”, and create a pipeline to build the source, then it will fetch the image from the container registry, but it fails when trying to fetch the source code - see below;

My workaround
The workaround is quite simple, I put the following entry in .gitlab-ci.yaml

  variables:
    GIT_STRATEGY: none

and get my script to call a powershell screipt inside the container which performs a git clone

What I have discovered;
As with other people I have found that the windows runner can fail with a message

 encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The user name or password is incorrect. (0x52e)

unlike other people who have posted similar error messages which appear as part of the event log, the entry inside Event Viewer for this particular issue doesn’t actually tell you what command it was trying to run.

in other examples I have discovered online, there’s actually a like under the last one that tells you what the command was…eg:

Provider: 00000000-0000-0000-0000-000000000000] extra info: {“CommandLine”:“cmd.exe /C “ECHO 192.100.1.4 host.docker.internal \u003e\u003e %systemroot%\system32\drivers\etc\hosts \u0026 ECHO 192.100.1.4 gateway.docker.internal \u003e\u003e %systemroot%\system32\drivers\etc\hosts””,“User”:“Administrator”,“WorkingDirectory”:“C:\inetpub\wwwroot”,"Environment

My suggestion to the developers;
If possible add the command that was being issued when trying to fetch the source to the Windows Event log !

What I haven’t been able to figure out is why it can’t seem to resolve the name gitvm, when it should all be fine and dandy, and what is the command that the runner is trying to execute that it can’t, and how would I fix the issue so for now, I seem to be stuck with my workaround.

Anyone else have any ideas, suggestions, or solutions to my probelm?

I don’t know how else to get any additional information, and adding

log_level = "debug"

to config.toml hasn’t given any further insight.

Thanks for looking,
Paul.

You could probably save a lot of issues by actually using DNS rather than entries in hosts files. I would expect that would resolve a lot of your problems.

Instead of gitvm as a hosts entry, add it to a DNS server so that it resolves properly, eg: gitmv.example.com and ensure the machines running the gitlab-runner utilise the DNS server where you configure the domain so that they all also see and resolve gitvm.example.com.

Assuming you have Active Directory, then there is a DNS server in Windows where you could do this, and ensure all your machines use that DNS Server for domain resolution. If not, then it’s simple enough to make a DNS server with say bind on Linux and then use that server for resolving DNS. Whichever way, also ensure that whichever DNS server you use has forwarders configured, so that it can resolve other domains external to your network, eg: google.com, docker.com, etc, etc. Effectively, an internal zone for example.com or whatever your domain would be where gitvm resolves, and everything else gets forwarded to external DNS servers.

Hi Ian,
that’s a good call. Unfortunatey I am not in control of our network and some (not all) machines do not resolve properly through DNS. Our IT Admin has configured our Zyxel router to be a DHCP and DNS server, and although it will dish out IP addresses it then won’t allow some machines to be pinged by name. Hence the entries in the hosts file.

We are only a small company and I don’t think orur IT guy has the router configured properly, but he wont give me the password to it in order to try to change things in case I screw up something else.

We only have the Zyxel router and a TrueNAS fileserver, so no Active directory. Perhaps I can try setting up a raspberry Pi to do the name resolution for the Windows host and/or docker container, then as you say pass it forward for anything it can’t resolve.

I will post back as/when I get chance to try this.

Paul,

Yeah you could do that. You could set up your own DNS server with, let’s say for example bind. For example on my bind install on Debian, in /etc/bind/named.conf.options you configure a forwarders section like this:

	 forwarders {
		208.67.222.222;
		208.67.220.220;
	 	8.8.8.8;
		8.8.4.4;
	 };

obviously, change the IP addresses to ones that you want to use - maybe your Zyxel already gives certain DNS server entries via DHCP, so you can check what your existing ones are using ipconfig under Windows and then use these in the forwarders section. My example is just using Google DNS and OpenDNS.

Then just configure yourself a zone for an internal domain that you want to resolve (so that it doesn’t conflict with something external), maybe mynetwork.lan and use this for your internal servers, etc. All your internal servers and desktops would need to be configured to use the DNS that you configured on the Raspberry PI or whatever you chose to use. This can be configured statically under Linux/Windows, but you can still get your IP via DHCP.