I’m trying to get a local (self-managed) gitlab-ce environment setup for testing CI/CD functions for eventual deployment into our team’s Gitlab installation - using Kubernetes as my executor. My pipeline starts ok but the gitlab-runner-helper container appears to have an issue trying to upload artifacts to the coordinator, as it’s my understanding the gitlab-runner-helper image is used to both push/pull artifacts at the end/start of each job in a gitlab pipeline.
From a job’s displayed log:
Uploading artifacts... dir1/**/dist/*.whl: found 1 matching files and directories ERROR: Uploading artifacts as "archive" to coordinator... error error=couldn't execute POST against http://gitlab/api/v4/jobs/31/artifacts?artifact_format=zip&artifact_type=archive: Post http://gitlab/api/v4/jobs/31/artifacts?artifact_format=zip&artifact_type=archive: dial tcp: lookup gitlab on 10.43.0.10:53: server misbehaving id=31 token=UzZXJHik
based on the log mentioning port 53, it sure looks like the gitlab-runner-helper is trying to resolve host ‘gitlab’ and via DNS that will fail as I’m using a couple of VirtualBox VMs to run gitlab-ce (omnibus 11.7.5), another VM to run the kubernetes cluster (using k3s), and I’ve got static IPs in my VMs. I’ve used the hostAliases config property for the Helm installed gitlab-runner chart. (v 0.20…1). to add my gitlab-ce host’s IP for the runner Pod for egress to non-cluster ports. These versions all selected to what was installed in ‘production’ some time ago. I’m planning on upgrading our ‘prod’ gitlab-ce versions to the current release this weekend. And ironically, if I’m right on the cause, this won’t be an issue in production as we reply upon DNS 100%. But I’m a big advocate for testing first, to reduce the risk of breaking our production Gitlab installation. Hence my efforts on a local test env.
So my question boils down to: Does the gitlab-runner-helper container run within the same Pod as the gitlab-runner? If so, shouldn’t the gitlab-runner-helper container get the same hostAliases as the runner Pod uses (I exec into the running gitlab-runner container & observe that the hostAliases settings I provided in the Helm gitlab-runner chart values.yaml is indeed present in the container’s /etc/hosts. But I cannot determine (yet) what the gitlab-runner-helper image is using. Would the above error message & other details be logged outside the container’s log?
And is there a supported means for providing such hostAliases to the gitlab-runner-helper image if not in the same Pod? I’m assuming that may resolve the above issue with artifacts not uploading to the coordinator.
Perhaps I’ve just got to customize my host’s systemd-resolved setup, to leverage the existing K8S networking rather than try to customize the gitlab-runner-helper container.
MTIA for any feedback, insights, suggestions.