External kubernetes cluster from gitlab-runner results in error

I have a dedicated gitlab-runner running on a VM in our datacenter registered to our own self managed gitlab-ce server. I am trying to get this gitlab-runner to connect to an external production k8s cluster so we can deploy our applications. However i get

error connecting to Kubernetes: invalid configuration: no configuration has been provided

and my config.toml is
concurrent = 1
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "gitlab-runners-local"
  url = "https://gitlab.mydomain.com/"
  token = "token-"
  executor = "kubernetes"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.kubernetes]
    host = ""
    bearer_token_overwrite_allowed = false
    image = ""
    namespace = "build"
    namespace_overwrite_allowed = ""
    privileged = false
    service_account_overwrite_allowed = ""
    pod_annotations_overwrite_allowed = ""
    [runners.kubernetes.pod_security_context]
    [runners.kubernetes.volumes]

the docs at Kubernetes executor | GitLab seem to imply i need to set the host and ca parameters but there are no examples i can find of what to set them to and they dont actually make sense to me. I have on the gitlab-runner server a completely working .aws/config and ./kube/config where i can do kubectl get deployments as the user gitlab is running as.

root@gitlab-runners-ash:/etc/gitlab-runner# kubectl get pods --all-namespaces
NAMESPACE             NAME                                                     READY   STATUS      RESTARTS   AGE
gitlab-managed-apps   certmanager-cainjector-598444f6-p6t7v                    1/1     Running     0          99m
gitlab-managed-apps   certmanager-cert-manager-5444bb696b-xpxdz                1/1     Running     0          99m
gitlab-managed-apps   certmanager-cert-manager-webhook-79dbfb9cd9-vwlz7        1/1     Running     2          99m
gitlab-managed-apps   ingress-nginx-ingress-controller-675c869cd8-6hl64        2/2     Running     1          99m
gitlab-managed-apps   ingress-nginx-ingress-default-backend-7d66f58f5f-dxh4s   1/1     Running     0          99m
gitlab-managed-apps   prometheus-kube-state-metrics-89f5f974-zqtfv             1/1     Running     0          97m
gitlab-managed-apps   prometheus-prometheus-server-865444f6c9-s9k9w            2/2     Running     0          97m
gitlab-managed-apps   tiller-deploy-5bb888969c-rzztc                           1/1     Running     0          100m
kube-system           aws-node-fg44m                                           1/1     Running     0          24h
kube-system           aws-node-pl8wl                                           1/1     Running     0          24h
kube-system           coredns-55c5fcd78f-6kt7h                                 1/1     Running     0          24h
kube-system           coredns-55c5fcd78f-bn2hv                                 1/1     Running     0          24h

so i guess i kind of assumed when it set “autodiscover” it would use the local kubernetes files, but it doesnt. As mentioned above, i dont know what to actually set the values to config.toml and there are no real world examples. I guess the biggest issue i have is there is no such thing as

  • cert_file : Optional Kubernetes apiserver user auth certificate
  • key_file : Optional Kubernetes apiserver user auth private key
  • ca_file : Optional Kubernetes apiserver ca certificate

when dealing with kubernetes. you have cluster-ca, cluster-token and service-account. nor am i sure how you specify the assume role arn needed for security.

Any help on what those values map to inside of aws/kubernetes or how to configure gitlab-runner to use the OS environment?

Hi,

Have the same problem but with GKE.
Did you discover how find this files from Kubernetes (cert_file, key_file and ca_file)?

Thank you.

i actually gave up trying to build my own runner, and just deployed the gitlab runner into the kubernetes cluster by going to kubernetes->applications->gitlab runner
and that installed everything fine.

rob

Thanks for your feedback even if I expected something else! :stuck_out_tongue: