Gitlab-CI Runner on Kubernetes | Unable to access Interactive Web Terminal | "Connection failure."

Recently I’ve deployed a fresh two-nodes Kubernetes cluster (using kubespray), and then I’ve configured a gitlab-ci runner to run on it using the “Set up a specific Runner automatically” option on Gitlab UI available on CI/CD Settings page.

The runner was successfully deployed inside the k8s cluster, and I’m able to run build Jobs using it and deploy environments.

But I’m unable to get shell access into a running environment using the “Interactive Web Terminal” feature. When I try to access a running environment - Operations -> Environments -> Terminal - I get the error message " Connection failure ".

By editing the ConfigMap used in runner’s configuration I’ve specified the [session_server] section as shown below:

bash-4.4$ cat /home/gitlab-runner/.gitlab-runner/config.toml
listen_address = "[::]:9252"
concurrent = 34
check_interval = 3
log_level = "info"

[session_server]
  listen_address = "0.0.0.0:8093"
  session_timeout = 2000
[[runners]]
  name = "runner-gitlab-runner-7b9ccb6999-959cq"
  url = "https://<anonymised>.com/"
  token = <anonymised>
  executor = "kubernetes"
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.kubernetes]
    host = ""
    bearer_token_overwrite_allowed = false
    image = "ubuntu:16.04"
    namespace = "gitlab-managed-apps"
    namespace_overwrite_allowed = ""
    privileged = true
    service_account_overwrite_allowed = ""
    pod_annotations_overwrite_allowed = ""
    [runners.kubernetes.volumes]

I can successfully Start and Stop the Environment through the Gitlab UI, and I can also use kubectl exec to get shell access into the deployment pod. But I’m unable to use Gitlab’s Interactive Web Terminal feature, which is something I desperately need.

Am I missing something in the configuration?

Please, help me solving this issue. Is there any step-by-step tutorial on how to properly configure gitlab runner on kubernetes?

2 Likes

I have the exact same issue. Please advise.

Hi Laura,

Meanwhile I found the solution.
I will post the solution here later today, after work.

Regards,
Carlos

Hello, wait on your help, please

Sorry for the delay. I’ve been too busy at work.

Web Terminal feature only works if the __CI_ENVIRONMENT_SLUG__ label is passed as shown below [when deploying into k8s].

I’m also replacing that variable inside the k8s deployment manifest, but probably this is redundant since I’m also passing the label ‘app=’. This is something I need to confirm. In any case, I’ve successfully launched an Interactive Web Terminal using the code below.

Please, feel free to reach me in case of difficulties. I might take some time to get back here, but eventually I will. I may take some time to respond, but I will asap.

From .gitlab-ci.yaml:

  stage: deploy
  image: lwolf/kubectl_deployer:latest
  services:
    - docker:dind
  environment:
    name: development/$CI_COMMIT_REF_NAME
    url: https://$CI_ENVIRONMENT_SLUG.gitlab.com
    on_stop: stop_deploy
  when: manual
  script:
    - echo "${KUBE_CA_PEM}" > kube_ca.pem
    - kubectl config set-cluster $K8S_CLUSTER_NAME --server=”$KUBE_URL” --certificate-authority="$(pwd)/kube_ca.pem"
    - kubectl config set-credentials $K8S_CLUSTER_NAME --token=”$KUBE_TOKEN”
    - sed -i "s/__CI_ENVIRONMENT_SLUG__/${CI_ENVIRONMENT_SLUG}/" deployment.yaml
    - kubectl apply -f deployment.yaml -l app=${CI_ENVIRONMENT_SLUG}


stop_deploy:
  stage: deploy
  image: lwolf/kubectl_deployer:latest
  environment:
    name: development/$CI_COMMIT_REF_NAME
    url: https://$CI_ENVIRONMENT_SLUG.gitlab.com
    action: stop
  when: manual
  script:
    - kubectl delete all -l app=${CI_ENVIRONMENT_SLUG}

From my demo deployment k8s manifest:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: __CI_ENVIRONMENT_SLUG__
  labels:
    app: __CI_ENVIRONMENT_SLUG__
spec:
  replicas: 1
  template:
    metadata:
      name: demo
      labels:
        app: __CI_ENVIRONMENT_SLUG__
1 Like