Using the gitlab helm chart, and gitlab-agent is installed in the same cluster via helm.
Running GitLab and Agent version v15.11.0
In the CI the following job works fine, no problems.
kubectltest:
stage: test
image:
name: bitnami/kubectl:latest
entrypoint: ['']
script:
- kubectl config get-contexts
- kubectl config use-context group/project:gitlab-agent-cluster1
- kubectl -n "$KUBE_NAMESPACE" -v9 get cronjobs
But if I try to apply a config, for eg.
kubectldeploy:
stage: deploy
image:
name: bitnami/kubectl:latest
entrypoint: ['']
script:
- kubectl config get-contexts
- kubectl config use-context group/project:gitlab-agent-cluster1
- kubectl -n "$KUBE_NAMESPACE" -v9 apply -f cron.yaml
Returns the following main error:
... error validating data: the server responded with the status code 426
but did not return more information; if you choose to ignore these errors,
turn validation off with --validate=false
The verbose log output is:
I0427 15:30:03.478358 49 round_trippers.go:466] curl -v -XGET -H "User-Agent: kubectl/v1.27.1 (linux/amd64) kubernetes/4c94112" -H "Authorization: Bearer <masked>" -H "Accept: application/json" 'https://kas.mydomain.com/openapi/v3/apis/batch/v1?hash=xxx&timeout=32s'
I0427 15:30:03.480391 49 round_trippers.go:553] GET https://kas.mydomain.com/openapi/v3/apis/batch/v1?hash=xxx&timeout=32s 426 Upgrade Required in 1 milliseconds
I0427 15:30:03.480423 49 round_trippers.go:570] HTTP Statistics: GetConnection 0 ms ServerProcessing 1 ms Duration 1 ms
I0427 15:30:03.480431 49 round_trippers.go:577] Response Headers:
I0427 15:30:03.480440 49 round_trippers.go:580] Referrer-Policy: strict-origin-when-cross-origin
I0427 15:30:03.480448 49 round_trippers.go:580] Date: Thu, 27 Apr 2023 15:30:03 GMT
I0427 15:30:03.480455 49 round_trippers.go:580] Content-Type: text/plain; charset=utf-8
I0427 15:30:03.480462 49 round_trippers.go:580] Content-Length: 76
I0427 15:30:03.480469 49 round_trippers.go:580] Upgrade: websocket
I0427 15:30:03.480476 49 round_trippers.go:580] X-Content-Type-Options: nosniff
I0427 15:30:03.480483 49 round_trippers.go:580] Strict-Transport-Security: max-age=63072000
I0427 15:30:03.480722 49 request.go:1188] Response Body: WebSocket protocol violation: Connection header "" does not contain Upgrade
error: error validating "cron.yaml": error validating data: the server responded with the status code 426 but did not return more information; if you choose to ignore these errors, turn validation off with --validate=false
I am using the NGINX ingress built-in to the GitLab helm chart.
The helm config is very basic, simply:
global:
ingress:
configureCertmanager: true
enabled: true
tls:
enabled: true
If I open a shell into the nginx ingress controller POD I can config that it contains the KAS server section with all the websocket bits in place.
And looking at the log of the gitlab-agent POD, I do see the connections come in and there are no errors.
So at this point nothing I’ve found in the forums or issue tracker seems to be applicable or helping solve this issue.