Connection issue between KAS and kubernetes agent

Infrastructure as Code & Cloud Native
1

I’ve set up KAS in our GitLab omnibus install, I’ve disabled websocket to enable grpc communication. The KAS config looks like this:

gitlab_kas['enable'] = true
gitlab_kas['listen_address'] = "0.0.0.0:8150"
gitlab_kas['listen_websocket'] = false

On the agent side, the connection url is grpc://10.4.192.48:8150. In the logs of the agent, everything seems fine. The only info that appears is:

{"level":"info","time":"2022-01-19T13:19:44.991Z","msg":"Feature status change","feature_name":"tunnel","feature_status":true}
{"level":"info","time":"2022-01-19T13:19:44.993Z","msg":"Observability endpoint is up","mod_name":"observability","net_network":"tcp","net_address":"[::]:8080"}

However, in the KAS logs I get a lot of 301 responses:

2022-01-19_13:33:37.40240 {"level":"error","time":"2022-01-19T14:33:37.402+0100","msg":"AgentInfo()","correlation_id":"01FSS7WHAXQAJXCF4447XQQ3P3","grpc_service":"gitlab.agent.reverse_tunnel.rpc.ReverseTunnel","grpc_method":"Connect","error":"HTTP status code: 301"}
2022-01-19_13:33:42.22272 {"level":"error","time":"2022-01-19T14:33:42.222+0100","msg":"AgentInfo()","correlation_id":"01FSS7WHAXQAJXCF444BWR7CZ5","grpc_service":"gitlab.agent.reverse_tunnel.rpc.ReverseTunnel","grpc_method":"Connect","error":"HTTP status code: 301"}
2022-01-19_13:34:00.96150 {"level":"error","time":"2022-01-19T14:34:00.961+0100","msg":"AgentInfo()","correlation_id":"01FSS7WHAMXWT7DXQ8MS5Z8WD8","grpc_service":"gitlab.agent.agent_configuration.rpc.AgentConfiguration","grpc_method":"GetConfiguration","error":"HTTP status code: 301"}

I’ve also tried enabling websockets and using the ws://10.4.192.48:8181/-/kubernetes-agent/, but this is also giving issues. I’m using an external reverse proxy, so that’s why I’m using the ip/port combinations as I need to bypass the proxy. Can anyone point me in the right direction?

2

After searching for more than a day, I found the issue… I needed to place a / after the gprc url in the agent config. I also needed to add a line to the GitLab config, which now looks like this:

gitlab_kas['enable'] = true
gitlab_kas['listen_address'] = "0.0.0.0:8150"
gitlab_kas['listen_websocket'] = false
gitlab_kas['gitlab_address'] = "http://10.4.192.48:8181"

I’m going to leave this here in case anyone else wants to set up the kubernetes agent without going over the internet.

3

Hi JeroenV,
In my agent config, the the kas-address included a trailing slash but I also get a lot of 301 http status code.
This is my agent command:

helm upgrade --install gitlab-agent gitlab/gitlab-agent \
    --namespace gitlab-agent \
    --create-namespace \
    --set config.token=Nrqm5Nm8AkWzHJNAspMUD99TVxE9dMqtDcGU6xjf1gLxvs4Q \
    --set config.kasAddress=grpc://x.x.x.x:8150/

What’s wrong here?