Kubernetes Dashboard (gitlab.com): "Gitlab Agent Server: Unauthorized"

Hello guys.

Problem to solve

I have the problem that the kubernetes dashboard on gitlab.com does not work for me and I don’t know why. Do you have an idea what could be responsible and how to fix it?
If there is any additional information you need: Please tell me and which and how to provide!

pasted file1266×174 12.4 KB

pasted file21295×359 22.8 KB

image1666×273 70.1 KB

I followed this guide to prepare and bootstrap my cluster.
Installation and Bootstrapping using these commands worked fine.

$ flux bootstrap gitlab --deploy-token-auth --owner=###### --repository=####### --branch=testing --path=clusters/testing
$ glab cluster agent bootstrap --manifest-path clusters/testing testing -b testing --environment-name testing

The log of the gitlab-agents deployed in the cluster

{"time":"2025-05-05T20:00:10.820403171Z","level":"INFO","msg":"Starting","mod_name":"agentk2kas_tunnel"}
{"time":"2025-05-05T20:00:10.820450139Z","level":"INFO","msg":"Starting","mod_name":"observability"}
{"time":"2025-05-05T20:00:10.820441052Z","level":"INFO","msg":"Starting","mod_name":"agent_registrar"}
{"time":"2025-05-05T20:00:10.820759027Z","level":"INFO","msg":"Observability endpoint is up","mod_name":"observability","net_network":"tcp","net_address":"[::]:8080"}
{"time":"2025-05-05T20:00:10.820457312Z","level":"INFO","msg":"Starting","mod_name":"gitops-manifest"}
{"time":"2025-05-05T20:00:10.820448245Z","level":"INFO","msg":"Starting","mod_name":"google_profiler"}
{"time":"2025-05-05T20:00:11.429672763Z","level":"INFO","msg":"attempting to acquire leader lease gitlab-agent/agent-#######-lock...","agent_id":#######}
{"time":"2025-05-05T20:00:11.435143869Z","level":"INFO","msg":"successfully acquired lease gitlab-agent/agent-#######-lock","agent_id":#######}
{"time":"2025-05-05T20:00:11.43534156Z","level":"INFO","msg":"Starting","mod_name":"starboard_vulnerability","agent_id":########}
{"time":"2025-05-05T20:00:11.435377868Z","level":"INFO","msg":"Starting","mod_name":"remote_development","agent_id":########}
{"time":"2025-05-05T20:00:11.435374341Z","level":"INFO","msg":"Event occurred","agent_id":########,"object":{"name":"agent-#######-lock","namespace":"gitlab-agent"},"fieldPath":"","kind":"Lease","apiVersion":"coordination.k8s.io/v1","type":"Normal","reason":"LeaderElection","message":"gitlab-agent-v2-##########-#### became leader"}
{"time":"2025-05-05T20:00:11.435394829Z","level":"INFO","msg":"Starting","mod_name":"flux","agent_id":#######}
{"time":"2025-05-05T20:00:11.43554452Z","level":"INFO","msg":"Starting GitRepository controller","mod_name":"flux","agent_id":#######}

{"time":"2025-05-05T20:00:10.939040756Z","level":"INFO","msg":"Starting","mod_name":"agent_registrar"}
{"time":"2025-05-05T20:00:10.939095896Z","level":"INFO","msg":"Starting","mod_name":"google_profiler"}
{"time":"2025-05-05T20:00:10.939155586Z","level":"INFO","msg":"Starting","mod_name":"gitops-manifest"}
{"time":"2025-05-05T20:00:10.939188946Z","level":"INFO","msg":"Starting","mod_name":"observability"}
{"time":"2025-05-05T20:00:10.939316317Z","level":"INFO","msg":"Starting","mod_name":"agentk2kas_tunnel"}
{"time":"2025-05-05T20:00:10.939509937Z","level":"INFO","msg":"Observability endpoint is up","mod_name":"observability","net_network":"tcp","net_address":"[::]:8080"}
{"time":"2025-05-05T20:00:11.494087547Z","level":"INFO","msg":"attempting to acquire leader lease gitlab-agent/agent-#######-lock...","agent_id":#######}

Steps to reproduce

I already followed the steps described here.
None of which yielded any result.

Configuration

Please tell me which configuration to insert here.

Versions

• Self-managed
• GitLab.com SaaS
• Dedicated
• Self-hosted Runners

Versions

• GitLab (Web: /help): GitLab Enterprise Edition 18.0.0-pre 48dd18c0f8c

Cloud-native

• Agent for Kubernetes (Operate > Kubernetes clusters): 17.11.1
• Kubernetes (kubectl version): 1.33.0
• Kustomize version: v5.6.0

$ kubectl get nodes                           
NAME          STATUS   ROLES                       AGE   VERSION
k3s-agent1    Ready    <none>                      98d   v1.31.4+k3s1
k3s-agent2    Ready    <none>                      98d   v1.31.4+k3s1
k3s-agent3    Ready    <none>                      98d   v1.31.4+k3s1
k3s-server1   Ready    control-plane,etcd,master   98d   v1.31.4+k3s1
k3s-server2   Ready    control-plane,etcd,master   98d   v1.31.4+k3s1
k3s-server3   Ready    control-plane,etcd,master   98d   v1.31.4+k3s1

Can you share the agent configuration file, specifically the user access entries matching the project paths? I suspect there might be a mismatch or typo causing the agent to fail.

Thank you for your quick reply.

My agent configuration looks like this:

user_access:
    access_as:
        agent: {}
    projects:
        - id: ######/homelab

It is the default file generated by glab in the project directory of which I am the owner.

Thanks, looks good. I found this bug report which says fixed but maybe not. Fix valid_authenticity_token? check for Rails 7.1 (#537688) · Issues · GitLab.org / GitLab · GitLab Can you review if it matches your experience, and comment there as well for engineers to see?

It seems to fit. Thank you for the reference.

However there are some differences of which I cannot verify the validity, since I do not inherently understand how all the features work.

1. I never connected the agent manually. I let the glab bootstrapper do its magic.
2. “Flux Sync” never said unavailable, it always says Updating.

But I will reference this thread in a comment there.

Thank you for your help.