Gitlab Omnibus CE + Prometheus + Grafana Kubernetes Integration

How to Gitlab Omnibus CE + Prometheus + Grafana Kubernetes Integration!?

I’d like to make Kubernetes cluster monitoring with GitLab CE integrated Prometheus and Grafana.

Prometheus and Grafana work well. Kubernetes are integrated with GitLab. But I don’t understand how is possible enable and configure their cooperation. Helm work. I read in GitLab FAQ

I deployed Prometheus into the gitlab-managed-apps but I see error.

I read these lines in the documentation:

To configure a resource to be monitored by Prometheus, simply set the following Kubernetes annotations:

  • to true to enable monitoring of the resource.
  • to define the port of the metrics endpoint.
  • to define the path of the metrics endpoint. Defaults to /metrics .

Where is it need this change? Maybe do you know the detailed instructions? Step by step.

prometheus error pod has unbound immediate PersistentVolumeClaims

  "kind": "PersistentVolumeClaim",
  "apiVersion": "v1",
  "metadata": {
    "name": "prometheus-prometheus-server",
    "namespace": "gitlab-managed-apps",
    "selfLink": "/api/v1/namespaces/gitlab-managed-apps/persistentvolumeclaims/prometheus-prometheus-server",
    "uid": "117cf105-df9a-11e9-9bd6-00155d011b19",
    "resourceVersion": "45005673",
    "creationTimestamp": "2019-09-25T13:40:42Z",
    "labels": {
      "app": "prometheus",
      "chart": "prometheus-6.7.3",
      "component": "server",
      "heritage": "Tiller",
      "release": "prometheus"
    "finalizers": [
  "spec": {
    "accessModes": [
    "resources": {
      "requests": {
        "storage": "8Gi"
    "volumeMode": "Filesystem",
    "dataSource": null
  "status": {
    "phase": "Pending"

Ok I resoved Prometheus:

kubectl create clusterrolebinding --user system:serviceaccount:gitlab-managed-apps:prometheus-kube-state-metrics default-gitlab-prometheus-sa-admin --clusterrole cluster-admin


executed yaml

apiVersion: v1
kind: PersistentVolume
  name: hostpatheightgigs
    storage: 8Gi
  volumeMode: Filesystem
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
      path: "/k8s/data1"

All Kubernetes Prometheus PODs Statuss become OK.
I see metrics in indoor prometheus metric POD
and I see Prometheus server UI

But I don’t see any new metrics from Kubernetes cluster in GitLab Prometheus?!
How does it have to work and see the result in GitLab? I didn’t find any video where I can see it.