KSM deployment — installing the metrics collector

The KSM deployment is the cluster’s metrics collector. The deployment is via manifest or Helm. The RBAC is the cluster’s access control. The HA is the production pattern. This lesson walks the deployment, the RBAC, the HA, and the verification.

The Helm chart

The Helm chart is the canonical deployment:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install kube-state-metrics prometheus-community/kube-state-metrics \
  --namespace monitoring \
  --create-namespace

The Helm chart installs:

• The Deployment.
• The ServiceAccount.
• The ClusterRole and ClusterRoleBinding.
• The Service.

The chart is the canonical deployment.

The manifest

The manifest is the alternative:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-state-metrics
  namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-state-metrics
rules:
- apiGroups: [""]
  resources:
  - configmaps
  - secrets
  - nodes
  - pods
  - services
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs: ["list", "watch"]
- apiGroups: ["apps"]
  resources:
  - statefulsets
  - daemonsets
  - deployments
  - replicasets
  verbs: ["list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: monitoring
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kube-state-metrics
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kube-state-metrics
  template:
    metadata:
      labels:
        app: kube-state-metrics
    spec:
      serviceAccountName: kube-state-metrics
      containers:
      - name: kube-state-metrics
        image: registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.10.x
        ports:
        - name: http
          containerPort: 8080
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
  name: kube-state-metrics
  namespace: monitoring
spec:
  selector:
    app: kube-state-metrics
  ports:
  - port: 8080
    targetPort: http
    name: http

The manifest is comprehensive.

The RBAC

The RBAC is the cluster’s access control:

apiGroups: [""]
resources:
  - configmaps
  - secrets
  - nodes
  - pods
  - services
  ...
verbs: ["list", "watch"]

The RBAC grants KSM the list/watch permissions on the cluster objects. The KSM does not need write permissions.

The Service

The Service exposes the KSM:

apiVersion: v1
kind: Service
metadata:
  name: kube-state-metrics
spec:
  selector:
    app: kube-state-metrics
  ports:
  - port: 8080
    targetPort: http

The Service is the endpoint for Prometheus.

The HA pattern

The HA pattern is a single replica with leadership:

flowchart LR
    A[KSM 1] --> B[API server]
    C[KSM 2] --> B
    B --> D[etcd]

The KSM can run multiple replicas; the leader election ensures only one is active. The HA is via the leader election.

spec:
  replicas: 2
  template:
    spec:
      containers:
      - name: kube-state-metrics
        args:
        - --leader-elect=true

The leader-election is enabled.

The single replica pattern

The single replica is the common production pattern:

spec:
  replicas: 1

The single replica is sufficient for most clusters. KSM is a passive listener; it does not lose data on restart.

The Prometheus scrape

The Prometheus scrape:

scrape_configs:
  - job_name: 'kube-state-metrics'
    static_configs:
      - targets: ['kube-state-metrics.monitoring:8080']

The Prometheus scrapes the KSM endpoint.

The verification

The verification:

# Verify the deployment
kubectl get pods -n monitoring -l app=kube-state-metrics

# Verify the metrics endpoint
kubectl port-forward -n monitoring kube-state-metrics-xxx 8080:8080
curl http://localhost:8080/metrics

# Verify the Prometheus scraping
promtool query instant http://prometheus:9090 kube_pod_info

The verification confirms the deployment.

The PrometheusOperator integration

The PrometheusOperator integration:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: kube-state-metrics
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: kube-state-metrics
  endpoints:
  - port: http

The PrometheusOperator auto-discovers the KSM.

Cross-course references

• The Prometheus course (Part LXXXVIII) covers the scraping.
• The RBAC course (Part III) covers the access control.
• The Helm course covers the chart installation.

Quiz

Production discipline

• Deploy KSM via Helm. The canonical way.
• Configure the RBAC. Read-only access.
• Use the single replica. Sufficient for most clusters.
• Enable leader election. For HA.
• Configure Prometheus scraping. The metrics flow.
• Document the deployment. The Helm values, the RBAC.

The KSM deployment is the cluster’s metrics collector. Operating it well is via the Helm chart, with the right RBAC, and the Prometheus scraping.