Repeated Panels and Rows

A service overview dashboard renders a single panel: “Request rate per service”. The on-call engineer wants to see the same panel once per cluster, side by side, with the cluster name in the title. The author has two options: copy the panel twelve times and edit each one, or use repeat panel by variable to multiply the panel across the cluster dropdown.

The second option is the answer. A single panel in the dashboard JSON becomes one rendered panel per value of the selected variable. The dashboard does not grow in JSON; it grows in pixels. The cost grows too: a panel repeated twelve times is twelve panel queries per refresh. A hundred-value variable is a hundred-panel dashboard that the browser is trying to render.

This lesson is about the shape, the cost, and the failure modes of repeated panels and rows.

What it is

Panel repetition is the Grafana 11 feature that multiplies a single panel in the dashboard JSON into one panel per value of a variable at render time. Row repetition is the same feature at the row level: a row with three panels becomes one row per cluster, with the three panels inside each row.

The dashboard JSON declares the panel or row once. The repeat field on the panel (or row) references a variable name. At render time, the panel is duplicated once per current value of the variable; the duplicated panels are laid out vertically, each with a unique title that includes the variable value.

  +-------------------+
  |  Panel template   |
  |  (in JSON, one)   |
  +-------------------+
            |
            |  repeat: $cluster
            v
  +-------------------+
  |  Request rate     |
  |  (prod)           |
  +-------------------+
  +-------------------+
  |  Request rate     |
  |  (staging)        |
  +-------------------+
  +-------------------+
  |  Request rate     |
  |  (dev)            |
  +-------------------+

Why a sysadmin cares

Three operational pains map directly to the repeat pattern:

1. One JSON, N panels. A service overview that repeats by cluster is one panel in the dashboard JSON, twelve panels in the rendered dashboard. The JSON stays small; the rendered output grows with the variable.
2. Per-value investigation. A panel repeated by $instance shows one panel per instance; the operator can scan twelve panels at once instead of clicking through a dropdown.
3. Drill-down by row. A row repeated by $cluster produces one row per cluster, each with the same set of panels (CPU, memory, request rate, error rate). The dashboard reads as “all the things, for all the clusters” without 200 hand-written panels.

The wrong shape shows up as a panel that repeats against a variable with 5,000 values; the dashboard tries to render 5,000 panels and the browser tab dies.

The row-with-variable pattern

A row is a dashboard-level grouping mechanism. A row holds panels and is itself a panel in the layout grid. A row can be repeated by a variable:

  +-------------------+   <-- row template (in JSON)
  |  CPU | MEM | REQ |   <-- panels inside
  +-------------------+
            |
            |  repeat: $cluster
            v
  +-------------------+
  |  Row: prod        |
  |  CPU | MEM | REQ |
  +-------------------+
  +-------------------+
  |  Row: staging     |
  |  CPU | MEM | REQ |
  +-------------------+
  +-------------------+
  |  Row: dev         |
  |  CPU | MEM | REQ |
  +-------------------+

Each row is a copy of the template; each panel inside is repeated within the row. The pattern is the natural template for “one cluster’s worth of panels, repeated per cluster”.

The cost is N_clusters x M_panels rendered panels. A 12-cluster deployment with 5 panels per row is 60 panels.

How it works

The repeat pipeline:

  panel JSON (one entry)
        |
        v
  +-------------------+   +-------------------+
  |  read repeat      |-->|  variable name    |
  |  field            |   |  in panel JSON    |
  +-------------------+   +-------------------+
        |
        v
  +-------------------+
  |  resolve the      |
  |  variable's       |
  |  current value    |
  |  list             |
  +-------------------+
        |
        v
  +-------------------+
  |  for each value,  |
  |  duplicate the    |
  |  panel; inject    |
  |  the value into   |
  |  $var and into    |
  |  the title        |
  +-------------------+
        |
        v
  +-------------------+
  |  lay out the      |
  |  duplicates       |
  |  vertically       |
  |  in the grid      |
  +-------------------+
        |
        v
  +-------------------+
  |  run panel        |
  |  queries for      |
  |  every duplicate  |
  +-------------------+

Three details that shape the cost:

• Duplication is at render time. The dashboard JSON has one panel; the rendered dashboard has N. Edit one panel; all N reflect the edit.
• Each duplicate runs its own panel query. A panel repeated 12 times is 12 panel queries. The panel JSON has one targets[]; the rendered dashboard has 12.
• Layout is vertical by default. Grafana stacks the duplicates vertically. A gridPos.y on the template determines the starting Y; each duplicate increments Y by the panel’s gridPos.h.

How to configure it

The canonical panel-level repeat:

{
  "type":       "timeseries",
  "title":      "Request rate [$cluster]",
  "datasource": { "type": "prometheus", "uid": "prom-prod" },
  "gridPos":    { "x": 0, "y": 0, "w": 24, "h": 8 },
  "targets": [
    {
      "refId":      "A",
      "datasource": { "type": "prometheus", "uid": "prom-prod" },
      "expr":       "sum by (service) (rate(http_requests_total{cluster=~\"$cluster\"}[5m]))",
      "legendFormat": "{{service}}"
    }
  ],
  "repeat":      "cluster",
  "repeatDirection": "h"
}

Two fields control the repetition:

• repeat — the variable name. The panel is duplicated once per value of this variable.
• repeatDirection — h for horizontal layout (panels side by side), v for vertical layout (default). A horizontal repeat of 12 panels does not fit on a screen; vertical is the common choice.

The title reads Request rate [prod] for the prod duplicate, Request rate [staging] for the staging duplicate. The $cluster in the title is interpolated just like a non-repeated panel title.

For row-level repetition, the pattern is identical on the row object:

{
  "type":   "row",
  "title":  "Cluster overview",
  "repeat": "cluster",
  "collapsed": false,
  "gridPos": { "x": 0, "y": 0, "w": 24, "h": 1 },
  "panels": [
    {
      "type":  "timeseries",
      "title": "CPU",
      "expr":  "sum by (instance) (rate(node_cpu_seconds_total{cluster=~\"$cluster\"}[5m]))"
    },
    {
      "type":  "timeseries",
      "title": "Memory",
      "expr":  "sum by (instance) (node_memory_MemAvailable_bytes{cluster=~\"$cluster\"} / node_memory_MemTotal_bytes{cluster=~\"$cluster\"})"
    }
  ]
}

The row is cloned per cluster; each clone contains the panels inside. The panels inside the row can use $cluster because the row injects it via scopedVars.

How to validate it

Three checks confirm the repeat pattern works.

Severity: READ-ONLY.

# 1. The JSON declares the repeat field on the panel or
#    row. jq confirms it.
curl -s -u admin:$ADMIN \
  https://grafana.example.com/api/dashboards/uid/svc-overview \
  | jq '.dashboard.panels[]
        | select(.repeat != null)
        | {title, repeat, repeatDirection}'

{
  "title":          "Request rate [$cluster]",
  "repeat":         "cluster",
  "repeatDirection": "h"
}

# 2. The variable has the expected value count. A repeat
#    against a variable with 100 values is 100 panels.
curl -G -s http://prometheus:9090/api/v1/query \
  --data-urlencode 'query=label_values(up{job="kube-state"}, cluster)' \
  --data-urlencode 'time='$(date +%s) \
  | jq '.data.result | map(.metric.cluster) | length'
# 12

# 3. The rendered dashboard shows the right number of
#    duplicates. Open the dashboard, count panels.
xdg-open "https://grafana.example.com/d/svc-overview?var-cluster=$__all"
# 12 timeseries panels titled "Request rate [prod]" ...
#     ... "Request rate [staging]" ... "Request rate [dev]"

The third check is the ground truth. The JSON declares one panel; the rendered dashboard declares 12. If the rendered count does not match the variable’s value count, the repeat is broken.

How it can fail

Six failure shapes appear repeatedly with panel and row repetition:

1. Repeat against a wide variable. A panel repeated by $instance where instance has 200 values is 200 panels. The browser tab freezes at 50-100 panels; the dashboard becomes unusable. Symptom: the dashboard loads but the browser hangs.
2. Repeat against an unbounded variable. A repeat against a variable with allValue: ".*" produces a panel per label value the data source has, plus one All panel. The total is unbounded. Symptom: the dashboard tries to render thousands of panels.
3. Row repetition that grows with the chain. A row repeated by $cluster and containing a panel repeated by $namespace is N_clusters x N_namespaces clones of the inner panel. Symptom: the dashboard tries to render 12 x 80 = 960 panels.
4. repeatDirection: "h" with too many duplicates. A horizontal repeat of 12 panels at gridPos.w: 24 does not fit on a 24-column grid. The panels wrap and the layout becomes a 12-column stack the dashboard did not expect. Symptom: the layout is broken at the right edge.
5. No template in the title. A panel repeated by $cluster with title Request rate shows the same title 12 times; the operator cannot tell which panel is which cluster. Symptom: 12 identical-looking panels.
6. Repeat variable removed from the dashboard. A panel with repeat: "cluster" but no $cluster variable in the dashboard renders one panel only, with no error. Symptom: the dashboard looks like the repeat is broken.

How to troubleshoot it

The diagnostic order:

1. Count the rendered panels. Compare the panel count in the rendered dashboard against the variable’s value count. A repeat that produces too few panels has a missing or empty variable; a repeat that produces too many has an unbounded variable.
2. Inspect the variable. Settings > Variables. The repeat field must reference a variable that exists in the dashboard’s templating.list.
3. Bound the variable. Set regex and allValue: ".*" on the variable so the value count is predictable. A repeat against a 12-value variable is 12 panels; against a 5,000-value variable is 5,000 panels.
4. Switch to a table panel for top-N views. A panel that wants “the top 10 instances by error rate” is a table panel with topk(10, ...) in the query, not a timeseries panel repeated 200 times with topk(1, ...).
5. Use row repetition with a small panel set. A row of 5 panels repeated by 12 clusters is 60 panels; a row of 10 panels repeated by 12 clusters is 120 panels. Keep the inner panel count small.

Security implications

• Repeat does not change permissions. A panel repeated 12 times is 12 panel queries with the same data-source permissions as the original. No additional access is granted.
• The repeated panels share the same data source UID. A repeat against a datasource variable that lists every Prometheus data source in the org produces a panel per Prometheus instance. The viewer must have access to each Prometheus instance to see each panel.
• The variable-cookie state still applies. A repeat variable’s value list is persisted in the cookie the same way any other variable is.

Performance implications

• The cost is N_panels_rendered = N_variable_values x N_panel_templates. A row of 5 panels repeated by 12 clusters is 60 panels, which is 60 panel queries per refresh.
• Layout cost. Grafana 11 lays out panels with a gridster-based engine. A 200-panel dashboard takes noticeable browser time to lay out; the SVG/canvas rendering adds more.
• Variable-query cost still applies. The repeat variable’s value list is resolved before the panel duplicates render. A repeat variable with a slow label_values is a slow repeat.

Production guidance

• Use row repetition for cluster-level layouts. One row per cluster with 5-8 panels is a reasonable shape.
• Use panel repetition sparingly. A single panel repeated by 12 values is fine; a row repeated by 200 values is not.
• Bound the repeat variable. Set regex and allValue: ".*" on the variable so the value count is predictable.
• Include the variable value in the panel title. A repeated panel with title Request rate is 12 indistinguishable panels; one with Request rate [$cluster] is 12 readable panels.
• Switch to a table panel for top-N views. A topk(10, ...) table is more readable than 200 panels, even if technically possible.
• Measure the dashboard size. A 100-panel dashboard takes longer to render than a 10-panel dashboard. Treat the rendered panel count as a budget item.

Verification

You should now be able to answer:

• What is the difference between panel repetition and row repetition?
• How does the repeat field multiply a panel in the rendered dashboard?
• What is the cost of a row with 5 panels repeated by 12 clusters?
• Why is a repeat against an unbounded variable dangerous?
• When should you use a table panel instead of a repeated timeseries panel?

Quiz