Variable Types

The on-call engineer paged the wrong team. The dashboard they were looking at was for production, not staging, but the operator had no idea which environment the URL pointed at. The variable at the top of the dashboard said prod but the value was a free-text field. The operator typed production, missed, and got the wrong data.

This is what the variable-type taxonomy exists to prevent. Grafana 11 has seven kinds of variable, and each one constrains the values the viewer can pick. A query variable returns what the data source returns; a custom variable returns what the author typed; a constant variable is a single fixed value; a datasource variable is the name of a data source; an interval variable is a time step; a textbox variable is whatever the viewer types; and a system variable is set by Grafana itself.

What it is

A variable kind is the discriminator that determines how a variable’s value list is computed. It is set at variable creation and visible in Settings > Variables > Type. The seven kinds:

  +----------------+-------------------------------------------------+
  | Kind           |  Value list source                               |
  +----------------+-------------------------------------------------+
  |  query         |  Run an expression against a data source;       |
  |                |  return the resulting label values.              |
  +----------------+-------------------------------------------------+
  |  custom        |  A static, comma-separated list defined by       |
  |                |  the author. No data source query.               |
  +----------------+-------------------------------------------------+
  |  constant      |  A single value defined by the author. Hidden    |
  |                |  by convention; the value is the variable.       |
  +----------------+-------------------------------------------------+
  |  datasource    |  List of every data source of a given type       |
  |                |  configured in the Grafana instance.             |
  +----------------+-------------------------------------------------+
  |  interval      |  A list of time steps (1m, 5m, 15m, 1h, 6h,     |
  |                |  12h, 1d, 7d). Used as $__interval step.        |
  +----------------+-------------------------------------------------+
  |  textbox       |  Free-form text typed by the viewer. No          |
  |                |  validation.                                     |
  +----------------+-------------------------------------------------+
  |  system        |  Grafana-supplied values: $__from, $__to,        |
  |                |  $__interval, $__range, $__org, $__user.        |
  +----------------+-------------------------------------------------+

The kind is not a style choice; it constrains the operator’s behaviour at view time. A constant variable cannot be changed from the dropdown; a textbox variable accepts anything; a query variable refreshes from the data source.

Why a sysadmin cares

Each kind encodes a different operational guarantee. Picking the right one is the difference between a dashboard that self-corrects and one that lets an operator type a typo into production.

1. Drift-free filtering. A query variable on label_values(up, cluster) always reflects what the data source knows. The dashboard cannot drift from reality while the data source is healthy.
2. Stable, named values. A custom variable for the list of business services guarantees that the operator picks from a known list, not whatever they remember the name to be.
3. Immutable per-dashboard constants. A constant variable pins a value (the dashboard name, the region, the team). The viewer cannot override it; URL pinning has no effect.
4. Multi-cluster fan-out. A datasource variable lets a panel switch between ten Prometheus instances without editing panel JSON.
5. Step-appropriate rate windows. An interval variable on $__interval gives the right denominator for a rate() query; an interval on $__rate_interval gives the right step with sub-queries and rules.

The wrong choice shows up as a typoed query that returns a blank panel, or as a fixed constant that drifts as the world changes.

The fields-hub

Every kind shares a common fields block visible in the variable editor:

  +----------------+-------------------------------------------------+
  | Field          |  Purpose                                         |
  +----------------+-------------------------------------------------+
  |  name          |  The identifier used in $name interpolation.     |
  |                |  Must be unique per dashboard.                   |
  +----------------+-------------------------------------------------+
  |  label         |  The dropdown label the viewer sees.             |
  +----------------+-------------------------------------------------+
  |  description   |  Tooltip and runbook excerpt.                    |
  +----------------+-------------------------------------------------+
  |  hide          |  Hide the variable from the dropdown; still      |
  |                |  available for $name interpolation.              |
  +----------------+-------------------------------------------------+
  |  skipUrlSync   |  Do not write this variable into the URL.        |
  |                |  Useful for tab-specific settings.               |
  +----------------+-------------------------------------------------+
  |  includeAll    |  Show an "All" pseudo-value alongside the data.  |
  +----------------+-------------------------------------------------+
  |  allValue      |  The string the "All" pseudo-value expands to    |
  |                |  when interpolated. Typically ".*" for regex.   |
  +----------------+-------------------------------------------------+
  |  multi         |  Allow selecting more than one value.            |
  +----------------+-------------------------------------------------+
  |  refresh       |  0 (never), 1 (on dashboard load), 2 (on time-   |
  |                |  range change).                                  |
  +----------------+-------------------------------------------------+
  |  current       |  Persisted selection; survives reloads.          |
  +----------------+-------------------------------------------------+

The fields-hub is what turns a variable into a working piece of UI. Skipping it produces a variable that works for the author and surprises everyone else.

Regen on time change

The refresh field has three values:

• 0 — the variable’s value list is computed once at dashboard load and never refreshed. A custom variable never refreshes; a constant variable has no value list to refresh; a query variable with refresh: 0 is a cached value.
• 1 — refresh on every dashboard load. The default for query variables. Cost is amortised across panels.
• 2 — refresh on every time-range change. This is the rare setting that produces a query variable whose value list shrinks when the time range narrows. Useful for label_values(up[1h], cluster) to keep the dropdown to recently-active clusters.

The system kind is special: its values are not refreshed at all. $__interval is computed from the time range and panel width; $__from and $__to are the time-range picker.

The All-value and the multi-value

Two pseudo-values appear in every kind that has a list:

• All. The first row of the dropdown when includeAll: true. Selecting All writes value="$__all" to the URL. When interpolated, it expands to allValue (default empty string, recommended .* for query variables that use =~).
• Multi. When multi: true, the viewer can pick more than one value. The URL value is value1,value2,.... When interpolated into a regex, the values become value1|value2|....

A correct multi query variable has a panel-side cluster=~"$cluster" to match the comma-list. A multi without =~ (using =) is the most common multi-value bug; only one value matches.

How it works

The seven kinds resolve values through different paths:

  +----------------+
  | query          |  data source call --> label list --> sort/dedup
  +----------------+
  +----------------+
  | custom         |  author-defined list (read from JSON)
  +----------------+
  +----------------+
  | constant       |  author-defined single value (read from JSON)
  +----------------+
  +----------------+
  | datasource     |  grafana-server in-memory list of
  |                |  /api/datasources filtered by type
  +----------------+
  +----------------+
  | interval       |  grafana-server hard-coded list
  +----------------+
  +----------------+
  | textbox        |  free-form viewer input (no validation)
  +----------------+
  +----------------+
  | system         |  computed per render from request context
  +----------------+

The query kind is the only one that makes an external call. The cost is bounded by the refresh setting and by the chain dependencies.

How to configure it

Below is a portable, idiomatic templating.list showing the four kinds you will use most often. The fifth and sixth (interval, textbox) get their own snippets.

{
  "templating": {
    "list": [
      {
        "name":       "cluster",
        "label":      "Cluster",
        "type":       "query",
        "datasource": { "type": "prometheus", "uid": "prom-prod" },
        "query":      "label_values(up{job=\"kube-state\"}, cluster)",
        "refresh":    1,
        "includeAll": true,
        "allValue":   ".*",
        "multi":      true,
        "sort":       1,
        "current":    { "selected": true, "text": "All", "value": "$__all" }
      },
      {
        "name":     "env",
        "label":    "Environment",
        "type":     "custom",
        "query":    "prod,staging,dev",
        "current":  { "selected": true, "text": "prod", "value": "prod" },
        "includeAll": false,
        "multi":    false
      },
      {
        "name":     "team",
        "label":    "Team",
        "type":     "custom",
        "query":    "platform,payments,frontend",
        "current":  { "selected": false, "text": "platform", "value": "platform" }
      },
      {
        "name":     "region",
        "label":    "Region",
        "type":     "constant",
        "query":    "eu-west-1",
        "current":  { "selected": true, "text": "eu-west-1", "value": "eu-west-1" },
        "hide":     0
      },
      {
        "name":     "ds",
        "label":    "Datasource",
        "type":     "datasource",
        "query":    "prometheus",
        "current":  { "selected": true, "text": "prom-prod", "value": "prom-prod" }
      }
    ]
  }
}

For a time-step knob and a free-form filter:

{
  "name":     "step",
  "label":    "Step",
  "type":     "interval",
  "query":    "1m,5m,15m,1h,6h,12h,1d,7d",
  "current":  { "selected": true, "text": "15m", "value": "15m" }
},
{
  "name":     "search",
  "label":    "Search",
  "type":     "textbox",
  "current":  { "selected": true, "text": "", "value": "" },
  "options":  { "current": { "selected": true, "text": "", "value": "" } }
}

Three details worth highlighting:

• query of the datasource variable is the type of data source, not the UID. The dropdown lists every data source of that type configured in Grafana.
• constant has no UI affordance for changing the value; the value is the query field. A constant is for immutable per-dashboard facts (region, cluster, team).
• textbox is the only kind without a value list. The viewer is typing into a string; validation is the panel query’s job.

How to validate it

Three checks: one on the JSON, one on the wire, one in the URL.

Severity: READ-ONLY.

# 1. The JSON declares the expected kind and the expected
#    query field for that kind.
curl -s -u admin:$ADMIN \
  https://grafana.example.com/api/dashboards/uid/svc-overview \
  | jq '.dashboard.templating.list[]
        | {name, type, query, refresh, includeAll, allValue}'

{
  "name":        "cluster",
  "type":        "query",
  "query":       "label_values(up{job=\"kube-state\"}, cluster)",
  "refresh":     1,
  "includeAll":  true,
  "allValue":    ".*"
}
{
  "name":        "env",
  "type":        "custom",
  "query":       "prod,staging,dev",
  "refresh":     0,
  "includeAll":  false,
  "allValue":    null
}
{
  "name":        "region",
  "type":        "constant",
  "query":       "eu-west-1",
  "refresh":     0,
  "includeAll":  false,
  "allValue":    null
}
{
  "name":        "ds",
  "type":        "datasource",
  "query":       "prometheus",
  "refresh":     0,
  "includeAll":  false,
  "allValue":    null
}

# 2. The query variable returns a value set that matches
#    what the dashboard expects. A label that no longer
#    exists in Prometheus returns an empty dropdown.
curl -G -s http://prometheus:9090/api/v1/query \
  --data-urlencode 'query=label_values(up{job="kube-state"}, cluster)' \
  --data-urlencode 'time='$(date +%s) \
  | jq '.data.result | map(.metric.cluster) | length'
# 4

# 3. The dropdown lists each value. Open the dashboard,
#    click the cluster dropdown, and confirm four cluster
#    names are present. The All-value is the first row.
xdg-open "https://grafana.example.com/d/svc-overview"

How it can fail

Six failure shapes appear repeatedly with the variable-type taxonomy:

1. textbox used as a filter. The viewer types prod,staging and the panel query does env="$env", matching the literal string. The panel shows nothing. Symptom: the panel is empty whenever the viewer picks more than one value.
2. constant used as a runtime filter. The author intends the value to be selectable but constant hides the dropdown. Symptom: the dropdown is absent; the value cannot change; the URL has no var-region.
3. query variable with a stale expression. The label is renamed in Prometheus but the variable query still references the old label. Symptom: the dropdown is empty; the panel cannot render; the dashboard shows a “Templating init failed” badge.
4. datasource variable pinned to a deleted data source. A data source is removed via provisioning but a panel still references $ds. Symptom: every panel shows “Data source not found”.
5. interval without an explicit query. Grafana 11 uses a default step list when query is empty. The default is fine for normal cases; for a panel that wants 5s,30s,1m,5m,15m,1h, the author must set it explicitly. Symptom: $__interval is too coarse for rate calculations on sub-minute data.
6. multi without a regex matcher. A multi: true variable with cluster="$cluster" matches exactly one value. Symptom: selecting two clusters blanks the panel; the legend shows “No data”.

How to troubleshoot it

The diagnostic order:

1. Inspect the variable. Settings > Variables. The Type dropdown must match the use.
2. Click Run query. A failed query is the obvious failure shape; an empty list with no error is the subtle one. Cross-check the query field against the data source’s own query interface.
3. Confirm the refresh policy. refresh: 2 on a query variable that depends on a time-windowed expression can produce different lists at different time ranges; verify by switching the time-range picker.
4. Check the chain. A query variable that depends on another query variable may resolve differently if the parent variable’s value list has changed.
5. Inspect the panel’s data source. A datasource variable that points to a missing UID is a silent failure. The dropdown lists the right data sources; the panel UID has drifted.

Security implications

• query variables run with the data source’s permissions. The query does not carry the viewer’s identity; it runs as the data-source plugin. This is normally fine; be aware that a dashboard author can drive a query that the viewer would not be authorised to author directly.
• **textbox variables are unvalidated input. A textbox interpolated into PromQL is escaped by Grafana’s query builder, but the meaning of the text is not validated. A viewer can type .* to match every value, which is not a security exploit but is an operational surprise.
• datasource variables list every data source of the type the viewer can see. A viewer with limited data-source access still sees only the data sources they have access to. This is enforced by Grafana’s RBAC on /api/datasources.

Performance implications

• query variables dominate load time. Each refresh: 1 query is one round trip to the data source; each refresh: 2 query is one per time-range change.
• custom and constant variables are essentially free. No data source call; no I/O.
• datasource variables are a single in-memory call to Grafana’s data source registry. Effectively free.
• interval and system variables are computed locally. Effectively free.

The lesson on variable query cost examines query cost in detail.

Production guidance

• Default new variables to query when the value set is data-driven; default to custom when the value set is small and known.
• Use constant only for immutable per-dashboard facts. Use textbox only for free-form search; never for a filter that controls what panels render.
• Set refresh: 1 on query variables unless a refresh: 2 knob is operationally required. The cost of refresh: 2 is hidden in dashboard load latency.
• Set multi: true plus includeAll: true plus allValue: ".*" on query variables that use =~. This is the production default for filtering dimensions.
• Set skipUrlSync: true on variables that are per-tab-only (e.g., a “compare against staging” toggle on a production dashboard). The URL stays clean and shared links do not pin the toggle.

Verification

You should now be able to answer:

• Name the seven variable kinds and the value-list source of each.
• Which kind is appropriate for an immutable per-dashboard region tag, and which is appropriate for a data-driven list of clusters?
• What is the difference between includeAll: true and multi: true?
• When does a query variable refresh, and what does refresh: 2 mean?
• What is the failure shape of multi: true combined with = rather than =~?

Quiz