Unit Consistency

The capacity review meeting ran the dashboard on the wall. The on-call engineer said “we are at 80% disk”. The SRE lead said “the dashboard says 60%”. They looked at the same Prometheus, the same metric, and the same time range. The numbers disagreed because one panel divided bytes by 1024^3 and the other divided by 1000^3. The first panel rendered “GiB used / GiB total” in IEC units. The second panel rendered “GB used / GB total” in SI units. The same byte count became two different percentages depending on which panel the viewer opened.

This is what unit consistency is for. Every panel in a Grafana 11 dashboard declares a unit in panels[].fieldConfig.defaults.unit. The unit tells Grafana how to render the numeric value: the display suffix (MiB, ms, reqps), the conversion factor, and the decimal precision. A dashboard whose panels on the same dimension declare different units — or no unit at all — produces numbers the operator cannot compare, sum, or reason about.

Unit consistency is the per-panel and per-dashboard contract that every numeric panel declares a known unit, that panels on the same dimension declare the same unit, and that any unit conversion in the panel expr is self-consistent and aligned with the declared unit.

What it is

A Grafana 11 unit is a string identifier in panels[].fieldConfig.defaults.unit. The unit encodes three things:

  +------------------+--------------------------------+
  | Unit string      | Meaning                        |
  +------------------+--------------------------------+
  |  bytes           | Raw bytes; no conversion       |
  |  decbytes        | SI decimal (1000-based); KB,   |
  |                  | MB, GB, TB, PB                |
  |  binbytes        | IEC binary (1024-based); KiB,  |
  |                  | MiB, GiB, TiB, PiB            |
  |  s               | Seconds (raw); or ms, µs, ns  |
  |  percent         | Ratio 0-1 rendered as percent |
  |  percentunit     | Ratio 0-100 rendered as       |
  |                  | percent                        |
  |  reqps           | Requests per second             |
  |  ops             | Operations per second          |
  |  none            | No unit; render as raw number  |
  +------------------+--------------------------------+

The full unit catalog lives in Grafana’s public/app/features/units/ and is documented in the Grafana units reference. The catalog is the contract; panels that reference a string outside the catalog render as raw numbers with no suffix.

The panel’s fieldConfig.defaults.unit is the rendering contract; the panel’s targets[].expr is the query that produces the value. The two must agree: a panel whose expr divides by 1024^3 and whose unit is decbytes (1000-based) renders a number ten percent smaller than the actual byte count.

Unit consistency has four checks:

1. Unit declared. Every numeric panel declares a known unit. A panel with unit: "none" and a byte-producing query renders without a suffix.
2. Unit appropriate. The declared unit matches the underlying dimension. A panel on a byte metric with unit: "s" renders the byte count as a raw number followed by the second suffix — confusing at best.
3. Unit consistent across panels on the same dimension. Two panels on node_memory_MemTotal_bytes both declare binbytes (or both declare decbytes); one panel cannot divide by 1024 and the other by 1000.
4. Expr conversion aligned with declared unit. A panel whose expr divides by 1024^3 and declares binbytes is internally consistent; a panel whose expr divides by 1024^3 and declares decbytes shows the wrong number.

A panel that fails check 1 or 2 renders without useful context. A panel that fails check 3 makes cross-panel arithmetic wrong. A panel that fails check 4 shows a number that does not match the underlying metric.

Why a sysadmin cares

Three operational pains map directly to unit consistency:

1. The capacity review that disagrees. A capacity review opens the storage dashboard; one panel says 60% used, another says 80% used; the meeting stalls because nobody knows which number is right. The disagreement is a unit inconsistency, not a data disagreement.
2. The alert that fires at the wrong threshold. An alert on disk > 90% fires when the IEC panel says 91%; the SI panel says 88%. The on-call engineer investigates a non-incident because the unit declaration was wrong.
3. The SLO that is wrong by a factor of 1000. A request-duration SLO is set in seconds; the histogram exposes the bucket in milliseconds; the SLO is wrong by 1000x. The dashboard looks healthy; the SLO is violated silently.

The wrong shape shows up as a capacity review that ends without a decision, an alert that fires on the wrong threshold, or an SLO that nobody trusts because the display does not match the SLO document.

The SI vs IEC boundary

The single most expensive unit error is the SI vs IEC boundary on byte counts:

  +----------------+--------------------------------+
  |  Base          |  Conversion                    |
  +----------------+--------------------------------+
  |  SI (decimal)  |  1 KB = 1000 bytes             |
  |                |  1 MB = 1000 KB                |
  |                |  1 GB = 1000 MB                |
  |                |  1 TB = 1000 GB                |
  |                |  1 PB = 1000 TB                |
  +----------------+--------------------------------+
  |  IEC (binary)  |  1 KiB = 1024 bytes            |
  |                |  1 MiB = 1024 KiB              |
  |                |  1 GiB = 1024 MiB              |
  |                |  1 TiB = 1024 GiB              |
  |                |  1 PiB = 1024 TiB              |
  +----------------+--------------------------------+

A 1 TiB disk (IEC) holds 1,099,511,627,776 bytes. A 1 TB disk (SI) holds 1,000,000,000,000 bytes. The marketing material says TB; the operating system reports TiB; the dashboard reports whichever unit the author declared. The disagreement is exactly 9.95%.

Prometheus exposes bytes natively (node_memory_*_bytes, node_filesystem_*_bytes). The convention in the Prometheus ecosystem is SI (the units are suffixed _bytes but the dashboards typically render in binbytes or decbytes). The right choice depends on the audience: operator-facing dashboards use binbytes (GiB); exec-facing dashboards use decbytes (GB) to match the marketing material. Never mix the two within the same dashboard.

How it works

The unit-consistency pipeline:

  dashboard JSON
        |
        v
  +-------------------------+
  |  parse panels[]         |   jq '.panels[]'
  +-------------------------+
        |
        v
  +-------------------------+
  |  for each panel:        |
  |  read unit field        |   jq '.fieldConfig.defaults.unit'
  +-------------------------+
        |
        v
  +-------------------------+
  |  classify each panel    |   group by dimension (memory, disk,
  |  by dimension            |   cpu, network, latency, rate)
  +-------------------------+
        |
        v
  +-------------------------+
  |  within each group:     |   confirm every panel declares the
  |  confirm units match    |   same unit; flag mismatches
  +-------------------------+
        |
        v
  +-------------------------+
  |  inspect expr for       |   grep for /1024 or /1000 in expr;
  |  conversion factors     |   cross-check with declared unit
  +-------------------------+
        |
        v
  +-------------------------+
  |  report                 |   list of unitless panels, mismatched
  |                         |   units, expr/unit misalignments
  +-------------------------+

How to configure it

The canonical pattern: a CI script that walks every panel, classifies by dimension, and checks unit consistency within each dimension.

#!/usr/bin/env bash
# scripts/check-units.sh
# Severity: READ-ONLY against the dashboard JSON.
set -euo pipefail

KNOWN_UNITS=$(cat <<'EOF'
bytes decbytes binbytes kibibytes mebibytes gibibytes
tebibytes pebibytes
s ms µs ns
percent percentunit
reqps ops
none
short
EOF
)

classify_dimension() {
  local expr=$1
  if echo "$expr" | grep -qE 'memory_Mem|meminfo|MemTotal'; then
    echo "memory"
  elif echo "$expr" | grep -qE 'filesystem_|node_filesystem'; then
    echo "disk"
  elif echo "$expr" | grep -qE 'duration_seconds|latency|histogram_quantile'; then
    echo "latency"
  elif echo "$expr" | grep -qE 'cpu|loadavg'; then
    echo "cpu"
  else
    echo "unknown"
  fi
}

check_panel() {
  local json_file=$1
  local title=$2
  local unit=$3
  local expr=$4
  if [[ -z "$unit" || "$unit" == "null" ]]; then
    echo "WARN: $json_file panel '$title' has no unit declared"
    return
  fi
  if ! echo "$KNOWN_UNITS" | grep -qw "$unit"; then
    echo "FAIL: $json_file panel '$title' declares unknown unit: $unit"
    return 1
  fi
  dim=$(classify_dimension "$expr")
  echo "  $dim  $title  $unit  $expr"
}

for json_file in dashboards/*.json; do
  jq -c '.panels[]?
         | select(.targets != null)
         | {title, unit: .fieldConfig.defaults.unit,
            expr: .targets[0].expr}' "$json_file" \
    | while read -r panel; do
        title=$(echo "$panel" | jq -r .title)
        unit=$(echo "$panel" | jq -r .unit)
        expr=$(echo "$panel" | jq -r .expr // "")
        check_panel "$json_file" "$title" "$unit" "$expr"
      done
done

A minimal panel with a correctly declared unit:

{
  "type": "timeseries",
  "title": "Memory usage",
  "datasource": { "type": "prometheus", "uid": "prom-prod" },
  "targets": [
    {
      "refId":  "A",
      "datasource": { "type": "prometheus", "uid": "prom-prod" },
      "expr":    "node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes"
    }
  ],
  "fieldConfig": {
    "defaults": {
      "unit": "binbytes",
      "decimals": 2
    },
    "overrides": []
  }
}

The unit is binbytes (IEC binary, GiB). The expr returns bytes. Grafana’s renderer divides by 1024^n to display GiB. The panel is internally consistent.

A panel that mixes units (the failure shape):

{
  "title": "Memory usage (GiB)",
  "targets": [
    {
      "expr": "(node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes)
               / (1024 * 1024 * 1024)"
    }
  ],
  "fieldConfig": {
    "defaults": {
      "unit": "binbytes",
      "decimals": 2
    }
  }
}

The expr divides by 1024^3; the unit is binbytes; the renderer divides by 1024^n again. The displayed value is bytes / 1024^6, which is wrong by a factor of roughly a million.

The fix is to declare unit: "bytes" (no conversion) and remove the / 1024^3 from the expr, or remove the / 1024^3 from the expr and keep unit: "binbytes". The CI step flags the inconsistency.

How to validate it

Five checks confirm the unit discipline is live.

Severity: READ-ONLY.

# 1. Every panel declares a unit. A panel with no unit
#    renders as a raw number with no suffix.
curl -s -u admin:$ADMIN \
  https://grafana.example.com/api/dashboards/uid/svc-overview \
  | jq '[.dashboard.panels[]
        | select((.fieldConfig.defaults.unit // "none") == "none")]
        | length'
# 0

# 2. Every declared unit is in the Grafana catalog.
#    A typo (e.g., "binbyte" instead of "binbytes") falls
#    back to raw rendering with no suffix.
curl -s -u admin:$ADMIN \
  https://grafana.example.com/api/dashboards/uid/svc-overview \
  | jq -r '.dashboard.panels[]
          | .fieldConfig.defaults.unit' | sort -u
# binbytes
# s
# reqps

# 3. Panels on the same dimension declare the same unit.
#    Two panels on node_memory_* should both declare
#    binbytes or both declare decbytes.
curl -s -u admin:$ADMIN \
  https://grafana.example.com/api/dashboards/uid/svc-overview \
  | jq -r '.dashboard.panels[]
          | select(.targets[0].expr | test("memory_"))
          | .fieldConfig.defaults.unit' | sort -u
# binbytes

# 4. Expr conversion aligns with declared unit. A panel
#    that divides by 1024 and declares binbytes is
#    consistent; the same panel declaring decbytes is not.
for expr in $(curl -s -u admin:$ADMIN \
  https://grafana.example.com/api/dashboards/uid/svc-overview \
  | jq -r '.dashboard.panels[].targets[].expr'); do
  unit=$(curl -s -u admin:$ADMIN \
    https://grafana.example.com/api/dashboards/uid/svc-overview \
    | jq -r ".dashboard.panels[] | select(.targets[].expr == \"$expr\")
            | .fieldConfig.defaults.unit")
  if echo "$expr" | grep -qE '/ *\(?1024'; then
    if [[ "$unit" != "binbytes" && "$unit" != "bytes" ]]; then
      echo "MISALIGN: $expr with unit $unit"
    fi
  fi
done

# 5. Cross-panel arithmetic is consistent. Two panels on
#    the same metric should yield the same ratio when
#    compared. A capacity review that adds "GiB used"
#    from one panel and "GB total" from another is wrong
#    by a factor of 1000/1024.

How it can fail

Six failure shapes appear repeatedly with unit consistency:

1. Bytes vs MiB mismatch. One panel declares unit: "bytes" and renders raw bytes (e.g., 8589934592); another declares unit: "binbytes" and renders the same metric as 8 GiB. Symptom: the same byte count appears as two different numbers in the same dashboard.
2. Seconds vs milliseconds. A request-duration histogram exposes buckets in seconds; the dashboard declares unit: "ms". The renderer divides by 1000; the displayed latency is 1000x smaller than the actual. Symptom: a p99 panel that says 1.5 ms for a service that actually has 1.5 s p99 latency.
3. Percentage without base. A panel on a gauge that already returns 0-100 declares unit: "percent" (which expects 0-1). Symptom: a “disk at 80%” panel that renders as 8000%.
4. Mixed SI/IEC on the same dimension. Two panels on node_memory_MemTotal_bytes declare binbytes and decbytes. Symptom: the same metric shows different numbers depending on which panel the viewer opens.
5. Expr double-conversion. The expr divides by 1024^3 and the unit is binbytes. Symptom: the displayed value is wrong by a factor of 1024^3.
6. No unit declared. A panel declares unit: "none" on a byte metric. Symptom: the panel renders the raw byte count (e.g., 8589934592) with no suffix; the viewer cannot tell what dimension the number is in.

How to troubleshoot it

The diagnostic order:

1. Open the panel. Click Edit. Inspect the fieldConfig.defaults.unit field.
2. Cross-check the panel against the underlying metric. A byte metric should declare a byte unit; a time metric should declare a time unit; a ratio should declare a percent unit.
3. Inspect the expr. A panel whose expr divides by 1024 or 1000 has done manual conversion; the declared unit must be bytes (raw) or none.
4. Open other panels on the same metric. Two panels on the same metric should declare the same unit. A disagreement is the canonical inconsistency.
5. Open the data source directly. Prometheus’s /graph with the panel’s expr confirms the raw value; compare against the dashboard’s rendered value to find the conversion factor.

Security implications

Unit consistency has no direct security implications. The field is metadata; the value is metadata; the data source credentials are unaffected. The indirect risk is operational trust: a dashboard that disagrees with itself loses credibility with leadership, and leadership that loses trust in the dashboard loses trust in the platform.

Performance implications

Unit consistency has no direct performance implications. The renderer is a constant-time conversion; the cost is negligible compared to the panel query. The indirect risk is query cost: a panel whose expr divides by 1024^3 and then is divided by 1024^n again is the same query; the double-conversion does not double the cost, it just renders the wrong number.

Production guidance

• Declare a unit on every numeric panel. unit: "none" is acceptable for dimensionless values (request counts, error counts); never use it for bytes, time, or ratios.
• Standardise on one unit per dimension within a dashboard. Memory panels declare binbytes; disk panels declare decbytes; latency panels declare s.
• Use binbytes for operator-facing dashboards and decbytes for exec-facing dashboards. The marketing material uses SI; the operating system uses IEC; pick the audience and stick to it.
• Avoid manual conversion in panel exprs. Let Grafana’s renderer do the conversion; the expr returns the raw value, the unit declares the display.
• Document the unit convention in the dashboard description. The viewer should know whether to expect GiB or GB before they read the panel.

Verification

You should now be able to answer:

• What are the four checks that define unit consistency?
• What is the difference between SI (decimal) and IEC (binary) byte units, and where does each apply?
• Where does Grafana 11 declare a panel’s unit, and what is the catalog?
• What is the failure shape of an expr that divides by 1024 with a declared unit of binbytes?
• Why is mixed SI/IEC on the same dimension the most expensive unit error?

Quiz