Business Dashboards

The executive team is in a quarterly review. The head of growth opens a slide titled “Conversion funnel, last 30 days”. The charts on the slide are screenshots of a Grafana dashboard. The dashboard was built by an engineer who left the company eight months ago. The dashboard has the team’s logo in the corner, the executive team’s user accounts in the permissions, and “Auto refresh: 30s” on the time picker. The dashboard has not been refreshed in 30 days because the time picker is set to “Last 30 days” and the data source is the test environment.

The slide is wrong. The charts show the test environment’s conversion rate, which is twice the production rate, because the test environment has the seeded demo data. The executive team notices the conversion rate is too high; the next question is “how did we get there so fast?”; the actual answer is “you didn’t”. The review ends with a confused executive team and a data team that has to spend the next week rebuilding the dashboard.

The failure is not the data. The failure is the dashboard. The dashboard had no owner, no documented methodology, no contact for “this is wrong”, and no review cadence. The dashboard was decoration. The dashboard was a screenshot.

What it is

A business dashboard is a Grafana panel that answers a business question with the live Prometheus data. A useful business dashboard has four attributes:

1. Owner. A team, a name, and a contact. The owner is the person who answers the question “this is wrong, who do I talk to?” without an investigation.
2. Question. A single sentence the panel answers. “What is the conversion rate from signup to activation, last 7 days, per plan?” is a question. “Conversion” is not.
3. Cadence. The data refresh interval and the time window the panel covers. A panel that refreshes every 30 seconds but covers the last 30 days is a misuse of the time picker.
4. Methodology. The formula that produced the number. The formula is a comment on the dashboard, not a link to a wiki.

The dashboard is decoration when the four attributes are missing. The dashboard is useful when the four attributes are present and correct.

  Decoration (no owner)         Useful (owned)
  --------------------------    --------------------------
  Owner:        missing         Owner: growth@example.com
  Question:     "Conversion"    Question: "What is the signup to
                                  activation conversion rate, last
                                  7 days, per plan?"
  Cadence:      varies          Cadence: 30s refresh, 7d window
  Methodology:  unknown         Methodology: documented in the
                                  panel description

The owner is the most important attribute. The other three can be inferred from the dashboard; the owner cannot. The team that has the owner on every dashboard has the discipline. The team that has the owner on a few dashboards has the start of the discipline.

Why a sysadmin cares

The operator is the team that builds Grafana. The operator is the team that maintains the dashboard. The operator is the team that catches the dashboard when the underlying metric changes.

Three reasons the operator cares:

1. The dashboard is the operator’s reputation. The executive team reads the dashboard. The dashboard that is wrong is the operator that is wrong. The owner field is the operator’s contract.
2. The dashboard is the operator’s documentation. The dashboard JSON is the source of truth for the queries. The team that owns the dashboard owns the queries.
3. The dashboard is the operator’s contract with the business. The methodology is the contract. The dashboard JSON is the executed contract. The two must agree.

The operator who treats the dashboard as a Grafana artifact and not a business artifact is the operator who has the orphan panel.

How it works

The Grafana dashboard JSON model is the implementation. The model has four key fields that correspond to the four attributes:

  Attribute       JSON field             Grafana UI field
  -------------   --------------------   -----------------------------
  Owner           "annotations.list"     Dashboard description
                  (with a contact link)
  Question        "panels[].title"       Panel title
                  "panels[].description" Panel description
  Cadence         "time"                 Time picker
                  "refresh"              Refresh interval
  Methodology     "panels[].description" Panel description
                  "panels[].links"       Panel links to runbook

The discipline is to populate every field. The owner field is not in the default JSON model; it is a Grafana annotation. The question is the panel title. The cadence is the time picker and the refresh interval. The methodology is the panel description and the runbook link.

How to configure it

The configuration is a complete Grafana dashboard with the four attributes and the four canonical panels.

# /etc/grafana/provisioning/dashboards/business.yaml
# Provisioning file that loads the business dashboard from the
# filesystem. The dashboard JSON is the source of truth.
apiVersion: 1
providers:
  - name: business
    orgId: 1
    folder: Business
    type: file
    disableDeletion: false
    updateIntervalSeconds: 30
    allowUiUpdates: true
    options:
      path: /var/lib/grafana/dashboards/business
      foldersFromFilesStructure: true

The dashboard JSON (excerpt) sets the four attributes:

{
  "title": "Conversion Funnel — Owned by Growth",
  "uid": "business-conversion",
  "description": "Owner: Growth (growth@example.com). Review cadence: quarterly. Methodology: rate(orders_completed_total[5m]) / rate(orders_started_total[5m]). Data source: Prometheus (prometheus-prod).",
  "tags": ["business", "owned", "growth"],
  "time": {
    "from": "now-7d",
    "to": "now"
  },
  "refresh": "30s",
  "templating": {
    "list": [
      {
        "name": "plan",
        "type": "query",
        "datasource": "Prometheus",
        "query": "label_values(orders_completed_total, plan)",
        "refresh": 2,
        "includeAll": true
      },
      {
        "name": "country",
        "type": "query",
        "datasource": "Prometheus",
        "query": "label_values(orders_completed_total, country)",
        "refresh": 2,
        "includeAll": true
      }
    ]
  },
  "panels": [
    {
      "title": "Funnel — per stage, last 7 days",
      "description": "Source: orders_<stage>_total. Unit: events. Last 7 days. Owner: growth@example.com.",
      "type": "timeseries",
      "datasource": "Prometheus",
      "targets": [
        {
          "expr": "sum(rate(orders_started_total{plan=~\"$plan\",country=~\"$country\"}[5m]))",
          "legendFormat": "started"
        },
        {
          "expr": "sum(rate(orders_paid_total{plan=~\"$plan\",country=~\"$country\"}[5m]))",
          "legendFormat": "paid"
        },
        {
          "expr": "sum(rate(orders_completed_total{plan=~\"$plan\",country=~\"$country\"}[5m]))",
          "legendFormat": "completed"
        },
        {
          "expr": "sum(rate(orders_refunded_total{plan=~\"$plan\",country=~\"$country\"}[5m]))",
          "legendFormat": "refunded"
        }
      ]
    },
    {
      "title": "Conversion rate — started to completed",
      "description": "rate(completed) / rate(started). Last 7 days. Owner: growth@example.com.",
      "type": "stat",
      "datasource": "Prometheus",
      "targets": [
        {
          "expr": "sum(rate(orders_completed_total{plan=~\"$plan\",country=~\"$country\"}[5m])) / sum(rate(orders_started_total{plan=~\"$plan\",country=~\"$country\"}[5m]))",
          "legendFormat": "conversion"
        }
      ]
    },
    {
      "title": "Revenue lost per minute, last 7 days",
      "description": "Recording rule: revenue_lost_usd_per_minute:impact. Unit: USD per minute. Last 7 days. Owner: growth@example.com.",
      "type": "timeseries",
      "datasource": "Prometheus",
      "targets": [
        {
          "expr": "sum(revenue_lost_usd_per_minute:impact{plan=~\"$plan\",country=~\"$country\"})",
          "legendFormat": "lost/min"
        }
      ]
    },
    {
      "title": "Top 10 countries by orders completed",
      "description": "Top 10 over the last 7 days. Owner: growth@example.com.",
      "type": "table",
      "datasource": "Prometheus",
      "targets": [
        {
          "expr": "topk(10, sum by (country) (increase(orders_completed_total[7d])))",
          "format": "table",
          "instant": true
        }
      ]
    }
  ]
}

The four panels are the canonical shape. The funnel is the per-stage rate. The conversion is the ratio. The revenue is the financial impact. The top cohorts is the slice. The methodology is in the panel description.

How to validate it

# Does the dashboard load?
curl -s http://grafana:3000/api/dashboards/uid/business-conversion \
  -H "Authorization: Bearer $GRAFANA_API_KEY" \
  | jq '.dashboard.title'
# Expected: "Conversion Funnel — Owned by Growth"

# Does each panel have a description?
curl -s http://grafana:3000/api/dashboards/uid/business-conversion \
  -H "Authorization: Bearer $GRAFANA_API_KEY" \
  | jq '.dashboard.panels[] | {title, description}'
# Expected: each panel has a description that includes the owner

# Is the dashboard level description present?
curl -s http://grafana:3000/api/dashboards/uid/business-conversion \
  -H "Authorization: Bearer $GRAFANA_API_KEY" \
  | jq '.dashboard.description'
# Expected: a description that includes "Owner:" and "Methodology:"

# Do the queries return data?
curl -s http://localhost:9090/api/v1/query \
  --data-urlencode 'query=sum(rate(orders_completed_total[5m]))' \
  | jq '.data.result | length'
# Expected: at least 1 series

# Is the dashboard folder permission set?
curl -s http://grafana:3000/api/folders \
  -H "Authorization: Bearer $GRAFANA_API_KEY" \
  | jq '.[] | select(.title == "Business") | .id'
# Expected: a folder ID

How it can fail

1. Orphaned dashboard. Symptom: the dashboard has no owner in the description; the on-call engineer cannot find the contact. The fix is to add the owner field; the discipline is the dashboard review.
2. Decayed panel. Symptom: a panel exists that no discipline owns; the next review moves the panel to a graveyard. The fix is the panel review; the discipline is the panel ownership.
3. Stale query. Symptom: the panel’s PromQL expression references a metric that has been renamed; the panel shows “No data”. The fix is to verify the query against the live metric; the discipline is the quarterly review.
4. Methodology drift. Symptom: the executive team asks “how is this number computed?” and the answer is “the metric team thinks it is computed like this”. The fix is the methodology in the panel description.
5. Time picker mismatch. Symptom: the panel covers the last 30 days but the refresh interval is 30 seconds; the panel is a misuse of the time picker. The fix is to set the time picker to the right window for the cadence.
6. Wrong data source. Symptom: the panel queries the test environment Prometheus; the numbers are the test data. The fix is to point the panel at the production Prometheus; the discipline is the data source contract.

How to troubleshoot it

1. Does the dashboard have an owner? Open the dashboard description; look for “Owner:”. If missing, the dashboard is decoration.
2. Does each panel have a description? Open each panel; look for the methodology. If missing, the panel is decoration.
3. Do the queries return data? Run each query in the Grafana Explore view. If empty, the metric is missing.
4. Is the data source correct? Check the panel’s data source. If the panel is querying the test environment, the dashboard is decoration.
5. Is the dashboard review documented? Look for the last review date. If missing, the dashboard is decoration.

Security implications

• Dashboard permissions. The Grafana folder permission model is the security boundary. A business dashboard with revenue data should be visible only to the teams that need to see the revenue. The folder permission is the enforcement point.
• Data source credentials. The Prometheus data source in Grafana is a read credential to the TSDB. The credential should be scoped to the business metrics, not the technical metrics. The data source configuration is the enforcement point.
• Methodology disclosure. The methodology in the panel description is public to the Grafana users. A methodology that exposes a sensitive business rule should be in the runbook, not the panel description. The discipline is the public-private boundary.

Performance implications

The performance cost of a business dashboard is dominated by the query cost, not the panel rendering. The cost model:

  Panels per dashboard    Queries per panel    Total queries per refresh
           4                     4                          16
           8                     4                          32
          16                     4                          64    <- the cliff

The cliff is the cost of the dashboard refresh. A 30-second refresh on a 16-panel dashboard is 64 queries every 30 seconds on the Prometheus backend. The cost is real; the right answer is to use recording rules for the slow-moving panels and the Grafana transformations for the in-panel arithmetic.

The trade-off is the dashboard cadence. A 30-second refresh on a 7-day window is a misuse of the time picker. A 5-minute refresh on a 7-day window is the right cadence for a business dashboard. The 30-second refresh is for live technical dashboards.

Verification

You should now be able to answer:

• What are the four attributes of a useful business dashboard?
• Why is the owner field the most important attribute?
• What is the canonical shape of a business dashboard, and what question does each panel answer?
• Why is the 30-second refresh a misuse of the 7-day window on a business dashboard?
• What is the first symptom that the dashboard review is not happening?

Quiz