Dashboard-to-Logs Anatomy

A Grafana dashboard shows a time-series panel: HTTP 5xx rate, grouped by service and route, over the last hour. The line for checkout-svc / POST /v2/cart climbs sharply at 14:32. The on-call engineer clicks the line. They expect to land in Explore with the right time window, the right labels, and a Loki stream that contains the actual errors that explain the spike.

What usually happens is one of three things. They land on a Loki page that returns every log line from the service for that hour — 47,000 of them — and the engineer scrolls for ten minutes. They land on a Loki page that returns nothing because the URL template used {{route}} but the metric label is called handler. They land on an Explore view that opens the whole-stack Loki Explore page with no filter at all, and the “correlation” they were promised is the same blank panel they started from.

The dashboard-to-logs workflow is the bridge between “the metric says something is wrong” and “the log line that explains it”. It is small in lines of YAML but unforgiving in detail: the URL template either captures the right values or it captures nothing.

What it is

A dashboard-to-logs pivot is a click in a Grafana panel that opens Loki (Explore, a Logs panel, or the Loki data source’s drilldown view) pre-populated with a query that is already scoped to the thing the operator clicked. Grafana 11.x exposes two mechanisms for this:

• Panel-level data links — the per-panel “Data links” tab in the panel editor. Each rule is a named link with a URL template that may reference series labels, field values, and the dashboard time range. This is the classical mechanism and remains the right answer for one-off panels.
• Dashboard-level datasource_correlations — a Grafana 11.0 feature that declares, at the dashboard JSON level, a correlation between a metric data source and a logs data source. The metric panel then renders a “View logs” button on each data point; the click opens Explore with the query pre-populated from a label-mapping the dashboard author defines once. This is the right answer when the same metric to-logs mapping appears on many panels across a dashboard or across the catalogue.

Both mechanisms do the same job: take the panel the engineer is looking at, and the label values on the line they clicked, and turn them into a Loki query that returns a small, relevant stream of log lines for the same time window. The shape of the resulting query, not the mechanism, is what matters to the engineer on call.

Why a sysadmin cares

The pivot is the difference between “we have metrics and we have logs” and “we use them together”. Four production shapes appear in teams that have not designed the pivot:

• The scroll-of-shame pivot. The URL template passes the service name and the time range but not the route or the status code. The resulting Loki stream is 100,000 lines of mixed info, warn, and error output. The engineer scrolls past the noise and gives up. The “one-click correlation” the dashboard advertises is a UI that makes incidents worse.
• The silent label-name drift. The panel uses handler in its metric legend; the URL template references ${__series.labels.handler} but the panel was renamed to route in a refactor six months ago. The link is rendered, the click goes to Loki, the query returns zero lines. The engineer concludes the service produced no logs.
• The wrong data source pivot. The URL template writes {job="${service}"} against the default Loki data source. In a multi-cluster setup that default points at the EU cluster; the engineer is paged for the US cluster. The pivot opens and returns nothing; the engineer concludes the US service produced no logs. The two services are unrelated.
• The time-window pivot. The dashboard is showing the last 6 hours; the panel data link does not pass a time range to Loki. The Explore view opens Loki with Loki’s default 6-hour window but at the wrong offset, or with the dashboard’s “now” which has since drifted forward by the time the engineer clicks.

How it works

The mental model is “the click is a query”. When the operator clicks a panel point, Grafana collects the labels on the series, the time of the point, and the dashboard’s current time range. It renders the data link’s URL template against those values. It opens that URL in a new tab (or in Explore in place, depending on configuration). The destination interprets the URL parameters; in the case of Loki, the parameters become a LogQL stream selector and a time range.

   metric panel                 data link              Loki Explore
   -------------                ----------             ------------
   series: checkout-svc        title='View logs'       query: {service="checkout-svc",
            /v2/cart           url=/explore?          route="/v2/cart",
   value: 5xx rate = 17.3      ds=loki-prod-us&        status="500"}
   time:  2026-08-13T14:32:00Z  query={service=        from: 14:31:30
                                "${service}",          to:   14:32:30
                                route="${route}",
                                status="500"}
                                &from=${__value.time:date-seconds}-60
                                &to=${__value.time:date-seconds}+60

Four observations on this shape:

1. Labels become LogQL stream selectors. The metric series carries labels; the LogQL stream selector uses the same names. A name like handler on the metric side has to match the Loki label name on the log side. The bridge either preserves names or has an explicit mapping.
2. Time becomes the time range. The point the operator clicked is the centre of a small window (60 s before, 60 s after by convention). The window is small enough that the resulting stream fits on one screen; the operator is investigating the anomaly, not the whole hour.
3. Filter values become pipe-filters. If the metric panel is 5xx rate by route, the data link passes status="500" to the Loki query as a pipe filter. The stream is pre-narrowed to the lines that explain the spike.
4. The data source UID becomes the destination. The URL template names the data source by UID, not by display name. A misspelled UID opens the wrong cluster, the wrong environment, or nothing.

How to configure it

There are two configurations that matter. The first is the panel-level data link. The second is the dashboard-level datasource_correlations block, which is the right choice when the same pivot is shared across many panels.

{
  "type": "timeseries",
  "title": "5xx rate by service and route",
  "datasource": { "type": "prometheus", "uid": "prom-prod-us" },
  "targets": [
    {
      "expr": "sum by(service, route) (rate(http_requests_total{status=~\"5..\"}[5m]))",
      "refId": "A"
    }
  ],
  "fieldConfig": {
    "defaults": {
      "links": [
        {
          "title": "View logs for ${__series.labels.service} ${__series.labels.route}",
          "url": "/explore?schemaVersion=1&panes=%7B%22logs%22%3A%7B%22datasource%22%3A%22loki-prod-us%22%2C%22queries%22%3A%5B%7B%22expr%22%3A%22%7Bservice%3D%5C%22%24%7B__series.labels.service%7D%5C%22%2Croute%3D%5C%22%24%7B__series.labels.route%7D%5C%22%2Cstatus%3D%5C%225%5C%22%7D%22%7D%5D%7D%7D&from=${__value.time:date-seconds}-120&to=${__value.time:date-seconds}+120",
          "targetBlank": true,
          "includeVars": true
        }
      ]
    }
  }
}

The relevant options:

• title is the link’s display label. The template ${__series.labels.<name>} is replaced with the value of that label on the clicked series. The label makes the link easy to scan when several appear on one row.
• url is the full Explore URL with the LogQL query, the data source UID, and the time range. The panes parameter is the URL-encoded JSON that Grafana 11.x uses to restore Explore state. expr is the LogQL stream selector; ${__series.labels.<name>} becomes the matched label value.
• targetBlank: true opens the link in a new tab. The alternative — opening in the same tab — replaces the dashboard the engineer is investigating. Prefer a new tab for incident workflows.
• includeVars: true forwards the dashboard template variables (e.g. ${env}) to the destination. Useful when the destination data source uses the same variable scheme.

The dashboard-level equivalent for the same pivot is a datasource_correlations block:

{
  "panels": [ /* ... the panel above ... */ ],
  "__elements": {},
  "__requires": [],
  "datasource_correlations": [
    {
      "uid": "prom-prod-us-loki-prod-us",
      "sourceUID": "prom-prod-us",
      "targetUID": "loki-prod-us",
      "label": "Logs",
      "description": "Pivot from any 5xx-rate series to the underlying log lines",
      "config": {
        "field": "service",
        "target": {
          "expr": "{service=\"${__field.labels.service}\", route=\"${__field.labels.route}\", status=\"5\"}"
        },
        "type": "logs"
      }
    }
  ]
}

The field value is the source label the operator clicked; __field.labels.<name> resolves to the clicked series’s label value. The dashboard-level form is harder to author by hand (the schema has shipped revisions across 11.x) and is the right choice when the same pivot appears on dozens of panels.

How to validate it

# READ-ONLY: list the data links on a panel.
curl -fsS -u grafana-admin:$GRAFANA_ADMIN \
  "http://grafana.internal:3000/api/dashboards/uid/${DASH_UID}" \
  | jq '.dashboard.panels[].fieldConfig.defaults.links // []'

# READ-ONLY: substitute the template with one set of label
# values and confirm the resulting URL renders to a valid
# Explore page.
SVC=checkout-svc
ROUTE=/v2/cart
T=1755100320  # 2026-08-13T14:32:00Z, in seconds since epoch
URL="http://grafana.internal:3000/explore?schemaVersion=1\
&panes=%7B%22logs%22%3A%7B%22datasource%22%3A%22loki-prod-us%22%2C%22queries%22%3A%5B%7B%22expr%22%3A%22%7Bservice%3D%5C%22${SVC}%5C%22%2Croute%3D%5C%22${ROUTE}%5C%22%2Cstatus%3D%5C%225%5C%22%7D%22%7D%5D%7D%7D\
&from=$((T-120))&to=$((T+120))"
curl -fsS -u grafana-admin:$GRAFANA_ADMIN "$URL" \
  -o /dev/null -w '%{http_code}\n'
# 200

# READ-ONLY: a direct LogQL probe against the same selector.
curl -fsS -u grafana-admin:$GRAFANA_ADMIN \
  --data-urlencode 'query={service="checkout-svc",route="/v2/cart",status="5"}' \
  --data-urlencode "start=$((T-120))000000000" \
  --data-urlencode "end=$((T+120))000000000" \
  --data-urlencode 'limit=20' \
  http://grafana.internal:3000/api/datasources/proxy/uid/loki-prod-us/loki/api/v1/query_range
# {"status":"success","data":{"resultType":"streams","result":[ ... ]}}

A clean validation: the rendered URL returns 200, the proxy returns Loki streams, and the streams contain log lines from the time window. A failure mode below maps to one of these signals failing.

How it can fail

The most expensive pivot failure modes from real production incidents.

1. Label-name drift. The metric legend uses handler; the Loki stream label is route. The URL template references ${__series.labels.route}; the substitution is empty; the LogQL {service="x", route="", status="5"} returns no lines. The symptom is a link that opens to “no results” without an error message.
2. Wrong data source UID. The URL targets loki-prod-eu while the metric comes from prom-prod-us. The Explore view opens against the EU Loki; the lines for the US service are not there. The symptom is “the logs are missing” for an outage in the US cluster.
3. No time range in the URL. The link opens Explore with Loki’s default time range, which is “last 6 hours” or whatever the data source default is. The panel was showing a 30-day window; the Explore view shows the wrong window. The symptom is “the click took me to the wrong time” and the engineer spends minutes adjusting the time picker.
4. No narrow-by-status filter. The URL passes {service="${service}"} and nothing else. Loki returns every line from the service for the window — including info and debug. The symptom is the scroll-of-shame pivot that drives engineers to ignore the link entirely.
5. Double-encoded JSON. The panes parameter is encoded once when it should be twice, or twice when it should be once. The link opens to a malformed Explore page; the URL bar shows the wrong characters. The symptom is a 200 with an empty panel.
6. includeVars: false on a dashboard with environment variables. The link forwards no variable context. The Explore view uses the data source’s default variable values. The symptom is “the pivot opened against staging when I was looking at production”.

How to troubleshoot it

The diagnostic order is “does the URL render?”, “does the URL return what is expected?”, “does the destination data source respond?”, “does the resulting LogQL query return lines?”.

1. Render the template by hand. Pick a panel point, substitute the labels and the time into the URL template manually. If the resulting URL does not parse, the template is wrong.
2. Open the URL in a private browser tab. A session with bad cookies, missing data source permissions, or a stale Grafana version produces different results from a fresh session. The private tab is the baseline.
3. Check the destination data source. Run the same query through /api/datasources/proxy/uid/<uid>/loki/api/v1/query. A working direct query that returns nothing through the link isolates the problem to the URL template.
4. Check the label names. Run a metric query that returns the labels on the panel series. Compare the labels to the LogQL stream selector. A name like handler on the metric side and route on the log side is a name drift.
5. Check the time range. The metric panel renders at a specific time. The data link should pass a window around that time, not the dashboard’s full range. A link that passes from=now-6h&to=now is a link that opens the wrong window.
6. Inspect Grafana’s logs. /var/log/grafana/grafana.log records every data link resolution and every proxy call. A 4xx response from Loki appears with the query string and the upstream message.

Security implications

• The pivot passes label values into a URL. A label like customer_id that exists on the metric becomes a URL parameter; the URL lands in browser history, in the server access log, and in the destination data source’s query log. Audit the labels a pivot passes; a customer_id pivot is a data-handling bug.
• The destination data source has its own permissions. A pivot that targets a Loki data source with broader read permissions than the source Prometheus allows an operator with metric-only access to read logs they should not. Review the data source’s RBAC alongside the pivot.
• The includeVars flag forwards dashboard variables. If a variable contains a secret or a PII label, the variable value lands in the destination URL. Treat variable values as values that flow.

Performance implications

• A wide pivot is a slow query. A pivot that passes {service="x"} and nothing else asks Loki for every line from the service. Loki returns them; the browser freezes. The cost of a wide pivot is paid in Loki CPU and browser memory.
• The time window matters. A 30-day window with no status filter asks Loki to scan the service’s chunk for the whole retention period. The query is slow at best, rejected at worst.
• A correct pivot is fast. A pivot that passes {service="x", route="y", status="5"} with a 4-minute window around the spike returns tens of lines in a few hundred milliseconds. The cost is paid once and the engineer sees the answer.

Production guidance

• Define the pivot once per (metric data source, logs data source) pair. Use datasource_correlations for the common case; use panel-level data links for the panel-specific case.
• Name the bridge labels explicitly. The metric legend and the Loki stream selector must agree on the label names. Map handler to route at the source if necessary; do not silently rely on luck.
• Always pass a small time window. Two minutes before the point, two minutes after. The window is the anomaly, not the hour.
• Always pass a status or severity filter when the metric is a rate of failures. The pivot should not return info lines.
• Pin the data source by UID, not by name. A rename of the data source in the catalogue should not break every pivot.
• Exercise pivots quarterly. A pivot that has not been clicked in production is a pivot that has drifted.

Verification

You should now be able to answer:

• What are the two Grafana 11.x mechanisms that pivot from a metric panel to Loki, and which is the right choice for a pivot that is shared across many panels?
• Which four template variables drive the URL the operator clicks, and which one controls the time window?
• What is the “scroll-of-shame pivot” and which two template-variable mistakes cause it?
• Why does the data source UID matter more than the data source display name when authoring a pivot?

Quiz