LogQL Parsers

The application team has been writing structured logs for years. The fields are right there in the line: {"ts":"2026-08-14T14:02:11Z","level":"error","request_id":"7f4a...","msg":"payment_intent_failed","latency_ms":1240}. But every Grafana panel still filters with |~ "level=error", because the parser is missing. The fields exist; the query engine does not know they exist.

The parser is the bridge between “log content” and “queryable fields”. Without it, every filter is a string match against the whole line.

What a parser is

A parser is a pipeline stage that runs after the line filters and extracts structured fields from the line content. LogQL exposes three parsers, each suited to a different log shape:

• | json — extracts every JSON object that appears as a standalone token in the line. The common case is a line whose entire body is a single JSON object. The parser handles nested objects, arrays, and most primitive types.
• | logfmt — extracts key=value pairs in the logfmt format (key1=value1 key2="value with spaces"). The format is popular in systemd, Hashicorp tools, and the Prometheus ecosystem.
• | regexp "<expression>" — runs RE2 against the line and extracts named capture groups as fields. Use this when the format is neither JSON nor logfmt, or when only a few fields matter and a full JSON parse is overkill.

After a parser, the extracted fields are available to the rest of the query as if they were labels — | level="error" after | json works because level is a parsed field.

# JSON line, parse everything.
{service="checkout"} | json | level="error"

# JSON line, parse and unwrap a numeric field for aggregation.
{service="checkout"} | json | unwrap latency_ms | __error__="" | sum by (service)

# logfmt line, parse and filter.
{service="nginx"} | logfmt | status="500"

# Free-form line, regex-extract the two fields that matter.
{service="legacy-app"} | regexp "user=(?P<user>\\S+) action=(?P<action>\\S+)" | action="login"

Why a sysadmin cares

The parser is what turns a wall of text into structured data that the query engine can pivot on. Without it, every filter is a substring match against the entire line, every aggregation has to be a regex over the body, and every Grafana panel that wants to group by something other than a stream label is doing unnecessary work.

• Aggregation. sum by (status) (...) requires a numeric field. The parser extracts it. Without | json | unwrap, the field is a string the aggregator cannot sum.
• Correlation. A parsed field becomes the join key between logs and metrics. error_count by (endpoint) matches the Prometheus metric label endpoint and lets the on-call engineer move between signals without translating.
• Readability. A Grafana panel that shows columns — level, request_id, latency_ms — is readable. A Grafana panel that shows a wall of JSON is not. The parser is what enables the columns.

How it works — the mental model

{service="checkout"}     <- selector (index lookup)
| json                   <- parser (extracts fields from each line)
| level="error"          <- filter on parsed field
| line_format "{{.msg}}"  <- format the output

The parser is a stage in the pipeline. It runs after every preceding filter and before every following operation. Lines that the upstream filters rejected never reach the parser; lines that fail to parse are tagged with the synthetic field __error__ so the query can either drop them or surface them.

log line (raw)
   v
filter chain (line filters)
   v
parser (json / logfmt / regexp)
   |- success: fields are extracted, attached to the entry
   |- failure: __error__ field is set with the reason
   v
downstream filter / aggregator / formatter

A parsed field is not a stream label. It is attached to the log entry in memory after the parser runs. It can be used as a filter (| level="error"), as an aggregation key (sum by (level)), and as an unwrap target (| unwrap latency_ms). It cannot be used in the stream selector (\{level="error"\} is wrong; \{...\} | json | level="error" is right).

How to configure it

The three parsers, with realistic examples for each.

# JSON. Every top-level field becomes a parsed field.
{service="checkout"} | json
# {service="checkout"} | json | level="error"
# {service="checkout"} | json | unwrap latency_ms | sum by (service)  (bytes)

# JSON with a specific prefix. Useful when the JSON is embedded.
{service="legacy"} | json --strict  | level="error"

# logfmt. Key=value pairs.
{service="nginx"} | logfmt
# {service="nginx"} | logfmt | status="500"

# Regex. Named capture groups.
{service="legacy-app"} | regexp "user=(?P<user>\\S+) action=(?P<action>\\S+)"
# {service="legacy-app"} | regexp "..." | action="login"

# Combined: parse, then filter on parsed field, then aggregate.
{service="checkout"}
| json
| level="error"
| unwrap latency_ms  (bytes)
| sum by (endpoint)

A pipeline that should be doing the parse at ingest, not in the query, looks like this in Grafana Alloy:

loki.process "checkout" {
  stage.json {
    expressions = { level = "level", endpoint = "endpoint" }
  }
  forward_to = [loki.write.default.receiver]
}

At ingest, the parser runs once per line; in the query, it runs every time the query is evaluated. The ingest path is the right place for any field that the team always wants to filter on.

How to validate it

# 1. The parser extracts the expected fields.
logcli --addr=http://loki:3100 query --since=5m --limit=1 \
  '{service="checkout"} | json' | jq .
# {
#   "timestamp": "2026-08-14T14:02:11Z",
#   "line": "{\"ts\":\"2026-08-14T14:02:11Z\",\"level\":\"error\",\"msg\":\"payment_intent_failed\",\"latency_ms\":1240}",
#   "level": "error",
#   "msg": "payment_intent_failed",
#   "latency_ms": 1240
# }

# 2. The parsed field is filterable.
logcli --addr=http://loki:3100 query --since=5m --limit=1 \
  '{service="checkout"} | json | level="error"'
# (returns the same line)

# 3. Parse failures are surfaced.
logcli --addr=http://loki:3100 query --since=5m --limit=5 \
  '{service="checkout"} | json | __error__!=""'
# {service="checkout"} 2026-08-14T14:02:11Z ... log="malformed line: ..."
# (a non-zero count is a pipeline bug)

# 4. The unwrap path produces a numeric series.
logcli --addr=http://loki:3100 instant --since=1h --output=stats \
  '{service="checkout"} | json | unwrap latency_ms | sum by (endpoint)'
# {endpoint="/checkout"} 1240
# {endpoint="/cart"} 312

# 5. The raw-form query would be worse.
logcli --addr=http://loki:3100 instant --since=1h \
  '{service="checkout"} |~ "latency_ms=\\d+" | line_format "{{.latency_ms}}"'
# (a regex with no anchor against the whole line; slow and fragile)

The single most useful validation is to inspect one parsed line end-to-end with jq. If the fields you expected are not there, either the application is not emitting them or the parser is misconfigured.

How it can fail

Six recurring failure modes. Each maps to an observable symptom.

1. The parser runs before the filter. \{...\} | level="error" | json tries to filter on a parsed field before the parser has run. Symptom: Loki returns zero lines or an “unknown label” error; the fix is to swap the chain order.
2. The line is not pure JSON. | json parses a line that starts with a timestamp and ends with a JSON object. The parser is happy with embedded JSON. A line where the JSON is truncated or contains a string with an unescaped quote fails the parse. Symptom: the __error__!="" query returns a non-zero count.
3. Nested fields look like flat ones. A line with a nested object {"http":{"status":500}} produces http as a parsed field, not http.status. Symptom: | http_status="500" returns zero lines; the right filter is | http_status=500 after | json | line_format rewrites the field, or after ingest rewrites the nested shape.
4. The regex parser’s named groups are wrong. A capture group named (?P<user>...) produces a parsed field user. A typo ((?P<urser>...)) produces a field the query never references. Symptom: the parsed field is absent; the query returns zero lines.
5. A parsed field is used in a stream selector. \{level= "error"\} | json puts level in the selector, but level is a parsed field, not a stream label. Symptom: Loki returns “no streams match”.
6. Parsing every line at query time is the hot path. A dashboard with ten panels, each parsing JSON against the whole chunk, runs the parser ten times against the same data. Symptom: the querier is CPU-bound; the dashboard times out.

How to troubleshoot it

The diagnostic order for “the parser extracted the wrong thing” or “the filter on a parsed field returns nothing”:

1. Inspect the raw line. logcli query --limit=1 '{...}' without the parser. See what is actually in the line.
2. Inspect the parsed line. logcli query --limit=1 '\{...\} | json' | jq .. See what fields the parser produced.
3. Look for __error__. \{...\} | json | __error__!="" — a non-zero count means the parser is rejecting lines.
4. Confirm the chain order. Parsed-field filters must come after the parser. Label-like filters (level="error" as a substring) come before. The order matters.
5. Check the field name. A typo in the field name is the most common cause of “the parsed field is empty”. level vs severity is the usual culprit when an application’s naming convention drifts.
6. Time-box the search. A 24-hour query is more useful than a 5-minute one when the field has just been introduced.

Security implications

• Regex parser injection. A pipeline that builds the regex from a user-controlled source — a configuration parameter, a request field — exposes the query to malformed patterns. RE2 is linear but a user-crafted expression can still match unexpectedly. Build regexes server-side from a controlled vocabulary.
• Sensitive fields parsed into queries. A parsed field containing a credential (api_key, token) flows into the query result set. The parser is doing its job; the application is the problem. The right place to scrub is the pipeline.
• Multi-tenant parse disclosure. Parsed fields are visible in the same way as labels — to anyone authorised to query the stream. RBAC is the authoritative boundary; the parser is not a security control.

Performance implications

• Cost per line. Parsing is O(n) in the line length with a constant proportional to the format. JSON is the most expensive because the parser walks the whole object; logfmt is cheaper; regex is cheapest when the expression is anchored and short.
• Where the parse runs. The same parser runs once per line per query. A dashboard with ten panels parses the same chunk ten times. Move the parse to the pipeline and the cost is paid once at ingest.
• Parser placement in the chain. Place the parser after the most selective filter. {...} |= "ERROR" | json | level="error" parses only the lines that match the substring filter. The reverse order parses every line.
• Parsed field as filter key. A filter on a parsed field is not indexed. Every chunk the selector opened is decompressed and every line that survived the line filter is parsed and tested. The right way to make this fast is to promote the field to a stream label.

Production guidance

• Parse at ingest for every field the team always wants to filter on. The pipeline is the right place for the canonical fields (level, method, status, env).
• Use | json for JSON lines, | logfmt for key=value lines, and | regexp only when the format is neither. The specialised parsers are faster and more forgiving than a hand-rolled regex.
• Place the parser after the most selective filter. The filter prunes the line set; the parser runs against what is left.
• Audit parse failures with __error__!="". A non-zero count is a pipeline bug or a malformed-line problem.

Verification

You should now be able to answer:

• Why must a parsed-field filter come after the parser in the chain?
• What is the cost of running | json on every line at query time versus once at ingest?
• How do you surface parse failures in a LogQL query?
• When is | regexp the right tool, and when is | json the better answer?

Quiz