LogQL Line Filters

The selector has narrowed the universe to the streams you care about. Now the question becomes “which lines inside those streams?”. That is what line filters do, and the answer to that question decides whether the query reads a kilobyte or a gigabyte.

What a line filter is

A line filter is the part of a LogQL query that runs after the stream selector and operates on the contents of each log line, not on its labels. The pipe character | separates the selector from the filter chain.

Four operators, mirroring the selector’s four but at line scope:

• |= "text" — substring match. The line contains text as a contiguous substring. Cheap and the most common filter.
• != "text" — negated substring. The line does not contain text. Implemented with a Bloom filter inside the chunk, so the common case is fast.
• |~ "regex" — regex match. The line matches the RE2 expression. The expression is evaluated against the line.
• !~ "regex" — negated regex. The line does not match the expression. Always more expensive than the positive form.

{service="checkout"} |= "payment"
{service="checkout"} != "healthcheck"
{service="checkout"} |~ "level=(error|warn)"
{service="checkout"} !~ "trace_id=00000000-0000-0000-0000-"

Line filters are ANDed across the chain. Multiple pipes form a conjunction: each filter must accept the line for it to be returned. The pipeline is {selector} filter1 filter2 parser ....

Why a sysadmin cares

The selector is bounded by the inverted index. The line filter is bounded by the bytes scanned inside the chunk. The chunk size is typically 1-2 MB compressed, and Loki opens every chunk that overlaps the time range.

• Cost. A selector that matches ten streams across a one-hour window might still read hundreds of megabytes of compressed chunks. The line filter decides whether the query reads those megabytes or only the matching lines.
• Latency. Substring filters are O(n) over the chunk in the common case. Regex filters are O(n) too, but with a larger constant. The constant matters at gigabyte scale.
• Diagnostic clarity. A selector that returns zero streams is a label problem. A selector that returns streams but no matching lines is a filter problem. The two failures look similar from the dashboard; the remediation is different.

How it works — the mental model

{selector}  filter1  filter2  parser  aggregator
   |           |        |        |         |
   v           v        v        v         v
stream    line    line    parse    reduce
index     filter  filter  field    over a
lookup                          range

Selector: O(1) per stream on the inverted index.
Line filter: O(n) over each chunk the selector opened.
Parser: O(n) over each line the filters accepted.
Aggregator: O(n) over each parsed line.

The selector narrows the universe. The line filter scans what is left. The parser extracts structure. The aggregator reduces the structured values to a series. Every step is bounded by the previous step’s output, but only the selector is bounded by the index.

How to configure it

# Substring: the default. Use when the text is a stable token.
{service="checkout"} |= "payment_intent_failed"

# Negated substring: exclude a known noisy line.
{service="checkout"} != "GET /healthz"

# Regex alternation: match several equivalent tokens.
{service="checkout"} |~ "level=(error|warn)"

# Anchored regex: bind the match to the start of the line.
{service="checkout"} |~ "^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.*ERROR"

# Combined filters: conjunction.
{service="checkout"} |= "payment" |= "failed" != "retry"

Three rules of thumb:

• Prefer |= and != over |~ and !~ unless you need a pattern. The Bloom filter shortcut is free; the regex engine is not.
• Anchor regexes when the match position is known. The Loki docs recommend ^pattern for any filter that is meant to match the structured prefix of a JSON or logfmt line.
• Place the most selective filter first. Loki evaluates filters in chain order; a |= "ERROR" filter that matches one in a thousand lines should run before a != "healthcheck" filter that matches every line.

How to validate it

# 1. The selector returns streams; the filter should narrow to lines.
logcli --addr=http://loki:3100 series --since=15m '{service="checkout"}' | wc -l
# 12

# 2. Inspect the line filter with --limit and a small window.
logcli --addr=http://loki:3100 query --since=5m --limit=5 \
  '{service="checkout"} |= "payment_intent_failed"'
# {service="checkout"} 2026-08-14T14:02:11Z error msg="payment_intent_failed" ...
# {service="checkout"} 2026-08-14T14:02:14Z error msg="payment_intent_failed" ...

# 3. Time the filter at production scale.
logcli --addr=http://loki:3100 instant --since=1h \
  '{service="checkout"} |= "payment_intent_failed"' --stats
# status: success
# summary: executed in 312.882ms
# bytesProcessed: 47.2MB

# 4. Compare against the unfiltered selector.
logcli --addr=http://loki:3100 instant --since=1h \
  '{service="checkout"}' --stats
# summary: executed in 487.123ms
# bytesProcessed: 412.8MB

# 5. Confirm the negated form works.
logcli --addr=http://loki:3100 query --since=5m --limit=1 \
  '{service="checkout"} != "GET /healthz"'
# {service="checkout"} 2026-08-14T14:02:11Z error msg="payment_intent_failed" ...

The right comparison is the same query with and without the filter, against the same window, against the same selector. The filter’s value is the difference between bytesProcessed for each. A filter that does not reduce the bytes processed is a filter that is not doing its job.

How it can fail

Six recurring failure modes. Each maps to an observable symptom.

1. The filter matches almost every line. A line filter like |= "2026" matches every line in a year’s worth of logs. The selector did its job; the filter does nothing. Symptom: bytesProcessed is identical with and without the filter.
2. The regex is unanchored. |~ "level=error" scans every byte of every line in every chunk. The same query anchored |~ "level=error.*" is no better. Symptom: the query takes seconds rather than milliseconds; CPU on the querier spikes.
3. The filter is too narrow. A typo or an off-by-one in the pattern returns zero lines. The on-call engineer assumes the service is silent. Symptom: logcli query returns an empty result set; the same selector without the filter returns data.
4. The negation matches everything. != "request_id=" is true for every line that does not contain the literal token. The filter is correct but the result set is the whole chunk. Symptom: bytesProcessed is identical with and without the negation.
5. A user-controlled value is fed into a regex. A pipeline that builds LogQL from a request body or a URL parameter passes a user string into |~. Symptom: under RE2 the linear guarantee holds, but the regex itself can still match unexpectedly or trigger expensive evaluation against every line.
6. The filter runs against parsed fields, not the line. The pipeline extracts level="error" from a JSON line, and the operator writes {service="checkout"} | level="error" — but level is a parsed field, not a stream label, and the line filter does not see it. Symptom: the parser-based filter syntax fails or returns zero lines.

How to troubleshoot it

The diagnostic order for “the right selector returned zero lines”:

1. Confirm the selector returns streams. logcli series '\{...\}' | wc -l. If the count is zero, the problem is the selector, not the filter.
2. Inspect a single line from the matching stream. logcli query --limit=1 '{...}'. See the actual content.
3. Run the filter against a known-good sample. Copy a line into a regex tester and confirm the pattern matches.
4. Strip the filter chain. Remove everything after the selector. If data returns, the problem is downstream.
5. Re-add filters one at a time. Each filter that does not reduce bytesProcessed is not doing its job. Replace it.
6. Check the parser position. A filter that depends on a parsed field has to come after the parser. \{...\} | json | level="error" parses first, then filters. \{...\} | level= "error" does not work.
7. Time-box the search. A 24-hour filter that returns nothing is more useful than a 5-minute one — the 24-hour version tells you whether the pattern is wrong or whether the line is missing entirely.

Security implications

• User input in regex. A regex constructed from a request parameter is a small but real DoS vector under any engine. RE2 is linear, but a user-crafted expression can still match a large number of lines and force the engine to scan every byte of every chunk. Treat label values and request inputs as data, not as code; build regexes server-side from a controlled vocabulary.
• Sensitive data in matched lines. A line filter that matches on a credit-card-shaped pattern returns the entire line, including the secret. The filter is doing its job; the line is the problem. The right place to scrub secrets is the pipeline.
• Result-set disclosure. A line filter that broadens a selector (!= "request_id=my-trace") can leak other users’ data into the result set. RBAC on the query endpoint is the authoritative control; the line filter is not a security boundary.

Performance implications

• Substring match (|=, !=). Bloom-filter-assisted; the common case is a single hash lookup per chunk, no decompression. The rare case (Bloom says “probably”) is a full decompression and scan.
• Regex match (|~, !~). No Bloom filter. Every chunk the selector opened is decompressed and the regex runs against every line. Anchored regexes cut the constant by an order of magnitude for patterns that match the line prefix.
• Negation cost. != is cheap when the Bloom filter says “definitely not present”. !~ is always expensive because every line has to be evaluated.
• Filter ordering. Filters run in chain order. Putting the most selective filter first lets the chain short-circuit earlier. Putting a cheap filter first that almost always matches does not help.

Production guidance

• Every line filter should be paired with a bytesProcessed measurement. If the filter does not reduce the bytes read, the filter is wrong.
• Regex filters must be anchored when the pattern has a known position. |~ "^ERROR" is the difference between a 50 ms query and a 5 s query.
• Filter ordering matters. Place the most selective filter first, then progressively broader filters.
• Parsed-field filters must come after the parser. \{...\} | json | level="error" is right; \{...\} | level="error" is not.

Verification

You should now be able to answer:

• Why is |= cheaper than |~ even when both match the same number of lines?
• Why does an unanchored regex hurt more than an anchored one?
• What is the correct chain order for a parsed-field filter?
• What is the right metric to compare a query with and without a filter?

Quiz