State table exhaustion symptoms — recognising the failures before the user does

The state table exhaustion lesson describes what happens at the limit. This lesson covers what the operator sees when it happens — the symptoms, the log signatures, the patterns that distinguish exhaustion from other failure modes, and the triage steps.

The signature symptom

The signature symptom of state table exhaustion is asymmetric failure: new connections fail while existing connections continue to work. A user with an open SSH session stays connected; a new SSH attempt fails. A browser with open tabs continues to load images from those tabs; opening a new tab to a new host fails. A mail client with an open IMAP connection keeps receiving mail; opening a new connection to download more fails.

The asymmetry is the clue. Rule mistakes affect all matching traffic; state table exhaustion affects only new flows. Link saturation affects everything. CPU saturation affects everything. The operator who sees asymmetric failure should suspect the state table first.

Reading the log

When PF drops a packet because it cannot allocate a state, the drop reason is memory — the same reason PF uses for any failed allocation. The OPNsense filter log shows:

<134>filterlog: 5,1671433603,,igb0,match,block,out,4,0,,64,0,0,0,DF,6,tcp,52,...

The fields to focus on:

• Rule number — the rule that matched. If the rule is a stateful pass rule, the packet should have created state; if it didn’t, the limit was hit.
• Action — block despite a pass rule means the state creation failed.

The OPNsense GUI shows the log entry with the rule’s label and the action; the operator should see block for traffic that the rule set normally permits.

Confirming with pfctl

The pfctl -s info output shows the live counters. The relevant ones:

$ pfctl -sm | grep '^states'; pfctl -si | grep -E 'current entries|memory'

states        hard limit   409600
current entries                   409600
memory                              1423            0.5/s

The confirmation is the pair: current entries at the states hard limit from pfctl -sm, and a memory counter that is climbing rather than merely non-zero. Two neighbouring counters mean something else — state-limit counts packets refused by a rule’s own max-states, and src-limit counts packets refused by max-src-states, max-src-nodes or max-src-conn.

Distinguishing from other failure modes

The triage table:

The last three rows are the most commonly confused. The operator who sees “new SSH fails, existing SSH works” and immediately checks the state table finds confirmation in 30 seconds with pfctl -s info. The operator who assumes it’s a rule mistake will spend an hour reading rules.

Common causes of exhaustion

State table exhaustion is usually caused by one of:

• Limit too small for the workload. A 1,000-user estate on a 2 GB appliance, where the RAM-derived default is 204,800 entries. Fix: increase the limit (covered in the previous lesson).
• Connection storm. A burst of new connections — typically from a misconfigured client, a malware infection, or a DDoS attack — fills the table quickly. Fix: identify the source via pfctl -s state | head, rate-limit the offending traffic.
• Long-lived states. A small number of long-lived flows (e.g. a database connection pool) keep entries occupied. Fix: tune timeouts (covered in lesson 62) or identify the long-lived flows.
• State table fragmentation. PF does not compact the table; over time, expired entries leave holes. The table may report occupancy below the limit but still have fragmentation issues. Less common; usually a non-issue.

The triage steps

When state table exhaustion is suspected:

1. Confirm the symptom. Check pfctl -sm for the states hard limit and pfctl -si for current entries — are they meeting? Is the memory counter climbing?
2. Identify the cause. Look at the log — what source IPs are the failed connections coming from? Is one IP generating most of the inserts? Run pfctl -s state | awk '{print $3}' | sort | uniq -c | sort -rn | head to see the top talkers.
3. Take immediate action. If exhaustion is acute (drops > 1/s), the quickest fix is to increase the table size temporarily. The operator should not wait for a planned change window.
4. Plan the long-term fix. Raise the table size persistently; identify the root cause (workload change, misconfigured client, attack); apply rate-limiting if appropriate.

$ pfctl -s state | awk '{print $3}' | sort | uniq -c | sort -rn | head -5

  1243 192.0.2.50
 432 192.0.2.51
 187 192.0.2.52
 156 203.0.113.10
  98 203.0.113.20

The connection storm scenario

A connection storm — many new connections per second from one source or to one destination — fills the table quickly. Symptoms:

• current entries rises rapidly over minutes.
• inserts rate is unusually high (e.g. > 1000/s on a small firewall).
• memory drops appear once the ceiling is reached. If the estate uses per-source limits (max-src-states, max-src-conn) the src-limit counter rises first, and it rises before the global table fills — which is the point of setting those limits.
• The top talker analysis shows one source with thousands of states.

The most common causes:

• Malware on a LAN host generating outbound connections (C2 beaconing, port scanning).
• Misconfigured web client retrying failed connections in a tight loop.
• DDoS — external traffic aimed at the firewall.
• Legitimate workload spike — a release event, a marketing campaign, a popular application.

The fix depends on the cause: rate-limit the offending traffic; block the source if malicious; tune the application if misconfigured.

Mitigations

The mitigations the operator can apply:

• Increase the table size. Direct fix; covered in lesson 206.
• Tune state timeouts. Shorter timeouts reduce average lifetime; covered in lesson 62.
• Apply per-rule max-src-states. Limits how many states a single source IP can have. Catches runaway clients without affecting the rest of the traffic.
• Apply per-rule max-src-conn-rate. Limits the rate at which a source IP can open new connections. Catches connection storms.
• Identify and stop the source. For malware or misconfigured clients, disconnect the host.

Summary

• The signature symptom of state table exhaustion is asymmetric failure: new connections fail while existing connections work.
• The log signature is block actions on rules that should pass.
• Confirmation: pfctl -sm gives the states hard limit, pfctl -si shows current entries at it and the memory counter climbing.
• Common causes: default size too small, connection storm, long-lived states, fragmentation.
• Triage steps: confirm symptom, identify cause (top talkers), take immediate action, plan long-term fix.
• Mitigations: raise table size, tune timeouts, apply per-source limits, identify and stop the source.