State exhaustion and DoS — sizing, symptoms and recovery

The PF state table has a maximum size. When the table fills, PF cannot allocate new states; new flows are dropped. In a busy production environment, the state table can fill under legitimate load. In a denial-of-service scenario, the state table is the target — the attacker floods the firewall with flows that exhaust the table and prevent legitimate traffic from establishing.

This lesson covers state-table sizing, the symptoms of exhaustion, tuning for production workloads, and the DoS scenario.

State-table sizing basics

The maximum number of state entries is a pf runtime limit, not a sysctl. It is compiled into the ruleset as set limit states <n> and it is read back with pfctl -sm. There is no net.pf.states_max or net.inet.pf.maxstates to set; the pf(4) sysctl tree does not expose the state limit.

On OPNsense the value comes from Firewall → Settings → Advanced → Firewall Maximum States. Left blank — which is the shipped state — OPNsense derives it from installed RAM: the generated ruleset gets one hundred states for every MiB of physical memory, which is the project’s rule of reserving about 10% of system memory for the state table. src-nodes is set to the same number.

Do not memorise a number. Read the value in force:

$ pfctl -sm

states        hard limit   409600
src-nodes     hard limit   409600
frags         hard limit     5000
table-entries hard limit   200000

$ pfctl -si

Status: Enabled for 14 days 03:27:18            Debug: Urgent

State Table                          Total             Rate
current entries                     4382
searches                        12847123           10.6/s
inserts                           189324            0.2/s
removals                          184942            0.2/s
Counters
match                            8452394            7.0/s
bad-offset                             0            0.0/s
fragment                              12            0.0/s
short                                  0            0.0/s
normalize                              3            0.0/s
memory                                 0            0.0/s
bad-timestamp                          0            0.0/s
congestion                             0            0.0/s
ip-option                              0            0.0/s
proto-cksum                            0            0.0/s
state-mismatch                        41            0.0/s
state-insert                           0            0.0/s
state-limit                            0            0.0/s
src-limit                              0            0.0/s
synproxy                               0            0.0/s
map-failed                             0            0.0/s

How to size the state table

Three approaches:

Rules of thumb

Each concurrent flow is one state entry. Translation does not add an entry — the same entry carries the translation — so the arithmetic is a straight count of concurrent flows.

• 50-200 states per active desktop user. A user browsing with multiple tabs, a mail client and a few background services typically holds 50-200 connections open at once. Each one is a single state.
• A 1,000-user estate is therefore 50,000-200,000 states at peak, before headroom. With a 3x factor for bursts and connection storms, 150,000-600,000 is the sizing target — not millions.
• Servers skew the count. A handful of hosts with large connection pools, or a NAT-heavy multi-WAN edge with many short-lived flows, can hold more states than hundreds of desktop users. Size from measurement, not from the user count, wherever a measurement exists.

Empirical sizing

Start from the derived default and monitor the state count over time:

pfctl -si
pfctl -sm

Look at current entries over a week and compare it to the states hard limit. If the peak is 250,000 against a 409,600 limit, the sizing is adequate with headroom. If the peak reaches 80% of the limit, raise it.

Memory-based sizing

Each state entry is roughly 256-300 bytes on a 64-bit system. 1 million states is roughly 300 MB. For a firewall with 4 GB of RAM, allocating 1 million states uses about 7% of memory.

The formula:

memory_for_states = max_states * 300 bytes

The symptoms of state exhaustion

When the state table fills:

• New flows fail. A new TCP SYN cannot create state; PF drops the SYN. The client sees a connection timeout.
• Existing flows continue. State entries that already exist are not affected. Only new state allocation fails.
• The memory counter in pfctl -si climbs. When the state limit is reached, the state allocation fails and PF drops the packet with reason memory. A memory counter that is rising, rather than merely non-zero from some historical event, is the signal.
• current entries is at or near the states hard limit from pfctl -sm. The table is saturated.

Two nearby counters mean something else and are commonly misread. state-limit counts packets refused by a rule’s own max-states option, not by the global limit. src-limit counts packets refused by max-src-states, max-src-nodes or max-src-conn. Neither is the global-exhaustion signal.

In a production environment, the symptoms may be intermittent: the table fills during peak hours and clears during off-peak.

Tuning the state table

Three parameters to tune:

Firewall Maximum States (GUI)

The primary knob. Firewall → Settings → Advanced → Firewall Maximum States. Set to the target size, or leave it blank to keep the RAM-derived default.

The change requires a config apply, which regenerates the ruleset with the new set limit states value. Confirm with pfctl -sm afterwards.

State timeouts

Shorter timeouts reduce state-table pressure at the cost of more frequent state refreshes. The Firewall → Settings → Advanced → Firewall Optimization profiles adjust the default timeouts:

• normal (default). TCP ESTABLISHED 24 hours, UDP 60 seconds, ICMP 20 seconds.
• high-latency. Longer timeouts for satellite and high-latency links.
• aggressive. Shorter timeouts for high-churn environments.

Per-rule state timeout overrides

For protocols with application-level pacing (SIP, IPsec NAT-T, streaming), set per-rule timeout overrides.

$ pfctl -sm; sysctl net.pf.states_hashsize

states        hard limit  1000000
src-nodes     hard limit  1000000
frags         hard limit     5000
table-entries hard limit   200000
net.pf.states_hashsize: 32768

The DoS scenario

State-exhaustion denial of service is a real attack vector. The attacker creates many short-lived flows that consume state entries.

The patterns:

• SYN flood with state allocation. The attacker sends SYNs from spoofed source IPs. Each SYN, if matched by a permitting rule, creates a half-open state.
• HTTP flood. The attacker sends many HTTP requests, each creating a state.
• UDP flood. The attacker sends many UDP packets from spoofed sources.
• Slowloris-style. The attacker opens many TCP connections and sends data very slowly.

The defence:

• SYN-proxy. OPNsense supports synproxy state, which handles the TCP handshake on the firewall’s behalf.
• State-timeout tuning. Shorter timeouts for ephemeral flows.
• Per-source state limits. PF supports max-src-states and max-src-nodes.
• Rate limiting. OPNsense supports traffic shaping and rate limiting.

Per-source state limits

PF supports limiting state per source IP. The syntax:

pass in on igb0 proto tcp from any to any port = 80 \
     keep state (max-src-states 100, max-src-nodes 100)

This rule permits HTTP traffic but limits each source IP to 100 simultaneous states.

Monitoring state-table health

Three signals tell the operator that the state table is healthy:

• current entries / states hard limit. Should stay below 0.8 during normal operation. The numerator comes from pfctl -si, the denominator from pfctl -sm.
• the memory counter. Should be flat. A climbing memory counter means PF is failing to allocate states.
• insert / remove rate. Should track closely.

The monitoring pattern: poll pfctl -si every minute and pfctl -sm on each config apply, alert on:

• current entries / states_hard_limit > 0.8 (warning).
• current entries / states_hard_limit > 0.95 (critical).
• increase(memory) > 0 over the poll interval (any increase is a finding).

$ pfctl -sm | grep states; pfctl -si | grep -E 'current entries|memory'

states        hard limit   409600
current entries                   409600
memory                              1423            0.5/s

Recovery from state exhaustion

Three responses, in order:

Raise the limit

If the cause is legitimate load that has grown beyond the initial sizing, raise the limit. The change is immediate; new flows allocate state with the new limit.

Identify and limit the offender

If the cause is a misbehaving source or application, identify it with pfctl -s state | awk '{print $3}' | sort | uniq -c | sort -rn | head. The top sources are candidates.

Apply SYN-proxy or no-state rules

For inbound services that do not need stateful tracking, use synproxy or no state to reduce state-table pressure.

Summary

• The state table maximum is a pf runtime limit (set limit states), read with pfctl -sm and set under Firewall → Settings → Advanced → Firewall Maximum States. Left blank, OPNsense derives it from RAM at about 100 states per MiB — roughly 409,600 on a 4 GB appliance.
• Each concurrent flow is one state entry, and each entry is roughly 300 bytes. Memory is the constraint.
• State exhaustion causes new flows to fail. Existing flows are unaffected.
• Symptoms: current entries near the states hard limit, the memory counter climbing, intermittent connection failures. state-limit and src-limit are per-rule and per-source counters, not the global signal.
• Tune Firewall Maximum States for capacity, the Firewall Optimization profile for default timeouts, per-rule timeout overrides for specific protocols.
• DoS attacks target the state table. Defence in depth: SYN-proxy, per-source limits, rate limiting, upstream DDoS mitigation.
• Monitor the current entries to states hard limit ratio and the memory counter; alert at 80% and 95% occupancy and on any increase in memory.