State creation and tracking in PF · OPNsense

PF state is the firewall’s working memory. Every flow that traverses the firewall has a state entry; the entry is what makes return traffic permitted, what tells PF whether a packet is part of an existing connection, and what tells the operator what the firewall actually did. This lesson covers how state is created for each protocol, how long it lasts, and how to read the state table for diagnostics.

What PF records in a state entry

A PF state entry is more than a five-tuple. It records:

The five-tuple: source IP, source port, destination IP, destination port, protocol.
The direction the first packet took (interface, gateway, NAT pre/post addresses).
The packet and byte counters for each direction.
The TCP flags observed so far, for TCP flows.
A timeout controlling when the entry is forgotten.

The state entry is created when a packet that matches a keep state, modulate state, or synproxy state rule is permitted. The entry is created on the first packet that matches (typically the SYN for TCP, the first datagram for UDP).

Read-only / Safepfctl -s state -v— Three state entries — three flows. `-v` adds a metadata line under each: age, remaining lifetime, per-direction packet and byte counters printed as out:in, and the rule that created the state. The first TCP flow is 3 minutes 21 seconds old with about 23 hours 57 minutes left of the 24-hour ESTABLISHED lifetime. The UDP flow has 15 seconds left of its 60-second lifetime. The address in parentheses is the translated address.

$ pfctl -s state -v

all tcp 203.0.113.50:443 <- 192.0.2.50:51820 (203.0.113.1:51820)   ESTABLISHED:ESTABLISHED
 age 00:03:21, expires in 23:56:39, 1421:1284 pkts, 142384:1820392 bytes, rule 15
all tcp 203.0.113.50:443 <- 192.0.2.51:51821 (203.0.113.1:51821)   ESTABLISHED:ESTABLISHED
 age 00:00:12, expires in 23:59:48, 12:9 pkts, 1840:11284 bytes, rule 15
all udp 8.8.8.8:53 <- 192.0.2.50:54210 (203.0.113.1:54210)   MULTIPLE:MULTIPLE
 age 00:00:45, expires in 00:00:15, 2:2 pkts, 148:312 bytes, rule 15

Illustrative output

The address in parentheses on the summary line is the translated address, shown next to the address it replaces. For traffic that has been NAT-translated, PF records the original (pre-NAT) tuple for state matching and the translated (post-NAT) tuple for forwarding — both in the same entry. When NAT is involved, the state table is the only place the operator can see both tuples.

State creation per protocol

TCP

TCP state is the most sophisticated. PF tracks the handshake:

SYN only — first packet of a new connection. PF creates a state in SYN_SENT:SYN_SENT (client side: SYN_SENT, server side: SYN_SENT). Timeout: 30 seconds.
SYN-ACK — second packet. PF updates state to ESTABLISHED:ESTABLISHED. Timeout: 24 hours.
ACK — third packet, completes the handshake. State remains ESTABLISHED:ESTABLISHED for the duration of the connection.
FIN or RST — connection closes. PF removes the state immediately.
Idle timeout — 24 hours without traffic removes the state.

PF also recognises half-closed connections (one side sent FIN, the other has not). Half-closed states live until the second FIN or until the timeout.

UDP

UDP has no handshake. PF creates a state on the first datagram in a single direction; the state permits return packets that match the reversed tuple. PF’s state for UDP is SINGLE:NO_TRAFFIC until the first return packet, then SINGLE:MULTIPLE until the timeout.

UDP state timeout is 60 seconds by default. Refreshed by any matching packet in either direction. Not refreshed by application-level keepalives (because the firewall does not parse the application layer).

For long-lived UDP flows (VoIP calls, video streaming, IPsec NAT-T keepalives), the default 60-second timeout is too short. Production tuning requires either raising the global UDP timeout or applying a per-rule timeout override.

ICMP

ICMP state is the simplest. The first packet (echo request, destination unreachable, time exceeded, etc.) creates a state; the matching reply uses the state. Default timeout: 20 seconds.

ICMP states are short because ICMP is meant to be transactional. For Path MTU Discovery, the ICMP type 3 code 4 state must live as long as the TCP connection that depends on it; the default 20-second timeout is fine because PMTUD messages only need to flow during the initial handshake.

Other protocols

PF tracks state for any IP protocol the operator permits, not just TCP/UDP/ICMP. GRE, ESP (IPsec), OSPF, and any other IP protocol can have state. The default behaviour for non-TCP/UDP state is conservative:

One packet creates state.
Matching packets in either direction match the state.
Timeout is 60 seconds for any non-TCP/UDP flow unless overridden.

For routing protocols (OSPF, BGP), the recommended pattern is to use stateless rules (no state) because the routing protocol itself maintains its own session state.

Reading the state table for diagnostics

The state table is the operator’s primary evidence for “what did the firewall do?” Three patterns to recognise:

Many short-lived states for the same source. A host making rapid connections (a port scanner, a buggy application) is creating and tearing down states quickly. Look at age to see how long each state has lived.
States with very high packet or byte counters. A flow that has transferred gigabytes. Useful for verifying that bulk transfer (backup, replication) is actually flowing.
States with the same five-tuple but different interfaces. Should not happen in normal operation — but if it does, asymmetric routing or NAT rebinding is the cause.

The most useful diagnostic commands:

pfctl -s state — table summary, one line per state. One flow is one entry; the entry matches both directions. pfctl -ss is an abbreviation of the same command.
pfctl -s state -v — adds a second line per state with age, remaining lifetime, packet and byte counters, and the rule number.
pfctl -s state | grep <address> — there is no filter expression for the state table, so narrowing is grep and awk. -i <interface> restricts the walk to one interface.
pfctl -si — show state-table statistics (entries, searches, insertions, removals) and the drop counters.
pfctl -sm — show the pf runtime limits, including the maximum number of states.

Read-only / Safepfctl -si— State-table statistics. 'current entries' (4382) is the live count; searches, inserts and removals are cumulative with a rate. The Counters block is one line per drop reason. There is no limit or occupancy field here — the configured maximum is a pf runtime limit, printed by pfctl -sm. A rising 'memory' counter is what a full state table looks like from this output.

$ pfctl -si

Status: Enabled for 14 days 03:27:18            Debug: Urgent

State Table                          Total             Rate
current entries                     4382
searches                        12847123           10.6/s
inserts                           189324            0.2/s
removals                          184942            0.2/s
Counters
match                            8452394            7.0/s
bad-offset                             0            0.0/s
fragment                              12            0.0/s
short                                  0            0.0/s
normalize                              3            0.0/s
memory                                 0            0.0/s
bad-timestamp                          0            0.0/s
congestion                             0            0.0/s
ip-option                              0            0.0/s
proto-cksum                            0            0.0/s
state-mismatch                        41            0.0/s
state-insert                           0            0.0/s
state-limit                            0            0.0/s
src-limit                              0            0.0/s
synproxy                               0            0.0/s
map-failed                             0            0.0/s

Illustrative output

State-table exhaustion is silent until it is not

The state table has a maximum size. It is a pf runtime limit (set limit states in the compiled ruleset), set on OPNsense under Firewall → Settings → Advanced → Firewall Maximum States. Left blank, OPNsense computes it from installed RAM — roughly 100 states per MiB of physical memory, which is the 10% of system memory the project reserves for the table. Read the value in force with pfctl -sm.

When the table is full, PF cannot allocate a new state, so the first packet of a new flow is dropped. Flows that already have state are unaffected.

The first symptom is usually “VPN tunnels keep dropping” or “some users can connect but most cannot”. The pfctl -si output will show the memory counter climbing while current entries sits at the states hard limit from pfctl -sm.

The fix is to size the state table for the firewall’s actual load. Each concurrent connection is one entry, so the target is the peak concurrent flow count plus headroom: a user with 50-200 open connections needs 50-200 entries, and a 1,000 user estate is therefore in the 50,000-200,000 range before headroom.

Tuning timeouts

Three places to change state timeouts:

Global default — System → Firewall → Advanced → Optimisation. Profiles like normal, high-latency, satellite, aggressive set global timeouts. normal is the default for most estates.
Per-rule override — edit a firewall rule and set “State timeout” in seconds. The rule’s timeout overrides the global default for matching flows.
The ruleset — timeouts are set timeout statements in the compiled ruleset. pfctl -st prints the values in force. The OPNsense GUI generates them; editing the generated ruleset by hand does not survive an apply.

Tuning is case-by-case. VoIP needs longer UDP timeouts. Video streaming needs longer UDP timeouts. SSH sessions are long-lived TCP — the 24-hour default is fine. RPC services with idle connections need longer timeouts than the default.

Summary

PF state records the five-tuple, direction, NAT translations, flags, counters, and a timeout.
TCP state is the most sophisticated, with handshake tracking and 24-hour ESTABLISHED timeout.
UDP state is single-packet initiated, 60-second default timeout, refreshed by any matching packet.
ICMP state is short-lived (20 seconds).
State-table exhaustion is a real production failure mode. Size the state table to expected concurrent flow count.
Tune timeouts per-rule for protocols whose natural pacing does not match the defaults.