Lab: Inspect, kill, and expire PF state

This lab makes you fluent with the state table — the firewall’s working memory. You will read it the way an operator reads it during an incident, identify specific flows, kill only the flows that need to die, and prove that the kill did not affect flows that should have survived. The discipline on display here (taketh the right state, leave the rest alone) is the same discipline a 03:00 incident response requires.

By the end you will have a feel for what pfctl -s state looks like on a real firewall, how to extract the entries that matter from a table that does not care about your question, and why pfctl -k is the right tool when pfctl -F state is the wrong one.

Objective

By the end of this lab, you can:

Read the PF state table with pfctl -s state, pfctl -s state -v, pfctl -si and pfctl -sm.
Filter the state table by IP, port, interface, and rule label.
Use pfctl -k to kill states involving a specific IP, a specific flow, or a labelled rule.
Distinguish a targeted kill from a bulk flush and pick the right tool for the change.
Verify after a kill that the intended states are gone and the unrelated ones are still there.

Requirements

An OPNsense instance you can reach via SSH and the GUI. Mode B (nested) or Mode C (simulation) is appropriate.
At least one client on the LAN that can generate reproducible traffic (curl to a known endpoint, an SSH session from your workstation, a long-running ping).
A pre-lab rule that permits the traffic you will generate. The default LAN anti-lockout rule plus the standard outbound allows are sufficient.
Permission to run pfctl commands. On OPNsense this is the root shell (option 8 from the console menu, or SSH as root).

Tasks

Task 1: Snapshot the state table before the change

Always record the state before you mutate it. The diff between “before” and “after” is your evidence that the change did what you intended.

pfctl -s state > /tmp/state-before.txt
pfctl -s state -v > /tmp/state-verbose-before.txt
pfctl -si > /tmp/state-stats-before.txt
pfctl -sm > /tmp/state-limits-before.txt
ls -lh /tmp/state-*.txt

Note the counts:

wc -l /tmp/state-before.txt
grep '^State Table' -A 12 /tmp/state-stats-before.txt | head -15

You should see a current entries number and a limit. The limit is the firewall’s configured state-table ceiling; you will compare later runs against this baseline.

Task 2: Generate a reproducible flow

You need a flow you can identify later in the state table. Pick a long-lived, low-traffic one — the SSH session you are using to run these commands is ideal because it is stable and easy to identify by the source IP.

who
# remember your pts/N

Open a second SSH session from a different host (or use tmux/screen on the firewall) and use it for the diagnostic commands. The first session is the “victim” — its SSH state will be the one you kill and then replace.

From the second session, generate another flow:

curl -v --max-time 60 https://example.com/ > /tmp/curl-test.out 2>&1 &
CURL_PID=$!
echo "curl pid: $CURL_PID"

(The curl finishes quickly, but its state persists for the configured TCP idle timeout — minutes by default. That is enough to demonstrate.)

For a long-running flow, open a third session on the LAN and start a tail:

ssh user@lan-host "tail -f /var/log/syslog > /dev/null" &

You now have three flows: your SSH (session 1), the curl from session 2, and the long-running ssh tail from session 3. Each should be visible in the state table.

Task 3: Find the flows in the state table

From session 2 (the diagnostic session), inspect:

# Substitute your own values before running:
# SRC_IP is the address the traffic in Task 2 comes from.
SRC_IP=192.0.2.25

pfctl -s state | grep "$SRC_IP"

You should see one entry per active flow — one for the SSH session, one for the curl flow, one for the long-running tail. A flow is a single state entry that matches both directions, so the count of lines is the count of flows, not twice it.

Get the verbose view of just the SSH state:

# SRC_IP: your workstation's LAN address (Task 3).
SRC_IP=192.0.2.25

pfctl -s state -v | grep -A 1 "$SRC_IP" | head -4

The verbose output adds a metadata line with the age, the remaining lifetime, the per-direction packet and byte counters, and the rule number. The SSH state is ESTABLISHED:ESTABLISHED with an age counted in minutes.

Narrow to TCP only. pfctl has no filter expression for the state table — -f loads a ruleset from a file — so this is an awk job:

# SRC_IP: your workstation's LAN address (Task 3).
SRC_IP=192.0.2.25

pfctl -s state | awk -v ip="$SRC_IP" '$2 == "tcp" && index($0, ip)'

Restrict the walk to one interface with -i:

# Substitute your own values before running:
# LAN_IF is the LAN NIC as ifconfig(8) names it: em1, igb1, vtnet1 ...
LAN_IF=igb1

pfctl -s state -i "$LAN_IF"

Get the aggregate statistics:

pfctl -si

Note the field names: the State Table block has current entries, searches, inserts and removals; the Counters block has match, memory, state-limit, src-limit and the rest of the drop reasons. There is no max, limit or full field. The inserts and removals counters should be moving (non-zero rates) on a firewall with active traffic.

Read the configured limit separately:

pfctl -sm

The states hard limit line is the maximum the table can reach. Occupancy is current entries divided by that number.

Task 4: Targeted kill — single IP

The most common state-killing operation: kill every state involving a specific IP. This is what you do when a host’s connections need to be terminated (IP reassignment, misbehaving host, security incident).

# SRC_IP: your workstation's LAN address (Task 3).
SRC_IP=192.0.2.25

pfctl -k "$SRC_IP"

The output shows how many states were killed. Compare against the state table:

# SRC_IP: your workstation's LAN address (Task 3).
SRC_IP=192.0.2.25

pfctl -s state | grep "$SRC_IP"

The grep should be empty. The states are gone.

But what about the curl flow and the long-running tail that came from the same source IP? They are also gone — pfctl -k kills every state involving the IP, in every direction, on every interface. This is the tool’s power and its danger.

Verify the second behaviour is what you expected:

# SRC_IP: your workstation's LAN address (Task 3).
SRC_IP=192.0.2.25

echo "expected: empty above"
echo "---"
# What if there were other flows? In a real environment, find
# the next-most-recent state for this IP in the saved snapshot:
grep "$SRC_IP" /tmp/state-before.txt | wc -l
echo "states for this IP in the pre-snapshot"

Task 5: Verify the kill did not affect unrelated flows

The whole point of using pfctl -k is that it is targeted. The proof of targeted behaviour is that other flows are unaffected.

Open a fourth session from a different source IP (a second machine on the LAN, or a tunnel). Have it start a long-running flow:

ssh other-user@other-host "sleep 600" &
OTHER_PID=$!

From the diagnostic session, confirm the flow is in the state table:

# Substitute your own values before running:
# OTHER_IP is the fourth session's source address.
OTHER_IP=192.0.2.60

pfctl -s state | grep "$OTHER_IP"

Now kill the first source IP again:

# SRC_IP: your workstation's LAN address (Task 3).
SRC_IP=192.0.2.25

pfctl -k "$SRC_IP"

Note how many states were killed this time. Compare against the state table for the other source IP:

# OTHER_IP: the second machine's LAN address (Task 5).
OTHER_IP=192.0.2.60

pfctl -s state | grep "$OTHER_IP"

The other flow’s state should still be present. The kill targeted only the IP you passed. If the other IP’s flow is gone, you have a bug — pfctl -k is supposed to be scoped to the IP you provided.

Task 6: Targeted kill — IP pair

The IP-pair form of pfctl -k kills only flows that match both addresses. This is the surgical tool for “the flow between these two specific endpoints”.

Re-establish a flow from your first source IP:

curl -v --max-time 30 https://example.com/ > /dev/null 2>&1 &

Now kill only the flow between your source IP and the curl target:

# SRC_IP: your workstation's LAN address (Task 3).
SRC_IP=192.0.2.25

# DST_IP: the address example.com resolved to for the curl above.
DST_IP=93.184.216.34

pfctl -k "$SRC_IP" -k "$DST_IP"

Note the count. Verify:

# SRC_IP: your workstation's LAN address (Task 3).
SRC_IP=192.0.2.25

pfctl -s state | grep "$SRC_IP"

Other flows from your source IP (e.g. the long-running tail started earlier) should still be present. The IP-pair form only killed the flow between the two addresses.

Task 7: Identify a rule’s label and kill by label

If you have a rule with a label, you can kill every state created by that rule. This is the cleanest pattern for “after a rule change, I want the new rule to apply to existing flows”.

In the OPNsense GUI, edit a rule and set its label under Advanced Options. For this lab, take a rule that permits outbound HTTPS from the LAN (or any rule with current traffic) and label it lab_https_out.

Apply the change. Generate a flow that matches the labelled rule:

curl -v --max-time 30 https://example.com/ > /dev/null 2>&1

Confirm the state exists:

pfctl -s state -v | grep -A 1 lab_https_out | head -5

Now kill by label:

pfctl -k label -k lab_https_out

The output shows the count of killed states. Confirm any remaining flows are unaffected:

pfctl -s state | grep lab_https_out
echo "expected: empty"

The label kill targets only the rule’s states. Other rules’ states are untouched.

Task 8: Recognise the bulk-flush boundary

You now have the targeted kill. The bulk flush is the escalation: pfctl -F state removes every state on the firewall.

You will not run this command in this lab. The exercise is the recognition: name three situations where pfctl -F state is the right tool, and three where pfctl -k is the right tool.

Right tool: pfctl -F state

A maintenance window is in effect and a full state reset is part of the planned work.
A firmware upgrade requires a state reset (this is a side-effect of the upgrade, not a separate action).
The state table is corrupted or exhausted and recovery requires a clean slate.

Right tool: pfctl -k

A specific host’s connections need to be terminated (IP reassignment, misbehaving host).
A rule change needs to apply to existing flows and only the affected flows need to be cleared.
A stuck TCP session needs to be terminated without affecting other sessions on the same host.

Write the lists in your post-lab report. The recognition is the discipline; running the right command is the consequence.

Task 9: Snapshot the state table after the change

pfctl -s state > /tmp/state-after.txt
pfctl -si > /tmp/state-stats-after.txt

# Diff the counts
diff <(grep '^State Table' -A 12 /tmp/state-stats-before.txt) \
     <(grep '^State Table' -A 12 /tmp/state-stats-after.txt)

The current entries number may have changed (because you killed some states). The inserts and removals counters should still be moving. The limit and max should be unchanged — those are configuration, not the live state.

Task 10: Restore the labelled rule

If you changed a rule’s label in Task 7, restore it now. Either remove the label in the GUI or revert to the value the rule had before the lab. Apply the change.

pfctl -s rules | grep -i lab_https_out

The grep should be empty (you removed the label) or show the permanent rule text without the lab-only label.

Validation

The state table was snapshotted before and after (files /tmp/state-before.txt and /tmp/state-after.txt exist).
pfctl -k <ip> terminated all states involving the IP, and the count output matched the number of states in the pre-snapshot involving that IP.
After a pfctl -k <ip>, an unrelated flow from a different IP was still present in the state table.
The IP-pair form pfctl -k <src> -k <dst> killed only the flow between the two addresses; other flows from the source IP were still present.
The label-based kill pfctl -k label -k lab_https_out cleaned states generated by the labelled rule; states from other rules were untouched.
No pfctl -F state was run. The post-lab report lists three legitimate use cases for it and three for pfctl -k.
The labelled rule that was edited in Task 7 is restored to its pre-lab state.

Cleanup

The lab is intentionally non-destructive at the configuration level — no rules were added, only one was temporarily labelled. The cleanup is whatever you did to the labelled rule:

# Remove the lab-only label from the rule in the GUI, or
# restore the rule to its pre-lab configuration. Apply the change.

# Confirm no lab artefacts remain
pfctl -s rules | grep lab_https_out
# expected: empty (or the rule is back to its permanent label)

The state table self-heals — the killed states are gone permanently, and the new flows you generated will time out on their own. No lab state needs to be cleared.

What you learned

The state table is the firewall’s working memory. Reading it with pfctl -s state, pfctl -s state -v, and pfctl -si is the first diagnostic step for any flow question.
pfctl -k <ip> is targeted and surgical. pfctl -k <src> -k <dst> is more surgical still. pfctl -k label -k <label> is the cleanest pattern after a rule change.
pfctl -F state is the bulk flush. It is a maintenance-window command. Outside a maintenance window, prefer pfctl -k.
Always snapshot before and after. The diff is the evidence that the change did what you intended and only what you intended.
Labels are recorded at state creation. A new label on a rule does not retroactively label existing states.

Objectives

Prerequisites