Firewall logs — default-log, the prefix VyOS generates, and reading the entries

Firewall logs are the operator’s primary source of forensic evidence when the firewall blocks traffic. Without logs, the operator cannot answer “why is this traffic dropped?”; with verbose logs, the operator drowns in noise. The defensive pattern: turn on default-log on production rule-sets, name those rule-sets so the generated log prefix is readable, and be deliberate about which individual rules also log.

This lesson covers the firewall logging configuration on VyOS 1.5 LTS, the default-log directive, the log prefix VyOS derives for you, per-rule logging and log-options, the log interpretation, and the production failure modes where firewall logging is too verbose or too sparse.

The firewall logging architecture

flowchart LR
  P["Packet arrives on eth1"] --> B["Base chain<br/>firewall ipv4 input filter"]
  B -->|"rule 10 action jump<br/>jump-target LAN-LOCAL"| F["Named rule-set<br/>LAN-LOCAL"]
  F --> R1["Rule 10<br/>state established, related<br/>action accept + log"]
  R1 --> R2["Rule 20<br/>action accept, dport 22"]
  R2 --> R3["default-action drop<br/>default-log"]
  R3 -.->|drop| L["log prefix<br/>ipv4-NAM-LAN-LOCAL-default-D"]
  R1 -.->|accept| L2["log prefix<br/>ipv4-NAM-LAN-LOCAL-10-A"]
  L --> SYS["kernel log<br/>kern facility"]
  L2 --> SYS
  SYS --> CENTRAL["Central syslog server"]

Two things are worth reading off that diagram before any configuration. First, the rule-set is not attached to an interface any more — in 1.4 and 1.5 a named rule-set is reached by a jump from one of the base chains, and the interface match lives on the jumping rule. Second, the strings in the log boxes are not something you configure. They are what VyOS emits, and their shape is the whole reason the log is greppable.

Enabling default-log

configure
set firewall ipv4 name LAN-LOCAL description 'traffic terminating on this router from the LAN'
set firewall ipv4 name LAN-LOCAL default-action 'drop'
set firewall ipv4 name LAN-LOCAL default-log
set firewall ipv4 name LAN-LOCAL rule 10 action 'accept'
set firewall ipv4 name LAN-LOCAL rule 10 state 'established'
set firewall ipv4 name LAN-LOCAL rule 10 state 'related'
set firewall ipv4 name LAN-LOCAL rule 10 log

# Reach the rule-set from the input base chain, matching on the interface
set firewall ipv4 input filter rule 10 action 'jump'
set firewall ipv4 input filter rule 10 jump-target 'LAN-LOCAL'
set firewall ipv4 input filter rule 10 inbound-interface name 'eth1'
commit
save

Four migrations are visible in that block, and each of them is a commit failure on 1.5 if you get it wrong:

• set firewall name X is now set firewall ipv4 name X. The family is explicit, and there is a parallel firewall ipv6 tree.
• enable-default-log is now default-log.
• state established 'enable' is now state 'established'. The old node took a leaf value; the new one is the value. A leftover enable is rejected.
• The per-interface binding set interfaces ethernet eth1 firewall local name 'LAN-LOCAL' is gone. Traffic reaches a named rule-set through action jump plus jump-target from input, forward or output filter, and the interface is matched with inbound-interface name / outbound-interface name.

After commit, every packet that falls through to the default action generates a log entry, and so does every packet accepted by rule 10.

Naming, since you cannot set a prefix

The prefix is derived, so the only lever you have on it is the rule-set name. That makes naming a logging decision, not just a readability one:

configure
set firewall ipv4 name WAN-IN description 'inbound from the transit provider'
set firewall ipv4 name WAN-IN default-action 'drop'
set firewall ipv4 name WAN-IN default-log
commit
save

The default-action drops from that rule-set arrive in the log as [ipv4-NAM-WAN-IN-default-D], which is directly greppable:

journalctl -k --since today | grep 'ipv4-NAM-WAN-IN-'

Three naming rules follow from how the prefix is built:

1. Distinct within fifteen characters, because of the truncation on the default-action line.
2. No characters you will have to escape. The name goes into a grep pattern for the rest of its life. Hyphens and alphanumerics; nothing else.
3. Name the traffic, not the rule. WAN-IN still means something after the rules inside it have been rewritten three times; ALLOW-SSH-AND-HTTPS does not.

Per-rule logging

configure
set firewall ipv4 name LAN-LOCAL rule 10 action 'accept'
set firewall ipv4 name LAN-LOCAL rule 10 state 'established'
set firewall ipv4 name LAN-LOCAL rule 10 state 'related'
set firewall ipv4 name LAN-LOCAL rule 10 log
set firewall ipv4 name LAN-LOCAL rule 10 log-options level 'info'
commit
save

Rule 10 now logs every packet it accepts, under [ipv4-NAM-LAN-LOCAL-10-A]. The rule number is in the prefix, so “which rule accepted this traffic?” is answered by the log line itself rather than by reconstruction.

That is also the trap. Renumbering a rule changes its log prefix. Insert a rule at 5 and renumber 10 to 20, and every saved query, dashboard filter and alert that matched -10-A silently matches nothing. Nothing fails; the graph just goes flat. If you have alerting built on firewall log prefixes, treat a rule renumber as a change to the alerting as well.

The log-options a rule accepts are:

level is the one you will use. Setting a drop rule to warn and a routine accept to debug lets the syslog configuration — not the firewall — decide what leaves the box, which is where that decision belongs.

Log interpretation

A netfilter log entry is the VyOS-generated prefix followed by the kernel’s own field dump:

$ journalctl -k --since today | grep 'ipv4-NAM-LAN-LOCAL-'

Aug 15 12:00:01 R1 kernel: [ipv4-NAM-LAN-LOCAL-default-D]IN=eth1 OUT= MAC=00:1a:2b:3c:4d:5e:aa:bb:cc:dd:ee:f0:08:00 SRC=192.0.2.50 DST=192.0.2.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=12345 PROTO=TCP SPT=54321 DPT=22 WINDOW=65535 RES=0x00 SYN URGP=0
Aug 15 12:00:05 R1 kernel: [ipv4-NAM-LAN-LOCAL-10-A]IN=eth1 OUT= MAC=00:1a:2b:3c:4d:5e:aa:bb:cc:dd:ee:f0:08:00 SRC=192.0.2.50 DST=192.0.2.1 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=12346 PROTO=TCP SPT=54321 DPT=22 WINDOW=501 RES=0x00 ACK URGP=0

The fields:

• [ipv4-NAM-LAN-LOCAL-default-D] — the generated prefix: IPv4, a named rule-set, LAN-LOCAL, the default action, D for drop. The second line is rule 10 with A for accept.
• IN=eth1 OUT= — arrived on eth1; no outgoing interface, because this packet was destined for the router itself rather than being forwarded.
• MAC= — destination MAC, source MAC and ethertype, concatenated.
• SRC=192.0.2.50 DST=192.0.2.1 — source and destination addresses.
• PROTO=TCP SPT=54321 DPT=22 — protocol, source port, destination port (22 is SSH).
• SYN — the TCP flags that were set. SYN with no ACK is a connection attempt; the second line’s bare ACK is mid-conversation.

Two things that are not in the line, and that operators coming from other firewalls look for anyway:

• There is no trailing DROP or ACCEPT word. The kernel dumps packet fields; it does not append the verdict. The action is in the prefix, as the final letter, and that is the only place it appears.
• There is no rule-set name for base-chain rules. A rule logged from firewall ipv4 input filter renders as [ipv4-INP-filter-10-D] — filter is a literal, not a name you chose.

Note also that the prefix runs straight into IN= with no separating space. That is not a rendering error; it is how netfilter concatenates a prefix with the field dump. Grep for the prefix including its closing bracket (grep 'ipv4-NAM-LAN-LOCAL-default-D]') and the boundary is unambiguous.

How the result is validated

show firewall
show firewall summary
show firewall statistics
show log firewall ipv4 input filter
journalctl -k --since today | grep 'ipv4-NAM-LAN-LOCAL-'

show firewall prints the rule-sets; show firewall summary adds the groups; show firewall statistics gives the per-rule packet and byte counters since boot; show log firewall ipv4 input filter filters the log down to one base chain. The journalctl line is the general case — everything the firewall logs is a kernel message, so -k plus the prefix is the query that always works.

A working firewall logging setup:

• Default-action drops appear under [<family>-<hook>-<name>-default-<action>].
• Rules configured with log appear under their own rule number.
• The counters in show firewall statistics move when the log lines appear. If the counter moves and the log does not, log is missing from that rule; if neither moves, the traffic is not reaching the rule-set at all.
• The log is exported to the central syslog server.
• The log volume is bounded.

Bounding the log volume

There is no option that rate-limits logging alone. This is the second thing 1.3 muscle memory gets wrong here: there is no log-options limit. What VyOS gives you is limit, which rate-limits the rule match — and that is a firewall change, not a logging change:

configure
set firewall ipv4 name WAN-IN rule 30 action 'drop'
set firewall ipv4 name WAN-IN rule 30 log
set firewall ipv4 name WAN-IN rule 30 limit rate '100/second'
set firewall ipv4 name WAN-IN rule 30 limit burst '200'
commit
save

Read that carefully before deploying it. Rule 30 now matches at most 100 packets per second. The 101st packet does not “get dropped without a log entry” — it does not match rule 30 at all, and continues to the next rule and eventually to the default action. If the default action is accept, a limit you added to control log volume has just punched a hole in the firewall under exactly the load where you least want one. Put limit on a rule whose fall-through you have thought about, and prefer these levers when the goal is only log volume:

• log-options level — mark routine rules debug and let the syslog configuration drop them before they reach the disk or the network.
• Log the drops, not the accepts. Accepted traffic is the bulk; the drops are the evidence.
• Bound the log at the log end — logrotate, journald’s SystemMaxUse, and retention on the central server — rather than by making the firewall match less traffic.

How it fails

The production failure modes a routing engineer must recognise:

• Logging not enabled. Dropped packets vanish silently and no query can recover them after the fact. The fix: set firewall ipv4 name <ruleset> default-log on every production rule-set, and log on the rules whose hits are evidence.
• enable-default-log typed from memory. On 1.5 the node is default-log; the 1.3 spelling is rejected at commit. This is the good failure — you find out immediately. The bad version of it is committing the rest of the change, seeing it succeed, and assuming logging came with it.
• Too verbose. Every accepted packet is logged, the disk fills, and the drops you needed are buried among millions of accepts. The fix: log drops and rejects; log accepts only while investigating.
• Rule-set names that collide under truncation. Two rule-sets whose names match for fifteen characters produce identical default-action prefixes. The fix is a rename, which means a change window, which is why this is worth getting right when the rule-set is created.
• A dashboard broken by a rule renumber. Queries matched on -10-D; the rule became 20; the graph went to zero and nobody noticed because zero looks like “no attacks”. The fix: match on the rule-set and action rather than the rule number, or treat renumbering as a change to the alerting.
• limit used as a log throttle. It is a match limit. Packets over the rate skip the rule entirely and fall through, so a rule that was dropping now sometimes does not. See the section above.
• Logs not exported. Local logs fill the disk and are lost with the box. The fix: remote syslog, verified by looking at the receiver rather than at the sender’s configuration.

Rollback

The recovery from a broken firewall logging configuration:

• Too verbose: delete firewall ipv4 name <ruleset> default-log, and delete firewall ipv4 name <ruleset> rule <n> log for the chatty rules. Both take effect on commit; no restart is involved.
• Not capturing: add them back, then prove it with traffic rather than with the configuration — show firewall statistics to see the counter move and journalctl -k to see the line appear.
• Disk full: this is a filesystem problem, not a firewall one. Rotate or delete the archived logs, fix the retention, and only then re-enable the logging you turned off.

Note that rollback <N> reverts the whole configuration to a stored revision and currently requires a reboot on VyOS. For a logging change, deleting the two nodes you added is faster, narrower, and does not take the router down.

Production discipline

Cross-course references

• XLVIII-VyOS-Logging (vyos-xlviii-01-local-logging, vyos-xlviii-05-remote-syslog) cover the rest of the logging subsystem.
• XXXVII-VyOS-Firewall covers the firewall configuration in detail.

Quiz