Log retention and rotation — keeping what you need, deleting what you do not · OPNsense

Logs are not free. Every line stored costs disk; every line retained past its useful life is a liability. The retention model on OPNsense is two-tier: about a month of daily files on the firewall itself, and whatever the remote destination is configured to keep. The discipline is knowing which tier owns which period, setting the two knobs that bound the local tier, and applying the rules that say when logs must be kept and when they must be deleted.

The two-tier retention model

Tier 1: the daily files on the firewall

One plain-text file per application per day under /var/log/<application>/.
Kept for Maximum preserved files files, which defaults to 31.
Trimmed by a housekeeping job, not overwritten in place.
Useful life: roughly a month, and searchable for the whole of it.
Purpose: investigation on the box, without leaving the box.

Tier 2: the remote destination

SIEM, log server, or cloud aggregation.
Indexed, cross-referenced against other sources, retained on the receiver’s policy.
Useful life: months to years.
Purpose: long-term investigation, compliance, threat hunting, correlation with the rest of the estate.

These are not redundant, and neither is a substitute for the other. Tier 1 is genuinely useful — a month of greppable firewall history is enough for most incidents — but it dies with the firewall, and a firewall that has been compromised is a firewall whose local logs cannot be trusted. Tier 2 survives the box and is the only tier that can be relied on for evidence.

The local tier is a real archive, but it is bounded by disk, not by time

The trap is not that the local logs vanish quickly. They do not: the default is 31 daily files, and nothing overwrites them in place. The trap is the other way round. Maximum preserved files bounds the number of files, and nothing in the default configuration bounds their size, so a firewall that starts logging a great deal more than it used to keeps its thirty-one files and grows /var/log until the partition is full.

The discipline follows: monitor free space where /var/log lives, and treat widening rule logging as a capacity change rather than a visibility change. Set Maximum file size on any firewall whose logging volume is not predictable, because the product of the two settings is the only hard ceiling available.

The two settings

Both live under System → Settings → Logging, on the “Local” tab.

Setting	Default	What it does
Enable local logging	on	Turn off to stop writing local files entirely — intended for installs on flash media, and it makes remote logging mandatory rather than optional
Maximum preserved files	31	How many files are kept per application. With one file per day this is a day count; with Maximum file size set it is a file count and no longer a day count
Maximum file size	empty	Size in MB at which the current file is rotated early. Empty means never

The interaction between the last two is the part that catches people. Left at the defaults, thirty-one files means thirty-one days. Set Maximum file size to 50 MB on a firewall that writes 200 MB of filter log a day, and the file rotates four times a day, so thirty-one files is now about eight days. Bounding the disk shortens the window; the two settings cannot both be chosen freely.

Read-only / Safea day that rotated on size— Three sequence-suffixed files for 14 August plus the current unsuffixed one, which means Maximum file size is set and this day has already rotated three times. latest.log points at the unsuffixed file, because that is the one syslog-ng is writing. The operator should read four files in one day as a signal that the retention window in days is much shorter than the file count suggests, and should divide accordingly before promising anyone a month of history.

$ ls /var/log/filter/ | tail -5; echo '---'; du -sh /var/log; df -h /var/log

filter_20260814.0001.log
filter_20260814.0002.log
filter_20260814.0003.log
filter_20260814.log
latest.log
---
1.4G    /var/log
Filesystem          Size    Used   Avail Capacity  Mounted on
/dev/gpt/rootfs      30G    8.2G     20G    29%    /

Illustrative output

What the housekeeping job actually does

Rotation on OPNsense is not size-triggered inside syslog-ng. syslog-ng starts a new file when the date changes, and everything else is done by a separate job — configctl syslog archive, which appears in the cron action picker as “Rotate log files”. OPNsense schedules it automatically once an hour. For each log directory it:

Checks the newest file against Maximum file size. If it is over, renames it with a four-digit sequence suffix — filter_20260814.0001.log — and restarts syslog-ng so a fresh unsuffixed file starts.
Sorts the files newest first and deletes everything past Maximum preserved files.
Repoints the directory’s latest.log symlink at the newest file.

Two consequences are worth holding on to. Deletion is permanent and unattended: a file trimmed by step 2 is unlinked, not archived, not compressed, not moved. And because step 1 restarts the daemon, a very small Maximum file size on a very busy firewall means frequent restarts, which is a reason to keep the value generous rather than tight.

Read-only / Saferunning the rotation job by hand— The job can be run on demand, which is the quickest way to reclaim space after lowering Maximum preserved files — the setting takes effect the next time the job runs, so without this the disk stays full until the hour turns. Thirty-two entries is thirty-one daily files plus the latest.log symlink, which is the default retention doing exactly what it says. Running it more often than hourly is a supported choice: add it under System > Settings > Cron using the Rotate log files action.

$ configctl syslog archive; echo '---'; ls /var/log/filter/ | wc -l

OK
---
32

Illustrative output

What still uses newsyslog

A handful of components write their own files rather than going through syslog-ng, and those are rotated by FreeBSD’s newsyslog on its own schedule and its own generation count. Suricata’s /var/log/suricata/eve.json is the significant one; the Squid access log behaves the same way when the plugin is configured to write directly rather than through syslog.

This matters when planning disk. The Maximum preserved files setting has no effect on these files at all — their retention is configured on the component that owns them, on the IDS settings page in Suricata’s case. A firewall sized on the assumption that one setting governs all of /var/log will be surprised by the IDS.

Remote retention

The remote destination owns its own retention, typically in tiers:

Hot: indexed and fast, 30 to 90 days.
Warm: indexed on slower storage, 90 days to a year.
Cold: archived and not indexed, needing re-ingestion to query, one to seven years.
Legal hold: exempt from expiry for the duration of an investigation.

Anything beyond the local month lives here. This is also the tier that survives the firewall being rebuilt, replaced, or compromised.

Defining the retention policy

Retention is a business question with technical consequences. The inputs:

Legal requirements: regulated industries have mandatory periods. PCI DSS requires a year of audit history with three months immediately available. Some healthcare regimes require six years. The UK GDPR sets no fixed period but requires that retention be no longer than necessary for the stated purpose.
Operational needs: how far back investigations actually reach, and how long post-mortems take to start.
Storage cost: the marginal cost per month per terabyte, which is what makes cold tiers worth having.
Risk: longer retention widens the blast radius of a breach of the log store itself.

A worked policy matrix:

Log type	Firewall (local)	Hot	Warm	Cold
Filter	31 days	30 days	90 days	1 year
Audit and authentication	31 days	30 days	90 days	1 year
IDS alerts	per IDS settings	90 days	1 year	3 years
Web proxy	per plugin settings	30 days	90 days	1 year
VPN	31 days	30 days	90 days	1 year
System and service	31 days	7 days	30 days	—

The numbers depend on the estate. The discipline is to write the policy down, configure both tiers to enforce it, and audit the actual retention against it — including confirming that the firewall’s local window is what the policy assumes rather than what the defaults happen to give.

Why a file count rather than a size budget

There are two obvious ways to bound a log store: cap the bytes, or cap the units. OPNsense caps the units — a count of files per application — and this is worth understanding rather than working around.

Capping bytes across all of /var/log would mean that a noisy component could evict a quiet one. A burst of IDS activity would silently shorten the retained window of the audit log, and the operator investigating an authentication incident would find the evidence gone for reasons that had nothing to do with authentication. Per-application file counts give each component an independent, predictable window.

The cost is that the total is not bounded by the settings alone; it is bounded by the settings multiplied by whatever a day of that component’s logging weighs. That product is the number to compute when sizing a firewall, and Maximum file size is the knob that makes it computable rather than empirical.

The other design choice is that nothing is compressed. Compressing would roughly triple the retained window for the same disk, but it would also mean grep no longer works on the archive, and the whole model depends on the files staying readable by ordinary tools. Compression is the receiver’s job, where nobody greps by hand.

Deletion discipline

The policy defines when logs must be deleted. The discipline around it:

Automatic deletion: both tiers expire on their own. Verify it is actually happening rather than assuming — a SIEM whose expiry job has been failing silently for a year is a compliance finding, not a bonus.
Legal hold override: an investigation must be able to exempt specific data from expiry, and lift that exemption afterwards. On the firewall’s local tier there is no such mechanism, which is another reason evidence is collected from the remote tier.
Right to erasure: logs recording only addresses, ports, and counts are generally not personal data; logs naming a user usually are. There must be a process that can satisfy an erasure request without destroying unrelated records.
Audit of deletion: a record of what was deleted, when, and under which policy. The deletion record is itself a log, and it outlives what it describes.

Summary

The local tier is about thirty-one daily files per application, trimmed hourly by configctl syslog archive and readable for the whole window.
Maximum preserved files bounds the file count, not the disk footprint. Maximum file size bounds each file, at the cost of shortening the window in days.
Deletion by the housekeeping job is immediate and permanent; lowering the retention setting destroys the difference on the next run.
Suricata and the proxy rotate their own files with newsyslog and are not governed by these settings.
Long-term retention, correlation, and anything that must survive the firewall belong on the remote destination.
Retention policy is a business question — legal, operational, cost, risk — not a technical default.