audit rules and watch points - what to record

Audit rules define what the kernel records. Too few rules miss incidents; too many overwhelm the system. This lesson covers the right rules for production.

Rule categories

1. File watch: changes to specific files (config, binaries, sensitive data).
2. Syscall watch: specific syscalls (execve, setuid, bind).
3. User watch: actions by specific users.
4. System watch: kernel module loads, time changes, etc.

CIS Benchmark rules

The CIS Benchmarks include audit rules. Common ones:

# /etc/audit/rules.d/10-base-config.rules

# Identity files
-w /etc/group -p wa -k identity
-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/gshadow -p wa -k identity
-w /etc/security/opasswd -p wa -k identity

# sudoers
-w /etc/sudoers -p wa -k sudoers
-w /etc/sudoers.d/ -p wa -k sudoers

# SSH
-w /etc/ssh/sshd_config -p wa -k sshd

# Login/logout
-w /var/log/lastlog -p wa -k logins
-w /var/log/faillog -p wa -k logins
-w /var/log/wtmp -p wa -k logins

# Privileged commands
-a always,exit -F path=/usr/bin/sudo -F perm=x -k privileged
-a always,exit -F path=/usr/bin/su -F perm=x -k privileged
-a always,exit -F path=/usr/bin/passwd -F perm=x -k privileged
-a always,exit -F path=/usr/bin/chsh -F perm=x -k privileged
-a always,exit -F path=/usr/bin/chfn -F perm=x -k privileged
-a always,exit -F path=/usr/bin/newgrp -F perm=x -k privileged
-a always,exit -F path=/usr/bin/gpasswd -F perm=x -k privileged
-a always,exit -F path=/usr/bin/mount -F perm=x -k privileged
-a always,exit -F path=/usr/bin/umount -F perm=x -k privileged
-a always,exit -F path=/usr/bin/ssh-agent -F perm=x -k privileged

# Kernel module loads
-w /sbin/insmod -p x -k modules
-w /sbin/rmmod -p x -k modules
-w /sbin/modprobe -p x -k modules
-a always,exit -F arch=b64 -S init_module -k modules
-a always,exit -F arch=b64 -S finit_module -k modules
-a always,exit -F arch=b64 -S delete_module -k modules

Load with:

sudo augenrules --load
sudo auditctl -l

Freeze the rule set: -e 2

Everything above is undone by one command:

sudo auditctl -D    # delete all rules

An attacker who reaches root does this before doing anything else, and the audit trail of what they did next does not exist. The control that prevents it is the enabled flag. auditctl(8):

To lock the audit configuration so that it can’t be changed, pass a 2 as the argument. Locking the configuration is intended to be the last command in audit.rules […] Any attempt to change the configuration in this mode will be audited and denied. The configuration can only be changed by rebooting the machine.

Put these in the highest-numbered rules file so they sort last:

# /etc/audit/rules.d/99-finalize.rules

# Make loginuid unchangeable once set, so auid cannot be
# rewritten to disown an action
--loginuid-immutable

# MUST BE THE LAST LINE: freeze the rule set until reboot
-e 2

Confirm it took:

sudo auditctl -s | grep -w enabled    # expect: enabled 2

Capacity behaviour: the settings that can halt the host

auditd.conf decides what happens as /var/log/audit fills. Several of the actions are not logging actions at all — they take the machine down, by design, on the reasoning that a system that cannot record is a system that must not run.

# /etc/audit/auditd.conf
max_log_file = 50
num_logs = 10
max_log_file_action = ROTATE

space_left = 20%
space_left_action = SYSLOG

admin_space_left = 10%
admin_space_left_action = SYSLOG

disk_full_action = ROTATE

The values that bite, per auditd.conf(5):

Tuning for performance

Audit generates events. On busy hosts, too many rules slow the system.

# Exclude noisy processes
-a never,exclude -F exe=/usr/sbin/chronyd
-a never,exclude -F exe=/usr/sbin/rsyslogd

# Use a larger buffer
-b 8192

# Increase the failure mode (1 = printk instead of panic)
-f 1

Exclude processes that legitimately do many syscalls (log daemons, monitoring agents).

Reducing noise without losing coverage

There is no time-of-day filter. auditctl -F accepts a fixed field list — arch, auid, uid, exe, path, perm, success, msgtype and the rest — and none of them is a clock. Any rule you see written with -F 'msg<=08:00' is invented: there is no msg field either (the record-type field is msgtype, one word), and auditctl rejects the rule rather than silently ignoring it.

That is the right outcome. “Stop auditing outside office hours” would blind the audit trail during precisely the window an intruder would choose.

Reduce volume by excluding known-noisy actors, not time ranges. Suppression rules must come before the recording rules in /etc/audit/rules.d/, because the kernel stops at the first matching rule:

# /etc/audit/rules.d/10-exclusions.rules  - loaded first
-a never,exit -F arch=b64 -F exe=/usr/sbin/chronyd
-a never,exit -F arch=b64 -F exe=/usr/lib/systemd/systemd-journald
-a never,exit -F arch=b64 -F auid=unset -F uid=prometheus

# Drop a whole record type that is pure volume for your estate
-a never,exclude -F msgtype=CWD

Then the recording rules in a later-numbered file. Verify what the kernel actually loaded, since a malformed rule is refused individually while the rest of the file loads:

sudo augenrules --load
sudo auditctl -l                 # the rules in force, in order
sudo auditctl -s | grep -E 'lost|backlog'   # non-zero lost = the real volume problem

If lost is climbing, the answer is a bigger -b backlog and tighter exclusions — not less auditing during the night.

What to record vs skip

The rule: record what is needed to investigate an incident. Routine activity can be inferred from what is not recorded.

Document and version

Place audit rules in /etc/audit/rules.d/ with version control:

/etc/audit/rules.d/
├── 10-base-config.rules     # CIS baseline
├── 20-custom.rules          # site-specific
└── 99-debug.rules           # temporary debugging (off by default)

Configuration management owns these files. Drift detection catches ad-hoc changes.

Knowledge check