Lab: auditd watch and search

This lab sets up auditd rules, generates events, and queries the audit log. By the end you will have the discipline for production audit deployment.

Requirements

A disposable lab host or VM with auditd installed. Do not run this on a shared or production system: you load a rule file into /etc/audit/rules.d/ and touch identity files.
Root access via sudo.
Nothing else. The lab needs no spare disk, no loop device, and no test filesystem.

Tasks

Task 1: Verify auditd is running

sudo systemctl status auditd
sudo auditctl -s     # status

Task 2: Write rules

sudo tee /etc/audit/rules.d/99-lab.rules <<EOF
-w /etc/passwd -p wa -k passwd
-w /etc/shadow -p wa -k shadow
-w /etc/sudoers -p wa -k sudoers
-w /usr/bin/sudo -p x -k privileged
EOF
sudo augenrules --load
sudo auditctl -l

Task 3: Trigger events

A watch fires on the syscall, not on the result. A metadata write is recorded even when the values it writes are identical to the values already there. That is the whole trick: you can exercise a watch on a sensitive file without weakening it.

Record the starting state first, so you can prove nothing moved:

stat -c '%n %a %U:%G %y' /etc/passwd /etc/shadow /etc/sudoers

Now generate the events. touch -r FILE FILE reads the timestamps from the file itself and writes them straight back, so the utimensat call is always issued and the file is never altered:

sudo touch -r /etc/passwd  /etc/passwd
sudo touch -r /etc/shadow  /etc/shadow
sudo touch -r /etc/sudoers /etc/sudoers

# Privileged execution
sudo ls /root

Re-run the stat command. Every field is identical. Task 4 will still find the events.

Never loosen /etc/shadow to make an audit event

An older version of this lab used chmod 644 /etc/shadow. Do not do that, and refuse it in review if you see it in someone else’s runbook. /etc/shadow ships as 0640 root:shadow on Debian-family systems and 0000 root:root on RHEL-family systems, and it holds every password hash on the host. Mode 644 hands those hashes to every local user, every service account, and every container that bind-mounts /etc. Offline cracking then runs at the attacker’s leisure, against root included. A permission change is also silent: nothing breaks, so nobody notices until the hashes surface somewhere else.

The same reasoning rules out chmod 0640 /etc/gshadow as a “safe” substitute. That is only a no-op on Debian-family systems; on RHEL-family systems it loosens 0000 to 0640. Any command that names a literal mode has assumed a distribution and will eventually be wrong on one. The rule is simple: to trigger a watch, never supply a value of your own. Read the file’s value and write it back, which is exactly what touch -r does.

Task 4: Query

sudo ausearch -k passwd -ts recent
sudo ausearch -k shadow -ts recent
sudo ausearch -k sudoers -ts recent
sudo ausearch -k privileged -ts recent

The shadow records are the interesting ones. Read the SYSCALL line: syscall=utimensat, success=yes, and the PATH line names /etc/shadow. The watch caught the call even though the timestamps it wrote were the timestamps already there. An audit trail records intent, not diffs.

Task 5: Summarise

sudo aureport --summary -ts today
sudo aureport --file -ts today
sudo aureport --comm -i -ts today      # commands run; there is no --sudo report
sudo ausearch -m USER_CMD -ts today -i # the sudo-specific view

If any of these prints “no matches”, check that the events you expect are in the log at all with sudo ausearch -ts today | head before concluding the rules are wrong — an empty report and a missing rule look the same from here.

Task 6: Document

AUDITD LAB
==========
Host: <host>
Date: 2026-08-09

Rules loaded:
- /etc/passwd watched
- /etc/shadow watched
- /etc/sudoers watched
- /usr/bin/sudo execution watched

Events captured:
- metadata writes on the watched identity files
- privileged execution

Verification:
- ausearch returns expected events
- aureport summary shows the expected categories
- stat confirms no mode, owner or timestamp changed
- lab rule file removed and rules reloaded

Validation

The lab succeeded when all three hold:

# 1. The four lab rules are loaded.
sudo auditctl -l | grep -E 'passwd|shadow|sudoers|/usr/bin/sudo'

# 2. Each key returns at least one syscall record.
for k in passwd shadow sudoers privileged; do
  printf '%-12s %s\n' "$k" \
    "$(sudo ausearch -k "$k" -ts today 2>/dev/null | grep -c '^type=SYSCALL')"
done

# 3. The watched files are untouched.
stat -c '%n %a %U:%G' /etc/passwd /etc/shadow /etc/sudoers

Expected on a Debian-family host: /etc/passwd 644 root:root, /etc/shadow 640 root:shadow, /etc/sudoers 440 root:root. On a RHEL-family host /etc/shadow is 0 root:root. Either way, the modes must match what you recorded in Task 3.

Cleanup

Remove the lab rules and return auditd to its previous rule set:

sudo rm -f /etc/audit/rules.d/99-lab.rules
sudo augenrules --load
sudo auditctl -l

The final auditctl -l must no longer list the four lab rules. If the host had no rules before the lab, it prints No rules.

Confirm you left no permission damage behind:

stat -c '%n %a %U:%G' /etc/shadow /etc/gshadow

/etc/shadow and /etc/gshadow must still be 640 root:shadow on Debian-family systems, or 0 root:root on RHEL-family systems. If either is 644, fix it now:

# Debian-family
sudo chown root:shadow /etc/shadow /etc/gshadow
sudo chmod 0640 /etc/shadow /etc/gshadow

Then rotate every password on the host. A hash that has been world-readable is a leaked hash, whatever the mode says afterwards.

The audit records you generated stay in /var/log/audit/. Leave them: rotation removes them on its own schedule, and deleting audit logs by hand is its own bad habit.

Objectives

Prerequisites