DMZ monitoring and incident response — IDS, log forwarding, and what to do when a DMZ host is compromised

A DMZ is reachable from the Internet. Compromise of a DMZ host is not a matter of if, but when. The DMZ operator’s job is to detect compromise quickly, contain it, eradicate it, and recover — and to do all of this with the evidence intact. This lesson covers the monitoring that detects compromise, the IDS that catches known-bad traffic, the log forwarding that preserves evidence, and the incident response playbook that the operator follows when a DMZ host is compromised.

The DMZ exists to limit the blast radius of compromise. Monitoring and incident response exist to limit the duration of compromise. A compromise detected in minutes and contained in hours is recoverable; a compromise detected in months and contained in quarters is a breach disclosure.

Monitoring the DMZ

Monitoring the DMZ has three layers:

1. Host-level monitoring. Each DMZ host runs an agent (Prometheus node_exporter, Telegraf, Datadog agent) that reports CPU, memory, disk, network, and process metrics. Anomalies (high CPU, unfamiliar processes, unusual outbound traffic) are signals.
2. Network-level monitoring. The firewall captures packets on the DMZ interface. IDS (Suricata) inspects the packets for known signatures. NetFlow or sFlow records volume and peer information.
3. Log-level monitoring. The DMZ hosts send auth logs, application logs, and system logs to a central destination. The firewall sends its own logs to the same destination. The central destination indexes and alerts on anomalies.

The three layers together give the operator multiple views of the same events. A web shell uploaded to a DMZ web server shows up as an unfamiliar process (host), as a known-bad HTTP request (network), and as an unusual file modification (log).

IDS in the DMZ

OPNsense includes Suricata as a plugin. Suricata runs as an IDS (Intrusion Detection System) or IPS (Intrusion Prevention System). In the DMZ, IDS is the right mode — Suricata observes traffic and alerts, but does not block.

The deployment pattern:

1. Suricata listens on the DMZ interface. The firewall copies (mirrors) the DMZ traffic to Suricata. Suricata inspects every packet against its rule set.
2. Rule set is updated daily. Emerging Threats, Proofpoint, or other rule feeds provide daily updates. The operator applies the updates and reviews new rules.
3. Alerts go to the central logging destination. Suricata alerts are formatted as JSON or CEF and forwarded to the same log management destination as the host and firewall logs.
4. Operator reviews alerts. Alerts are triaged: false positive, known-bad-and-handled, or genuine incident.

$ tail -3 /var/log/suricata/eve.json

{"timestamp":"2026-08-14T03:14:01.123456+0000","flow_id":1234567890,"event_type":"alert","src_ip":"203.0.113.10","src_port":443,"dest_ip":"198.51.100.50","dest_port":54321,"proto":"TCP","alert":{"action":"allowed","gid":1,"signature_id":2024897,"rev":2,"signature":"ET WEB_SERVER Possible CVE-2024-XXXX Exploit","category":"Attempted Administrator Privilege Gain","severity":1}}

A single Suricata alert captures the timestamp, source, destination, signature, severity, and action. The operator triages the alert: false positive (tune the rule), known-handled (the patch is deployed), or genuine incident (escalate).

Log forwarding

DMZ hosts must forward logs to a remote destination. The standard pattern:

1. Syslog forwarding. The DMZ host runs rsyslog, syslog-ng, or a similar syslog daemon. The daemon is configured to forward auth, syslog, and application logs to a remote syslog server.
2. TLS-encrypted transport. The forwarding uses TLS (RFC 5425) to encrypt the logs in transit. The remote syslog server has a certificate the DMZ host trusts.
3. Application logs. Web servers, mail servers, and DNS servers have their own log files. The application log forwarder (e.g. filebeat, promtail) ships these to the central destination.
4. Central destination. The logs land in a log management service (Elasticsearch, Loki, Splunk) that indexes them, retains them for the required period, and alerts on anomalies.

The remote destination is typically on the management VLAN or in a private cloud. The DMZ cannot reach the destination by direct firewall rule; the forwarding is initiated by the DMZ host outbound to the destination.

$ sockstat -l -p 514

root     syslogd    1234  4  udp4   *:514                 *:*
root     syslogd    1234  5  tcp4   *:514                 *:*
root     syslogd    1234  6  tcp46  *:6514                *:*

The syslog server binds to all interfaces for compatibility, but the firewall ensures only the management VLAN (or specifically the log forwarders) can reach the syslog ports.

The incident response playbook

When a DMZ host is compromised (or suspected of compromise), the operator follows a playbook. The phases are based on NIST SP 800-61r2: Preparation, Detection and Analysis, Containment, Eradication, Recovery, and Post-Incident Activity.

Phase 1: Preparation

Preparation happens before the incident. The playbook exists. The team knows the playbook. The tools are configured:

• Forensic toolkit is ready (memory acquisition, disk imaging, packet capture).
• Communication channels are known (out-of-band: phone, signal, in-person).
• Legal and disclosure obligations are documented (regulatory requirements for breach notification).
• Backup snapshots are known and reachable.
• Rollback procedures are documented.

Phase 2: Detection and analysis

The incident is detected. The detection may come from:

• Suricata alert (known-bad signature match).
• Host monitoring anomaly (unfamiliar process, unusual outbound traffic).
• Log analysis anomaly (failed login spike, unusual file access).
• External notification (third party reports compromise).
• Operator observation (manual review).

The analysis phase gathers evidence to confirm the compromise and scope it:

• Capture a memory image of the host (if live analysis is possible).
• Capture a disk image of the host.
• Capture recent packet captures from the DMZ interface.
• Review auth logs, application logs, and IDS alerts.
• Identify the attack vector (the vulnerability exploited).
• Identify the scope (what was accessed, what was modified).

Phase 3: Containment

The compromise is contained to prevent further damage:

• Network isolation. Take the DMZ host off the network. The simplest method: change the firewall rule to block all traffic to/from the host’s IP. The host is still running but cannot communicate with anything.
• Account isolation. Disable any accounts that may have been compromised.
• Service isolation. Stop any services that are the vector of compromise.

The containment phase preserves evidence while preventing further damage. The operator does not yet wipe the host or rebuild it — that is the eradication phase.

Phase 4: Eradication

The compromise is removed:

• Identify the malware, the backdoor, the persistence mechanism.
• Remove the malware, the backdoor, the persistence mechanism.
• Patch the vulnerability that was exploited.
• Reset credentials that may have been compromised.

For a clean eradication, the safest pattern is to rebuild the host from a known-good image and apply all patches before reconnecting. Reimaging is faster and more reliable than cleaning a compromised host.

Phase 5: Recovery

The host is restored to service:

• Reimage the host from a known-good image.
• Apply all patches and configuration.
• Restore data from a known-good backup (one taken before the compromise).
• Verify the host is clean before reconnecting.
• Reconnect the host with elevated monitoring for a defined period.

Phase 6: Post-incident activity

Lessons are captured:

• What happened (timeline of events).
• What worked (playbook steps that were effective).
• What did not work (gaps in monitoring, detection, response).
• What changes are needed (new monitoring, new rules, new procedures).

The post-incident activity feeds back into preparation. The next incident response is faster because the last one was reviewed.

Indicators of compromise in the DMZ

Five indicators the operator should monitor actively.

Indicator 1: Unfamiliar processes

The DMZ host is running a process that is not part of the standard service stack. The process may be a reverse shell, a cryptocurrency miner, a backdoor daemon.

The detection: host monitoring agent reports unusual process names. The investigation: ps auxf, check process binary path, check process parent, check open files and network connections.

Indicator 2: Unusual outbound traffic

The DMZ host is making outbound connections to IPs or domains that are not in the standard allow-list. The traffic may be command-and-control, data exfiltration, or beaconing.

The detection: Suricata alerts on known C2 signatures; firewall log shows unusual outbound destinations; netflow analysis shows traffic to unusual ASNs. The investigation: identify the process making the connection, identify the destination, capture the traffic.

Indicator 3: Web shell or modified application files

The DMZ web server has files that should not be there (web shells, droppers) or files that have been modified unexpectedly.

The detection: file integrity monitoring (AIDE, Tripwire, OSSEC), or manual review of recent file modifications. The investigation: identify the modification time, identify the source of the modification, capture the file.

Indicator 4: Failed authentication spike

The DMZ host sees a spike in failed SSH or application logins. The spike may be brute force, credential stuffing, or post-compromise reconnaissance.

The detection: log analysis (Loki, Elasticsearch, Splunk) flags the spike. The investigation: identify the source IPs, identify the targeted accounts, identify whether any succeeded.

Indicator 5: Resource exhaustion

The DMZ host is at high CPU, memory, or network utilisation unexpectedly. The exhaustion may be a DoS, a cryptocurrency miner, or data exfiltration.

The detection: host monitoring alerts on resource thresholds. The investigation: identify the process consuming the resource, identify the cause.

$ tail -20 /var/log/messages | grep -E 'sshd|auth'

Aug 14 03:12:01 dmz-web sshd[12345]: Failed password for admin from 198.51.100.50 port 54321 ssh2
Aug 14 03:12:02 dmz-web sshd[12346]: Failed password for admin from 198.51.100.50 port 54322 ssh2
Aug 14 03:12:03 dmz-web sshd[12347]: Failed password for admin from 198.51.100.50 port 54323 ssh2
Aug 14 03:12:04 dmz-web sshd[12348]: Failed password for admin from 198.51.100.50 port 54324 ssh2
Aug 14 03:12:05 dmz-web sshd[12349]: Failed password for admin from 198.51.100.50 port 54325 ssh2

A five-line extract of auth log captures a brute-force pattern. The operator’s response is fast: block the source IP, verify the host is hardened against password auth, audit for successful logins.

A monitoring checklist

For every DMZ deployment:

• Suricata IDS is running on the DMZ interface.
• Suricata rule set is updated daily.
• DMZ hosts forward auth logs to a remote syslog destination.
• DMZ hosts forward application logs to a remote log management destination.
• The remote destinations are on a network the DMZ cannot reach by direct rule.
• Host monitoring agents are deployed on every DMZ host.
• The firewall forwards its logs to the same central destination.
• Alerts are configured for failed authentication spikes, unusual outbound traffic, and host resource anomalies.
• The incident response playbook is documented and the team is trained.
• Forensic tools are ready (memory acquisition, disk imaging, packet capture).

Summary

• DMZ monitoring has three layers: host, network, and log. Together they give multiple views of the same events.
• Suricata IDS inspects traffic on the DMZ interface. Alerts go to the central logging destination.
• DMZ logs are forwarded to a remote, immutable destination. Local logs can be wiped by an attacker.
• The incident response playbook follows NIST phases: preparation, detection, containment, eradication, recovery, post-incident.
• Compromised hosts are rebuilt from a known-good image, not cleaned. Forensic analysis happens on a copy.
• Five indicators of compromise: unfamiliar processes, unusual outbound traffic, modified files, failed authentication spikes, resource exhaustion.