Skip to main content
RunBook Academy

← All labs in OPNsense

Lab · advanced · ~60 min

Lab: Hairpin NAT vs split-horizon DNS — measure both, pick one

B · Nested virtualisationC · Simulation

Objectives

  • Configure NAT reflection for a port-forwarded service and verify it works
  • Measure the latency and packet path of a hairpin-resolved connection
  • Disable NAT reflection and configure Unbound split-horizon DNS instead
  • Compare the two approaches on latency, packet count, and failure modes
  • Recognise the operational failure modes of each approach

Prerequisites

This lab compares two ways of letting internal clients reach a port-forwarded service by hostname: NAT reflection (hairpin) and split-horizon DNS. You will configure both, measure the latency and packet count for each, and write a short report describing which approach is right for which scenario. The discipline on display is the one a capacity planner needs: measure before committing, document the trade-offs, and name the failure modes of each approach.

By the end you will have empirical evidence that split-horizon DNS is faster than hairpin NAT, and you will have hands-on experience with both failure modes (the override that points to the wrong IP, and the ISP that blocks hairpin traffic).

Objective

By the end of this lab, you can:

  • Configure NAT reflection for a port-forwarded service and verify it works.
  • Measure the latency and packet path of a hairpin-resolved connection.
  • Configure Unbound split-horizon DNS for the same service.
  • Compare the two approaches on latency, packet count, and side effects.
  • Recognise the operational failure modes of each approach.

Requirements

  • An OPNsense instance with at least two interfaces (LAN and WAN).
  • A port forward configured for an internal service (a web server is ideal because it has a TCP handshake and an application exchange you can measure).
  • A LAN client that can resolve via the firewall’s Unbound.
  • A second LAN client you can configure to use a public DNS (for the NAT reflection test).
  • A means of capturing packets on both the LAN and WAN sides (the firewall’s own tcpdump is sufficient).

Tasks

Task 1: Establish the baseline

You need a port forward to work with. If one already exists from a previous lab, document it. If not, create one:

  • Interface: WAN
  • Protocol: TCP
  • External port: 8443
  • Internal: INTERNAL_SERVER_IP:443
  • Description: Lab port forward for hairpin/split-DNS test

Apply the change. From the external client, verify:

# Substitute your own values before running:
FW_WAN_IP=203.0.113.10          # the firewall's WAN address

curl -k --max-time 5 "https://$FW_WAN_IP:8443/" > /dev/null && echo "external reach OK"

Now confirm that internal clients cannot reach the service by the public IP. NAT reflection is disabled by default.

From a LAN client, test:

# Substitute your own values before running:
FW_WAN_IP=203.0.113.10          # the firewall's WAN address

curl -k --max-time 5 "https://$FW_WAN_IP:8443/" 2>&1 | head -5

If the connection fails (timeout or connection refused), you have the baseline. NAT reflection is disabled, and the WAN link is not traversed by the connection attempt — the packet leaves the LAN, fails to come back, and the client times out.

Task 2: Configure NAT reflection

In the GUI, navigate to Firewall → Settings → Advanced. Find the NAT Reflection setting. Three options:

  • Disable NAT Reflection (default)
  • Enable NAT Reflection (NAT + proxy)
  • Enable NAT Reflection (proxy only)

Select Enable NAT Reflection (NAT + proxy). Save.

Verify the reflection rule was generated:

pfctl -s nat -v | grep -A 1 reflect

You should see a generated rdr rule for the LAN interface that handles traffic from the LAN to the firewall’s WAN IP.

Task 3: Test the hairpin connection

From the LAN client, repeat the test:

# Substitute your own values before running:
FW_WAN_IP=203.0.113.10          # the firewall's WAN address

curl -k --max-time 5 "https://$FW_WAN_IP:8443/" > /dev/null && echo "hairpin OK"

The connection should now succeed. The packet path: the LAN client sends a SYN to the firewall’s WAN IP, the firewall recognises the destination, applies the rdr rule, rewrites the destination to the internal server, and forwards the packet on the LAN interface. The internal server sees the source as the LAN client (the firewall does not double-NAT the source on the hairpin path).

Task 4: Measure the hairpin latency

# Substitute your own values before running:
FW_WAN_IP=203.0.113.10          # the firewall's WAN address

for i in 1 2 3 4 5; do
  curl -k -o /dev/null -s -w "%{time_total}\n" \
    --max-time 5 "https://$FW_WAN_IP:8443/"
done

The output is the round-trip time in seconds. Record the results:

hairpin latency: 0.045, 0.043, 0.044, 0.042, 0.044 (approx 43 ms)

(Your numbers will differ depending on the firewall’s hardware and the network between the test client and the firewall.)

Now make the same request directly to the internal server’s IP on the LAN, bypassing the port forward entirely:

# Substitute your own values before running:
INTERNAL_SERVER_IP=192.0.2.10   # the LAN host behind the port forward

for i in 1 2 3 4 5; do
  curl -k -o /dev/null -s -w "%{time_total}\n" \
    --max-time 5 "https://$INTERNAL_SERVER_IP:443/"
done

Compare the two. The hairpin number should be larger than the direct number — the hairpin path traverses the firewall twice (out to WAN, back in, then to the server), while the direct path is one hop.

Task 5: Capture the hairpin packet path

Start a capture on the WAN interface:

# Substitute your own values before running:
LAN_CLIENT_IP=192.0.2.50        # the LAN host running the curl test

tcpdump -ni igb1 "host $LAN_CLIENT_IP and port 8443" -c 20 \
  -w /tmp/hairpin-wan.pcap

In another session, run the curl. Repeat on the LAN side:

# Substitute your own values before running:
LAN_CLIENT_IP=192.0.2.50        # the LAN host running the curl test

tcpdump -ni igb0 "host $LAN_CLIENT_IP and port 8443" -c 20 \
  -w /tmp/hairpin-lan.pcap

After the runs, inspect both captures:

tcpdump -nr /tmp/hairpin-wan.pcap | head -10
tcpdump -nr /tmp/hairpin-lan.pcap | head -10

The WAN capture should show the SYN from the LAN client to the firewall’s WAN IP, plus the response. The LAN capture should show the rewritten packet from the firewall to the internal server.

The captures together prove that the hairpin path goes through the firewall twice — once on the WAN interface and once on the LAN interface. This is the “loopback” path that gives hairpin its name.

Task 6: Disable NAT reflection

Navigate back to Firewall → Settings → Advanced and set NAT Reflection to Disable. Save.

Verify the reflection rule is removed:

pfctl -s nat -v | grep -A 1 reflect

The grep should be empty. The auto-generated hairpin rule is gone.

From the LAN client, confirm the connection no longer works:

# Substitute your own values before running:
FW_WAN_IP=203.0.113.10          # the firewall's WAN address

curl -k --max-time 5 "https://$FW_WAN_IP:8443/" 2>&1 | head -5

The connection should fail. The packet leaves the LAN, fails to come back, and the client times out.

Task 7: Configure split-horizon DNS

Navigate to Services → Unbound DNS → Overrides → Add.

  • Host: service
  • Domain: lab.local (or whatever your internal domain is)
  • Type: A
  • Value: INTERNAL_SERVER_IP
  • Description: Split-horizon override for lab service

Apply the change.

Verify the override is loaded:

unbound-control list_local_data | grep service

You should see the override entry.

Task 8: Configure the LAN client to use the firewall’s DNS

The split-horizon DNS works only if the client uses the firewall’s Unbound as its resolver. Verify the DHCP server hands out the firewall’s IP as the DNS resolver:

# On the firewall
configctl dhcpd show lan | grep -i 'domain-name-servers'

The output should show the firewall’s LAN IP. If it does not, edit the DHCP server’s LAN configuration to set the DNS server to the firewall’s LAN IP.

For static hosts, configure the client’s DNS manually to the firewall’s LAN IP. Or use drill from the firewall itself (which uses Unbound by default):

drill service.lab.local

The output should show the internal server’s IP.

Task 9: Test the split-horizon connection

From the LAN client:

curl -k --max-time 5 https://service.lab.local:443/ > /dev/null && echo "split-DNS OK"

The connection should succeed. The client resolves the hostname to the internal IP, connects directly via the LAN, and the firewall is not in the path.

Task 10: Measure the split-horizon latency

Repeat the latency test:

for i in 1 2 3 4 5; do
  curl -k -o /dev/null -s -w "%{time_total}\n" \
    --max-time 5 https://service.lab.local:443/
done

Record the results:

split-DNS latency: 0.008, 0.007, 0.008, 0.008, 0.008 (approx 8 ms)

The split-horizon latency should be substantially lower than the hairpin latency. The packet goes from the client to the server directly; the firewall is not in the path.

Task 11: Capture the split-horizon packet path

Repeat the captures:

# Substitute your own values before running:
LAN_CLIENT_IP=192.0.2.50        # the LAN host running the curl test
INTERNAL_SERVER_IP=192.0.2.10   # the LAN host behind the port forward

tcpdump -ni igb0 "host $LAN_CLIENT_IP and host $INTERNAL_SERVER_IP" -c 10 \
  -w /tmp/splitdns-lan.pcap

Run the curl. The capture shows the packets flowing directly between the client and the server. The WAN interface should be silent — no traffic for this connection.

# Substitute your own values before running:
LAN_CLIENT_IP=192.0.2.50        # the LAN host running the curl test

tcpdump -ni igb1 "host $LAN_CLIENT_IP" -c 10 -w /tmp/splitdns-wan.pcap

The WAN capture should be empty for the duration of the test. The split-horizon path keeps the traffic on the LAN.

Task 12: Test the failure modes

For each approach, induce the failure mode and observe the behaviour.

NAT reflection failure mode: client uses a public DNS.

Configure the LAN client to use 8.8.8.8 as its DNS resolver. With NAT reflection still disabled, the client cannot reach the service by the public IP:

curl -k --max-time 5 https://service.lab.local:443/ 2>&1 | head -3

The connection fails because the client resolves to the public IP and there is no reflection. With reflection enabled, the connection would succeed via hairpin. Test both:

# Test with reflection enabled
# (re-enable in the GUI, apply)
curl -k --max-time 5 https://service.lab.local:443/ > /dev/null && echo "hairpin fallback OK"

# Test with reflection disabled
# (disable in the GUI, apply)
curl -k --max-time 5 https://service.lab.local:443/ 2>&1 | head -3

The first works (hairpin), the second fails (no path).

Split-horizon failure mode: stale resolver cache.

From the LAN client (with the firewall as DNS), once successfully resolve the hostname:

drill service.lab.local

Then point the client to a public DNS temporarily and resolve the same hostname:

# Use drill with a public resolver
drill @8.8.8.8 service.lab.local

The public DNS returns the public IP. The client’s resolver cache may now hold the public IP for the duration of the TTL. Switch back to the firewall’s DNS and try again:

drill service.lab.local

If the cache has not expired, drill returns the public IP, not the internal one. The fix is the TTL discipline from the lab callout above, or running unbound-control flush service.lab.local on the firewall.

Task 13: Write the lab report

HAIRPIN VS SPLIT-HORIZON DNS LAB REPORT
========================================

Setup
-----
- Firewall: WAN `<firewall-wan-ip>`, LAN 192.0.2.0/24
- Internal server: `<internal-server-ip>` on LAN
- Service: HTTPS on 443, port forward WAN:8443 → server:443
- Test client: LAN host `<lan-client-ip>`

Hairpin (NAT reflection)
- Latency: <record numbers> ms (avg)
- Packet path: client → firewall WAN → rdr → firewall LAN → server
- Failure mode: client uses public DNS, reflection enabled → still works
- Failure mode: client uses public DNS, reflection disabled → timeout

Split-horizon (Unbound override)
- Latency: <record numbers> ms (avg)
- Packet path: client → server directly (firewall not in path)
- Failure mode: client uses public DNS → resolves to public IP, fails (or hairpin if reflection on)
- Failure mode: stale cache → public IP cached, fix with TTL or flush

Conclusion
----------
- Split-horizon is faster, simpler, and survives WAN outages
- Hairpin is the fallback for clients that cannot use split DNS
- For our environment, split-horizon is the right primary, hairpin is the fallback

Validation

  • NAT reflection was configured, tested, measured, and disabled.
  • Split-horizon DNS was configured, tested, measured, and the override was verified with unbound-control list_local_data.
  • The latency comparison shows split-horizon is faster than hairpin.
  • The packet captures show the firewall in the path for hairpin, and not in the path for split-horizon.
  • Both failure modes were induced and the behaviour was observed.
  • The lab report is written and includes the latency numbers and the operational conclusion.

Cleanup

Restore the firewall to its pre-lab state: disable NAT reflection (if not already disabled) and leave the split-horizon override in place if it is useful for the environment, or remove it if the lab was on a temporary test bench.

# Disable NAT reflection if not already
# Firewall → Settings → Advanced → NAT Reflection → Disable

# Optionally remove the override
# Services → Unbound DNS → Overrides → delete the lab entry

# Confirm the firewall is back to the baseline
pfctl -s nat -v | grep reflect
# expected: empty

If you made the override permanent, refresh the cache:

unbound-control flush service.lab.local

What you learned

  • NAT reflection solves the hairpin problem: internal clients can reach the firewall’s own public IP. The cost is a longer packet path and reflection rules in the NAT table.
  • Split-horizon DNS solves the same problem differently: the internal DNS returns the internal IP, and the firewall is not in the path. The cost is a DNS configuration that must be kept in sync with the internal IP.
  • The two approaches differ on latency (split-horizon is faster), WAN independence (split-horizon works without the WAN), and operational complexity (split-horizon requires DNS discipline).
  • For most production deployments, split-horizon is the right primary and hairpin is the fallback for legacy clients that hard-code the public IP.
  • The failure modes are real: a stale resolver cache, a mis-set client DNS, a client that bypasses the firewall’s resolver. The operator must test those failure modes, not just the happy path.

Deliverables

  • · A NAT reflection configuration that works for an internal client
  • · A split-horizon DNS configuration that returns the internal IP for the same hostname
  • · Latency measurements for both approaches and a written comparison
  • · A short report on which approach to use for this scenario

Verification status

Last reviewed
2026-08-14
Executed end to end
not yet run on hardware

The commands and configuration here have been reviewed against the verified software versions, but nobody has run this lab start to finish on a system meeting its prerequisites. Treat the Expected Outcome as the intended result rather than an observed one, and keep the Cleanup section to hand.