Lab: Capture packets with tcpdump and diagnose a handshake failure

This lab uses tcpdump to capture live TCP handshakes under three scenarios: success, connection refused, and connection timed out. By the end you will have pcap files and annotated output for each scenario.

Objective

By the end of this lab, you can:

• Use tcpdump to capture packets with the right filters.
• Read the captured output and identify handshake stages.
• Distinguish RST (refused) from no response (timeout).
• Extract timing and key state from a pcap file.

Architecture

You need:

• A host with tcpdump installed (apt install tcpdump or dnf install tcpdump).
• A reachable remote host with a listening service (e.g. an SSH server you can reach).
• Optionally a second host to test “no listener” and “firewall drop” scenarios.

If you only have one host, you can simulate using loopback or by stopping a service temporarily.

Tasks

Task 1: Capture a successful handshake

sudo tcpdump -i eth0 -nn -s 0 -w /tmp/success.pcap 'tcp port 22'

# In a second terminal, ssh to a host:
ssh user@10.0.0.5

# Stop the capture with Ctrl-C after a few seconds.

The capture contains the SSH session. The handshake is the first three packets:

12:34:56.789 IP 10.0.0.10.51234 > 10.0.0.5.22: Flags [S], seq 1000
12:34:56.792 IP 10.0.0.5.22 > 10.0.0.10.51234: Flags [S.], seq 2000, ack 1001
12:34:56.793 IP 10.0.0.10.51234 > 10.0.0.5.22: Flags [.], ack 2001

The flags are:

• [S] = SYN (client to server)
• [S.] = SYN-ACK (server to client)
• [.] = ACK (client to server, completing the handshake)

Task 2: Capture connection refused

Stop a service or pick a port nothing is listening on:

sudo tcpdump -i eth0 -nn -s 0 -w /tmp/refused.pcap 'tcp port 2222'

# In a second terminal:
nc -vz 10.0.0.5 2222

# Stop capture.

The capture should show SYN, then RST:

12:34:56.789 IP 10.0.0.10.51234 > 10.0.0.5.2222: Flags [S], seq 1000
12:34:56.790 IP 10.0.0.5.2222 > 10.0.0.10.51234: Flags [R], seq 0, ack 1001

The RST (Reset) flag is [R]. The server replies with RST because nothing is listening on port 2222.

Task 3: Capture connection timeout

Pick a host that does not respond (e.g. an IP with no host behind it, or behind a firewall that drops packets):

sudo tcpdump -i eth0 -nn -s 0 -w /tmp/timeout.pcap 'tcp port 22 and host 192.0.2.1'
# 192.0.2.1 is the RFC 5737 TEST-NET-1 range; safe to probe.

nc -vz -w 5 192.0.2.1 22

# Stop capture.

The capture shows SYN, then retries, then no response:

12:34:56.789 IP 10.0.0.10.51234 > 192.0.2.1.22: Flags [S], seq 1000
12:34:57.793 IP 10.0.0.10.51234 > 192.0.2.1.22: Flags [S], seq 1000
12:34:59.797 IP 10.0.0.10.51234 > 192.0.2.1.22: Flags [S], seq 1000
...

The retransmitting SYN (same seq) with no SYN-ACK or RST is the timeout signature. The remote is unreachable (firewall drop or host down).

Task 4: Read pcap files

sudo tcpdump -nn -r /tmp/success.pcap
sudo tcpdump -nn -r /tmp/refused.pcap
sudo tcpdump -nn -r /tmp/timeout.pcap

You can read captured pcaps without re-capturing. The -r flag reads from a file. Filter on read:

Mind the direction. In Task 1, 10.0.0.5 is the SSH server, so every packet it sends has source port 22 and a destination port equal to the client’s ephemeral port:

# Server -> client
sudo tcpdump -nn -r /tmp/success.pcap 'src 10.0.0.5 and src port 22'

# Client -> server
sudo tcpdump -nn -r /tmp/success.pcap 'dst 10.0.0.5 and dst port 22'

# Both directions of the same conversation
sudo tcpdump -nn -r /tmp/success.pcap 'host 10.0.0.5 and port 22'

# Now run the wrong one and watch it print nothing at all:
sudo tcpdump -nn -r /tmp/success.pcap 'src 10.0.0.5 and dst port 22'

That last filter is syntactically valid, so tcpdump accepts it without complaint — and it can never match, because a packet from 10.0.0.5 has source port 22, not destination port 22. This is the most common tcpdump mistake there is. Silence means “nothing matched the filter”, not “there was no traffic.” When a capture comes back empty, suspect your filter before you suspect the network.

sudo tcpdump -nn -r /tmp/success.pcap 'tcp[tcpflags] & (tcp-syn) != 0'

# Show packets with payload:
sudo tcpdump -nn -r /tmp/success.pcap -X | head -50

# Hex dump only:
sudo tcpdump -nn -r /tmp/success.pcap -xx | head -20

Task 5: Annotate the output

For each pcap, write a one-line description of every packet in the handshake:

12:34:56.789 SYN  -> SYN with seq=1000, client to server
12:34:56.792 SYN-ACK  <- SYN with seq=2000, ACK 1001, server to client
12:34:56.793 ACK  -> ACK 2001, completing handshake
...

Compare across the three scenarios. Note which packets are present in success vs refused vs timeout.

Task 6: Apply the diagnostic discipline

For each scenario, write down:

1. Symptom: what the user reported.
2. Layer: where the failure happens (4 = transport, 1 = physical).
3. Evidence: which tcpdump output proved the failure.
4. Likely cause: what to investigate next.

For example:

Scenario: refused
Symptom: nc returns "Connection refused"
Layer: 4 (transport)
Evidence: tcpdump shows [S] then [R]
Likely cause: nothing listening on that port, or firewall
  rejected with RST

Validation

• Three pcap files exist (/tmp/success.pcap, /tmp/refused.pcap, /tmp/timeout.pcap).
• Each pcap is annotated with the packets and their meanings.
• Every filter in Task 4 either returns a non-empty result, or you can state in one sentence why it cannot match.
• The diagnostic discipline output covers all three scenarios.

Cleanup

Remove the pcaps if they are not needed:

rm -f /tmp/success.pcap /tmp/refused.pcap /tmp/timeout.pcap

What you learned

• tcpdump captures the wire. The signature of a refused connection is [S] followed by [R].
• The signature of a timeout is retransmitting [S] with no reply.
• Reading pcap files with -r lets you reanalyse captures later.