IPsec proposals and ciphers — encryption, integrity, and DH group choices

The proposal set on a Phase 1 and Phase 2 entry is the most consequential decision in an IPsec deployment. A weak proposal is a vulnerability even if every other layer is correct; a too-restrictive proposal breaks interoperability with third-party peers. The OPNsense UI exposes the same transform lists as strongSwan, with every modern cipher available and the legacy ones marked deprecated. This lesson covers what each part of the proposal means, what the safe choices are in 2026, and how to balance strength with interoperability when the peer is a third-party device whose capabilities are not fully documented.

What a proposal consists of

A proposal is a tuple of (encryption algorithm, integrity algorithm, Diffie-Hellman group) — repeated identically for Phase 1 and Phase 2 except where noted.

For Phase 1, the proposal also covers the PRF (pseudo-random function), which derives keys from the master secret. Modern proposals use SHA-384 or SHA-512 as the PRF. For Phase 2, no separate PRF is negotiated because Phase 2 derives keys from the Phase 1 PRF.

Encryption: AES-GCM and ChaCha20-Poly1305

The production choices for encryption in 2026 are:

AES-256-GCM is the typical default for production. AES-128-GCM is acceptable if the threat model does not extend to quantum adversaries — 128-bit AES has a 256-bit quantum security level, which is fine for any classical adversary. ChaCha20-Poly1305 is the choice on systems that do not have AES-NI acceleration (older ARM CPUs, cheap virtualisation without AES-NI passthrough); it is roughly two to four times faster than AES in software on those systems.

Ciphers the operator should retire:

• 3DES (Triple DES) — 64-bit block size is broken in practice; the de-facto standard requires retirement by 2024. StrongSwan still supports it for legacy peer compatibility, but a tunnel that requires 3DES should be replaced.
• DES — 56-bit key, broken in the 1990s.
• AES-CBC (without an integrity HMAC) — CBC without integrity has padding oracles and ciphertext malleability. Acceptable only when paired with a strong HMAC AND when the deployment needs to interop with a legacy peer that does not support GCM.
• NULL encryption — no encryption. Used only for debugging or for tunnelling protected-only-by-AH traffic. Never use NULL for a production tunnel.

Integrity: SHA-2 and the AEAD shortcut

For non-AEAD modes (CBC, CTR), the integrity algorithm is the HMAC over the packet:

For AEAD modes (GCM, ChaCha20-Poly1305), integrity is built into the encryption operation. The OPNsense UI hides the integrity dropdown when AEAD is selected and uses the AEAD mode’s own authentication tag.

The legacy choice that operators should avoid: SHA-1. SHA-1 is broken for digital signatures (the SHAttered collision in 2017 produced two colliding PDF files), but SHA-1 in HMAC mode is still acceptable. NIST guidance is to phase it out anyway; SHA-256 is the modern default and is universally supported.

Diffie-Hellman groups

The DH group determines the size of the key exchange. In 2026, the modern choices are:

For Phase 1, the typical production choice is DH group 14 (2048-bit), 19 or 20 (256/384-bit ECP), or 31 (Curve25519). Group 31 has become the modern default because it is fast on most hardware, has no known weak-key properties, and is what WireGuard uses internally — a nice symmetry for an operator who is configuring both VPN types on the same firewall.

Perfect Forward Secrecy (PFS)

A Phase 2 SA can be rekeyed with or without a fresh Diffie-Hellman exchange:

• Without PFS: Phase 2 keys are derived from the Phase 1 master secret. If the Phase 1 master secret is ever compromised (a long-term key leak, a poor PRF), all Phase 2 SAs derived from it are also compromised.
• With PFS: A new Diffie-Hellman exchange happens for every Phase 2 rekey. The Phase 2 keys depend on a fresh DH exchange that an attacker who only recovers the Phase 1 master secret cannot compute.

PFS is the operational answer to “what if the long-term credentials leak?”. For a PSK that was generated as 32 random bytes and stored in a secret manager, the risk is low; for a long-lived PSK that many operators have seen over the years, PFS is critical.

OPNsense exposes PFS under the Phase 2 entry — set the “PFS” dropdown to “on” and pick the DH group. Group 14 or 19 is the typical choice. The performance cost is one extra DH exchange per rekey, which happens once per Phase 2 lifetime (an hour, typically) — negligible in practice.

Building a complete proposal

A production proposal for both Phase 1 and Phase 2 on OPNsense looks like this:

Phase 1:

• Authentication method: certificate (RSA-2048 or ECDSA-P256)
• Encryption: AES-256-GCM (or ChaCha20-Poly1305 on ARM/older hardware)
• Integrity: GCM-integrated (no separate HMAC when GCM is selected)
• DH group: 14 or 31 (Curve25519)
• Lifetime: 24 hours

Phase 2:

• Mode: tunnel
• Protocol: ESP
• Encryption: AES-256-GCM (matching Phase 1 for consistency)
• Integrity: GCM-integrated
• PFS: on, DH group 14
• Lifetime: 1 hour

This proposal is strong (256-bit AES-GCM, DH group 14+, PFS) and interoperable (every modern IPsec implementation supports these algorithms). The compromise is that very old third-party peers (pre-2010 Cisco ASA, some legacy Check Point configurations) may not support AES-GCM — those peers force the operator to use AES-CBC plus a SHA-256 HMAC.

Reading the negotiation logs

StrongSwan logs every proposal negotiation in charon.log. When the negotiated proposal does not match the operator’s expectation, the log says so:

$ tail -f /var/log/charon.log | grep -E 'proposal|selected|received'

12:34:56.789012  16[CFG] received proposals:
IKE:AES_CBC_256/HMAC_SHA2_256_192/PRF_HMAC_SHA2_384/ECP_384
IKE:AES_CBC_256/HMAC_SHA2_256_192/PRF_HMAC_SHA2_384/MODP_2048
12:34:56.789013  16[CFG] configured proposals:
IKE:AES_GCM_16_256/PRF_HMAC_SHA2_384/ECP_384
IKE:AES_GCM_16_256/PRF_HMAC_SHA2_384/MODP_2048
12:34:56.789014  16[IKE] no matching proposal found
12:34:56.789015  16[IKE] failed to find acceptable proposal
12:34:56.789016  16[ENC] generating IKE_AUTH response
12:34:56.789017  16[ENC]   IKE_NOTIFY: NO_PROPOSAL_CHOSEN

The diagnostic is unambiguous: received proposals lists what the peer sent; configured proposals lists what the operator set; no matching proposal found is the failure mode. The fix is to align the two sides — typically by adding a fallback algorithm to the operator’s list, or by working with the peer to upgrade to a stronger proposal.

Summary

• Encryption: AES-256-GCM is the modern default; ChaCha20-Poly1305 for ARM/older hardware.
• Integrity: SHA-256 minimum, SHA-384 or SHA-512 for high-value tunnels.
• DH group: 14 or higher; never accept groups 1, 2, or 5; Curve25519 (group 31) is the modern favourite.
• Use AEAD modes (AES-GCM, ChaCha20-Poly1305) so encryption and integrity come from one operation.
• Enable PFS in Phase 2 to make Phase 1 master secret compromise recoverable.
• The proposal mismatch in charon.log (NO_PROPOSAL_CHOSEN) is the diagnostic when proposals do not match on both sides.