iSCSI and multipath

Why this matters in production

iSCSI provides block storage over a network — a SAN. For VM workloads that need high IOPS and don’t want Ceph’s complexity, iSCSI is the classic enterprise answer. Multipath is what makes iSCSI fault-tolerant across redundant fabric paths.

iSCSI in 60 seconds

flowchart LR
  T[iSCSI target / SAN] -->|TCP 3260| I[iSCSI initiator / Proxmox host]
  I --> L[Block device]
  L --> VM[VM disk]

The initiator opens TCP sessions to one or more target portals (different IPs of the SAN). Each session carries SCSI commands. The SAN presents one or more LUNs (Logical Unit Numbers) — block devices.

Configuring the initiator

apt install -y open-iscsi

iscsiadm -m discovery -t sendtargets -p san01.lab.example.com

iscsiadm -m node -l

After login, the LUNs appear as /dev/sdX devices. Each LUN can be used as:

• A physical volume for LVM.
• A direct device for ZFS (a “sparsified” approach).
• A backing device for a Ceph or other storage layer.

Multipath

A single iSCSI session through one cable, one NIC, one switch is a single point of failure. Multipath uses multiple sessions through different paths and aggregates them into one device.

flowchart TB
  subgraph SAN[Storage]
    LUN[LUN]
  end
  LUN --> P1[Portal A]
  LUN --> P2[Portal B]
  P1 --> S1[Switch A]
  P2 --> S2[Switch B]
  S1 --> H1[Host NIC 1]
  S2 --> H2[Host NIC 2]
  H1 --> MP[Multipath device]
  H2 --> MP

apt install -y multipath-tools

systemctl enable --now multipathd && multipath -ll

Adding iSCSI LUNs to Proxmox

GUI: Datacenter → Storage → Add → iSCSI.

pvesm add iscsi iscsi-san --portal san01.lab.example.com --target iqn.2000-01.com.synology:NAS --content images

After adding the iSCSI storage, add individual LUNs:

pvesm add lvm vmdata-lun --vg-name vg-vmdata --base /dev/disk/by-id/dm-uuid-mpath-... --content images,rootdir --shared 1

CHAP authentication

For authenticated iSCSI, configure CHAP:

iscsiadm -m node -T iqn.2000-01.com.synology:NAS -o update -n node.session.auth.authmethod -v CHAP
iscsiadm -m node -T iqn.2000-01.com.synology:NAS -o update -n node.session.auth.username -v <username>
iscsiadm -m node -T iqn.2000-01.com.synology:NAS -o update -n node.session.auth.password -v <password>

Failure modes

Production considerations

Common mistakes

• iSCSI over the management network.
• Single-path iSCSI for production (no redundancy).
• Failing to enable multipath after LUN discovery.
• CHAP credentials in /etc/iscsi/iscsid.conf without securing the file.

Key takeaways

• iSCSI provides block storage over TCP.
• Multipath is required for production fault tolerance.
• Dedicated storage network.

Knowledge check