ZFS replication lab

This lab sets up ZFS-based VM replication between two nodes and walks through a planned failover. It demonstrates a simple DR scenario.

Steps

1. Verify ZFS storage on both nodes

# On both nodes
zpool list
# Expected: a pool named "tank" (or similar) with sufficient capacity

2. Identify a VM to replicate

pvesh get /nodes/<source-node>/qemu/<vmid>/config
# Look for scsi0 referencing the ZFS pool

3. Add the replication target storage

In the GUI: Datacenter → Storage → Add → ZFS.

On the target node, this storage should already exist (the same pool name, but on different physical disks). If not, add it.

4. Schedule replication

In the GUI: Datacenter → Replication → Add.

Source node: pve-01
Target node: pve-02
Schedule: */15 (every 15 minutes)
VM: 100
Rate limit: (none)
Remove pending: yes

Or via CLI:

# Substitute your own values before running. The job id is
# <vmid>-<jobnum>, so the first job for VM 100 is 100-0:
JOB_ID=100-0
TARGET_NODE=pve-02

pvesr create-local-job "$JOB_ID" "$TARGET_NODE" --schedule "*/15" \
  --rate 0 --remove_snapshots

5. Trigger a sync

pvesr schedule-now
# Or wait for the next 15-minute mark

Verify:

pvesr status
# Expected: VM 100 with last_sync time = recent

6. Perform a planned failover

On the target node (pve-02):

# Substitute your own value before running:
VMID=100

# Migrate the replicated VM to the target
qm migrate "$VMID" pve-02 --with-local-disks

7. Verify the failed-over VM

# Substitute your own value before running:
VMID=100

qm status "$VMID"
# Expected: running on pve-02

# Inside the VM
df -h | grep tank
# Expected: ZFS dataset mounted
# Application should be intact

8. Reverse replication

Now that the VM is on pve-02, set up reverse replication:

# The reverse job needs its own job number, so 100-1 rather than 100-0:
JOB_ID=100-1

pvesr create-local-job "$JOB_ID" pve-01 --schedule "*/15" \
  --rate 0

9. Test a planned failover back

# Substitute your own value before running:
VMID=100

qm migrate "$VMID" pve-01 --with-local-disks

Verify both directions work and the VM returns cleanly.

Verification

• Replication completes within the schedule window (15 min in this lab)
• Migrating the VM to the target node works without losing data
• The replicated VM is identical (same disk content, same config)
• Reverse replication completes the loop

Cleanup

# Substitute your own values before running:
VMID=100
JOB_ID=100-0

# Remove replication jobs (repeat for the reverse job, 100-1)
pvesr delete "$JOB_ID"
# Destroy the test VM
qm stop "$VMID" && qm destroy "$VMID"

Notes

• For real DR, combine ZFS replication with PBS backups. ZFS gives you near-zero RPO; PBS gives you immutable, offsite copies.
• Replication is per-VM; a 100-VM environment means 100 replication jobs (or use scheduler groups).
• For multi-site, add an SDN zone with VPN between the sites and configure cross-site replication.

Cleanup details

Replication jobs are listed with pvesr list. Delete with pvesr delete <job-id>.