Technical

Clustered iSCSI as LVM on LVM-Thin SAN: Proxmox Setup with TrueNAS

Clustered iSCSI as LVM on LVM-Thin SAN: Proxmox Setup with TrueNAS

Clustered iSCSI as LVM on LVM-thin SAN means no cluster filesystem, just plain LVM on each node. The forum threads make this sound complicated, but it’s not—use LVM, not GFS2 or OCFS2, and let Proxmox handle the locking.

Detailed view of fiber optic cables connected to a server rack, showcasing modern technology.

Why not a cluster filesystem?

People get hung up on GFS2 or OCFS2. Don’t. Those are for shared block storage where multiple nodes write to the same filesystem at the same time. For VMs, you don’t need that—Proxmox uses LVM on top of the iSCSI LUN, and each VM disk is a logical volume. The cluster just needs to know which node owns which LV, and Proxmox’s built-in locking handles that. Adding a cluster filesystem just adds overhead and a whole new set of problems.

A complex network of cables in a data center with a monitor in the foreground.

Setup with TrueNAS

First, create a zvol on TrueNAS and share it via iSCSI. Then on each Proxmox node, install the iSCSI initiator and connect to the target. Here’s the basic flow:

apt install open-iscsi
iscsiadm -m discovery -t st -p 192.168.1.100
iscsiadm -m node --login

After login, you’ll see a new block device, like /dev/sdb. Create a physical volume and volume group on it:

pvcreate /dev/sdb
vgcreate san-vg /dev/sdb

Then in Proxmox, add the LVM storage pointing to that VG, and check the “shared” box. That’s it. The cluster will see the same LVM metadata because it’s on the shared LUN.

Gotchas

The big one: don’t use thin provisioning on the SAN side if you’re using LVM-thin on top. Double thin is a recipe for out-of-space errors and weird performance. Either thick-provision the zvol and use LVM-thin on Proxmox, or thin-provision the zvol and use regular LVM. Pick one.

Also, make sure all nodes see the same LUN ID. TrueNAS sometimes changes it if you recreate the target. And don’t forget to enable multipath if you have redundant paths—otherwise a cable pull kills your VMs.

If you’re already running a Proxmox cluster with NFS, this setup is similar in concept but with block storage. For more on cluster storage quirks, check out Proxmox Cluster NFS: Frontend Connects, Backend Fails.

Leave a comment

Comments are reviewed before they appear. Your email is never published.