Technical

Proxmox LVM Cache Thin Provisioning — Make It Actually Work

Proxmox LVM Cache Thin Provisioning — Make It Actually Work

Setting up NVMe LVM cache with thin provisioning in Proxmox is a damn mess if you follow the scattered forum threads. The docs never combine cache and thin pools, and the promotion/cleaner settings are left to guesswork. Here’s the sequence that actually sticks.

Focused detail of a modern server rack with blue LED indicators in a data center.

Why bother with LVM cache on thin pools?

You’ve got a big slow HDD array for your VMs and a fast NVMe drive sitting there. LVM cache lets the NVMe act as a read/write cache so your frequently accessed blocks fly. Combined with thin provisioning, you don’t waste space pre-allocating everything. The problem is Proxmox’s installer doesn’t set this up, and the official wiki stops short of tying it all together.

What you need before starting

  • Proxmox installed on its own disk (don’t try to cache the root disk — headache city).
  • Your slow HDD volume group (let’s call it pve) already created.
  • An NVMe drive partitioned but not in any volume group yet.

Step 1: Create the thin pool on the slow disks

If you don’t have a thin pool, make one inside your existing pve volume group. This is where your VM disks will live — thin-provisioned, so they only take up actual used space.

lvcreate -L 500G -T pve/thinpool

Adjust the size to whatever you want the pool to grow to. The -T flag sets it up as a thin pool with metadata and data volumes automatically.

Step 2: Create the cache pool on the NVMe

First, add the NVMe partition to a new volume group — I’m calling it cache_vg. Then create a cache pool LV. This is where the magic happens.

vgcreate cache_vg /dev/nvme0n1p1
lvcreate -L 100G -n cachepool cache_vg
lvconvert --type cache-pool --poolmetadata cache_vg/cachepool_meta cache_vg/cachepool

That last command converts the LV into a cache pool. You might need to create a small metadata LV first if the tool complains — just add --poolmetadatasize 1G to the lvcreate and skip the separate metadata LV.

Focused view of a modern data server rack with blinking lights in a blue-lit environment.

Step 3: Attach the cache to the thin pool

Now the part that forum posts screw up: you attach the cache pool to the thin pool’s data LV, not the thin pool itself. Find the data LV name with lvs — it’s usually pve/thinpool_tdata. Then convert it to a cached LV.

lvconvert --type cache --cachepool cache_vg/cachepool pve/thinpool_tdata

This wraps the thin pool’s data in a cache. All reads and writes go through the NVMe cache from here on.

Step 4: Fix the cache settings or it’ll be slow as hell

Out of the box, the cache mode is writethrough — safe but kills write performance. You want writeback for speed, but that means you need to tune the cleaner and promotion settings so the cache doesn’t get clogged with stale data. Forum threads dance around this; here’s what I’d set:

lvchange --cachemode writeback pve/thinpool_tdata
lvchange --cachepolicy cleaner pve/thinpool_tdata
lvchange --cachesettings 'migration_threshold=2048 random_threshold=4 sequential_threshold=512' pve/thinpool_tdata

Those settings tell the cache to promote blocks more aggressively and clean dirty blocks faster. The defaults are tuned for enterprise workloads, not a homelab where you’re thrashing VMs. You can tweak the numbers, but this gets you out of the mud.

Step 5: Use the thin pool in Proxmox

In the Proxmox web UI, go to Datacenter → Storage → Add → LVM-Thin. Point it at the pve volume group and select thinpool as the thin pool name. Proxmox will see it as a thin-provisioned storage backend, and the cache layer underneath is transparent.

Create a VM disk on that storage and watch lvs -a — you’ll see the cache usage climb as data gets promoted. If it’s not hitting the cache, double-check that you attached it to thinpool_tdata and not the pool itself. That’s the mistake most people make.

Things that’ll bite you

  • Don’t remove the NVMe while the cache is attached. You’ll corrupt the thin pool. If you need to detach, use lvconvert --uncache pve/thinpool_tdata first.
  • The cache LV must be in its own VG. Mixing it with the slow disks’ VG causes all sorts of metadata issues.
  • If you’re using an LXC container with this storage, you might hit terminal weirdness — here’s the fix for that.

That’s it. It’s not pretty, but it works. The Proxmox team really should bake this into the installer — until then, this sequence keeps your VMs fast without wasting NVMe space.

Nguyen

I write two things here: notes from a homelab that mostly runs Proxmox, and essays in Vietnamese about a country I left. The technical posts are whatever broke that week, written down so it costs you less time than it cost me.

All posts →

Leave a comment

Comments are reviewed before they appear. Your email is never published.