Technical

IO Pressure Stall – Unexplained: Diagnosing Proxmox LXC and ZFS I/O Hangs

IO Pressure Stall – Unexplained: Diagnosing Proxmox LXC and ZFS I/O Hangs

IO pressure stall – unexplained is the most annoying kind because the system looks fine, but containers just freeze for seconds at a time. Usually it’s ZFS doing something dumb or an LXC hitting its IO limit. Here’s how to figure out which one.

Detailed view of a black data storage unit highlighting modern technology and data management.

Check if ZFS is the culprit

ZFS loves to stall IO when it’s doing a scrub, resilver, or just handling a ton of sync writes. Run zpool status and look for any scrub or resilver in progress. If there is one, that’s probably your stall. You can pause it with zpool scrub -p yourpool and see if the stalls stop.

Sync writes are another classic. If you have a database or something doing a lot of fsync, ZFS can choke if your SLOG device is slow or nonexistent. Check zpool iostat -v 1 and look for high latency on the pool. If you see huge numbers, you might need a faster SLOG or to set sync=disabled on datasets where you don’t care about data loss on power failure (like a media library).

LXC IO limits

If you’re running containers, check if you set any IO limits on them. In Proxmox, you can set IO limits on the container’s disk. If a container hits that limit, it’ll stall IO for everything else on that storage. Look at /etc/pve/lxc/<CTID>.conf for lines like iops or mbps. If you see one, try raising or removing it temporarily to see if stalls go away.

Also check the container’s cgroup IO stats. Run cat /sys/fs/cgroup/blkio/lxc/<CTID>/blkio.throttle.io_service_bytes and see if any container is hitting a limit. If so, that’s your stall.

A woman using a laptop navigating a contemporary data center with mirrored servers.

Other things to check

Sometimes it’s not ZFS or LXC. Check dmesg for any storage errors, like SATA link resets or NVMe controller issues. If you see those, it’s a hardware problem. Also check iostat -x 1 on the host and look for devices with high await or %util. If one disk is maxed out, that’s your bottleneck.

If you’re using a single SSD for everything, like in a Jellyfin setup with one SSD, you might just be hitting the disk’s limits. IO pressure stalls are often just the disk being too slow for what you’re asking it to do.

My take

Start with ZFS. It’s the most common cause of unexplained stalls on Proxmox. If that’s not it, check your LXC limits. And if you’re on a single disk, consider adding a second one or moving some workloads to a different storage. Don’t overthink it—just check the obvious stuff first.

Leave a comment

Comments are reviewed before they appear. Your email is never published.