Technical

PBS Backup Host Outage: Evidence Checklist After a Kernel Update

PBS Backup Host Outage: Evidence Checklist After a Kernel Update

Your Proxmox host keeps dropping dead during PBS backups after a kernel update — damn annoying. Before you start randomly rolling back kernels, grab the right evidence. Here’s exactly what to collect so you’re not guessing blind.

Stack of internal hard disk drives for digital storage on white background.

Check the kernel logs first

This is the most direct clue. The kernel ring buffer often catches the last gasp before a crash. Run:

journalctl -k -b -1

That shows kernel messages from the previous boot. Look for oops, panic, hung tasks, or anything about I/O errors near the time of the outage. If the host hard-locks without a trace, you might need to set up netconsole or a serial console to capture it live — but start here.

Grab the PBS task log

On the PBS server, check the backup task that ran when the host died:

proxmox-backup-manager task list --limit 20

Then get the full log for that task ID. It’ll show you how far the backup got — was it during the dirty-bitmap phase? Mid-transfer? That narrows down whether it’s I/O load, network, or something in the QEMU block layer.

Watch the system load during a backup

If you can reproduce it (and you probably can), run this on the Proxmox host while a backup kicks off:

dmesg -w &
vmstat 1 | tee /tmp/backup-stats.log

This dumps memory and I/O stats every second. If you see a spike in blocked processes (the ‘b’ column in vmstat) right before the outage, it’s likely an I/O stall — maybe a storage driver bug or a timeout in the kernel block layer.

Detailed view of computer hard disk drive components, showcasing circuitry and connectors.

Check for hardware or driver issues

New kernels sometimes break driver compatibility. Look at your hardware and driver messages:

lspci -k
dmesg | grep -i error

If you see AER (Advanced Error Reporting) or PCIe errors, that’s a smoking gun. Also check SMART data on the backup target disks — a failing drive can lock up the whole bus.

Compare kernel versions

Boot the previous kernel and see if the problem disappears. That’s the fastest test. To list installed kernels:

dpkg --list | grep pve-kernel

Then pin the working one in grub. If it’s stable, you’ve confirmed a kernel regression. Report it to the Proxmox forum with the evidence above — they’ll actually have something to work with.

My take

Don’t waste hours tweaking PBS settings or blaming network hiccups. This smells like a kernel I/O path issue, and the logs will tell you. If you’re seeing hangs during other heavy I/O, like when VM processes stall when rebooting an adjacent node, it might be the same root cause. Collect the data, isolate the kernel, and move on. This is a homelab — don’t let it eat your weekend.

Leave a comment

Comments are reviewed before they appear. Your email is never published.