Technical

NVMe controller is down on Proxmox — check ASPM and kernel params before blaming the drive

NVMe controller is down on Proxmox — check ASPM and kernel params before blaming the drive

That nvme nvme0: controller is down; will reset: CSTS=0xffffffff, PCI_STATUS=0x10 error on Proxmox looks like a dead drive, but it’s usually PCIe power management screwing things up. Don’t go RMA’ing the NVMe yet — try these steps first.

Detailed view of a server rack with a focus on technology and data storage.

Check dmesg for the real story

Run dmesg | grep -i nvme right after the error hits. You’ll probably see a bunch of AER (Advanced Error Reporting) messages, maybe some PCIe Bus Error lines, and then the controller drops. If the drive comes back after a reset, it’s almost certainly a link state issue, not a hardware failure.

Disable ASPM — the usual suspect

Active State Power Management lets the PCIe link drop to low power, and some NVMe controllers just can’t handle it on Linux. The fix is adding pcie_aspm=off to your kernel command line.

  1. Edit /etc/default/grub and add pcie_aspm=off to the GRUB_CMDLINE_LINUX_DEFAULT line.
  2. Run update-grub and reboot.

If you’re booting with systemd-boot (common on newer Proxmox installs), edit the loader entry directly in /boot/efi/loader/entries/ and append the parameter.

Detailed close-up of an old hard drive showing internal mechanical parts and electronic circuits.

Kernel boot parameters to test

If ASPM off doesn’t cut it, try these one at a time. They’re less aggressive and might keep your power savings:

  • nvme_core.default_ps_max_latency_us=0 — disables NVMe power states entirely.
  • pci=noaer — shuts up AER reporting, which sometimes stops the reset loop.
  • pci=nomsi — forces INTx instead of MSI interrupts; some controllers are picky.

Add them the same way as ASPM off, one per boot, and see which one sticks. My money’s on default_ps_max_latency_us=0 — it’s fixed this exact crap on a couple of WD and Samsung drives I’ve dealt with.

When it actually is the drive

If none of that helps, check the NVMe’s SMART data with nvme smart-log /dev/nvme0. Look for media_errors or num_err_log_entries climbing. Also, reseat the drive — a slightly loose M.2 connection can cause link down events. But honestly, I’d exhaust the kernel params first.

This is annoying, I know. The Proxmox forum thread on this is a mess of half-answers. If you’ve got a similar setup, also check Proxmox Hypervisor Restarts itself — check logs and watchdog first because sometimes these NVMe errors trigger a full host reboot.

Leave a comment

Comments are reviewed before they appear. Your email is never published.