Upgrade Proxmox VE 9 to 9.1 Without Breaking Your Cluster

Proxmox VE 9.1 is a point release on the PVE 9.x branch, and upgrading from 9.0 takes roughly 15 minutes per node. By the end you will have an updated hypervisor with native OCI container image support in LXC, a refreshed kernel, and a clean package state on Debian Bookworm — no new repository configuration required.

Key Takeaways

• Upgrade time: 10–15 minutes per node; upgrade one node at a time in a cluster.
• No new repos: PVE 9.1 ships through the existing pve-no-subscription or enterprise channel — no sources.list changes needed.
• OCI LXC: Native OCI image support means any registry image can be pulled directly into LXC without template conversion.
• Backup first: There is no clean rollback path, so snapshot critical VMs before starting.
• Cluster order: Migrate VMs off each node before rebooting; never upgrade two nodes simultaneously or you lose quorum.

Before You Start: Pre-Upgrade Checklist

Skipping prep is how a 15-minute task becomes a 3-hour recovery job. These checks take two minutes.

Verify your current version:

pveversion -v

Expected output on a healthy PVE 9.0 node:

proxmox-ve: 9.0-1 (running kernel: 6.8.4-2-pve)
pve-manager: 9.0.3 (running version: 9.0.3/abcdef01)

If you are on PVE 8.x, stop here. The 8-to-9 path requires a Debian Bullseye-to-Bookworm upgrade first and is a separate process entirely.

Check for held packages:

dpkg --get-selections | grep hold

Held pve-* packages will silently block dist-upgrade. Release any holds before proceeding.

Confirm Debian Bookworm base:

cat /etc/debian_version

You should see 12.x. PVE 9.x requires Debian 12.

Cluster quorum (if applicable):

pvecm status

Look for Quorum: 1 and all expected nodes listed as online. A degraded cluster must be repaired before upgrading.

Take a Backup Now

Before touching any node, verify a recent backup exists. If you rely on Proxmox Backup Server for automated backups, confirm the last scheduled job succeeded in the PBS web UI. For critical VMs, run an on-demand snapshot backup immediately:

vzdump 101 --storage your-backup-storage --mode snapshot --compress zstd

Replace 101 with your VM or container ID. Snapshot mode freezes state without stopping the guest — zero downtime, full recovery point.

Verifying Your APT Repository Configuration

PVE 9.1 is a point release, so it flows through the same repository you already have configured. What you need to verify is that the entries are clean and not accidentally mixing channels.

Check the Proxmox package repo:

cat /etc/apt/sources.list.d/pve-no-subscription.list

Correct entry for the free channel:

deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription

If you have an active subscription:

cat /etc/apt/sources.list.d/pve-enterprise.list

deb https://enterprise.proxmox.com/debian/pve bookworm pve-enterprise

Disable any accidental enterprise entry:

If you are not subscribed but the enterprise list is enabled (no leading #), apt update will throw a 401 authentication error. Silence it:

sed -i 's/^deb /#deb /' /etc/apt/sources.list.d/pve-enterprise.list

Ceph repository (Ceph clusters only):

cat /etc/apt/sources.list.d/ceph.list

For Ceph Reef:

deb http://download.proxmox.com/debian/ceph-reef bookworm no-subscription

Do not mix Ceph Reef and Quincy packages on the same node — verify this before proceeding.

How to Run the Upgrade to PVE 9.1

With repos confirmed, the upgrade is two commands:

apt update && apt dist-upgrade

dist-upgrade rather than upgrade is mandatory. Plain upgrade skips packages with new dependency requirements — exactly the type of change that accompanies a Proxmox point release. During the run:

• Expect 30–80 packages to update on a typical single-node install.
• Watch for The following packages have been kept back: — if any proxmox-ve or pve-kernel-* packages are held back, investigate before confirming.

The config file prompt is where things most often go wrong. When apt asks whether to keep your version or install the package maintainer's version, choose keep your version for:

• /etc/network/interfaces — accepting the new version overwrites your bridge config and your VMs lose networking after reboot. This is the single most common self-inflicted outage during a Proxmox upgrade.
• /etc/hostname and /etc/hosts — these are node-specific; the package default is generic.

You can safely accept the new version for /etc/default/grub if you have not customized it.

Once dist-upgrade completes:

reboot

The reboot is required to activate the updated PVE kernel. Running the old kernel with new userspace binaries produces unpredictable behavior in qm, pct, and the web UI — do not skip it.

Upgrading a Multi-Node Cluster Safely

Never upgrade two nodes simultaneously in a three-node cluster. You need two nodes up to maintain quorum — lose quorum and the cluster locks writes to prevent split-brain, freezing your VMs mid-operation.

Safe upgrade sequence:

1. Live-migrate all VMs off the target node:

   qm migrate 100 pve-node2 --online
   

   Or use the web UI Migrate button on each VM.

2. Upgrade and reboot the now-empty node using the commands above.

3. Verify it rejoins before touching the next node:

   pvecm status
   

   All nodes must show Online.

4. Repeat for each remaining node.

If you manage a larger cluster and want to script this across nodes with built-in quorum checks, Ansible playbooks for Proxmox VE make it straightforward to enforce the correct rolling order.

What Changed in Proxmox VE 9.1

OCI-Based LXC Container Deployment

This is the headline feature. Previous Proxmox LXC used .tar.xz and .tar.zst template archives pulled from the Proxmox template server — a curated but limited set of distributions. PVE 9.1 adds native OCI image support, letting you pull and run any OCI-compliant image from Docker Hub, GHCR, Quay, or a private registry directly as an LXC container.

The pct create command now accepts an oci: prefix:

pct create 200 oci:docker.io/library/debian:12 \
  --hostname oci-debian \
  --net0 name=eth0,bridge=vmbr0,ip=dhcp \
  --storage local-lvm \
  --rootfs local-lvm:8 \
  --unprivileged 1 \
  --start 1

The container boots as a standard LXC — it gets PID namespace isolation, cgroup limits, and full Proxmox management hooks. The difference is purely in the image source: instead of a pre-built Proxmox template, Proxmox pulls OCI layers and converts them at creation time.

Practical impact: If you already build custom base images for your infrastructure — a stripped Debian with your monitoring agent baked in, for example — you can now maintain those images in your existing container registry and pull straight into LXC. The manual template-packaging step is gone.

Note that OCI-sourced LXC containers are still LXC, not Docker. If you need the Docker daemon and Compose workflow inside a container, the configuration is unchanged from before — see Running Docker Inside LXC Containers on Proxmox for that setup.

Kernel and QEMU Updates

PVE 9.1 ships kernel 6.8.12-5-pve, which includes upstream fixes for AMD IOMMU and Intel VT-d interrupt remapping. If you were seeing intermittent GPU passthrough reset failures under PVE 9.0, this kernel bump addresses several of those upstream regressions.

QEMU is updated to 9.0.x in this release, with improved virtio-net throughput and better Windows 11 UEFI emulation accuracy.

Bug Fixes Worth Knowing About

• ZFS dataset operations under sustained write load no longer cause the storage plugin to report spurious I/O errors in the task log.
• Dual-NIC corosync configurations now correctly prefer the designated ring links rather than falling back to the management interface under packet loss.
• The web UI task log viewer no longer loses pagination state when switching nodes in a cluster view.
• pve-firewall correctly handles nft chain naming for bridge interfaces with non-alphanumeric characters in their names.

Post-Upgrade Validation Checklist

After rebooting, run through this in order. It takes five minutes and confirms you have a healthy node before putting workloads back on it.

1. Confirm the new version:

pveversion -v

proxmox-ve: 9.1-1 (running kernel: 6.8.12-5-pve)
pve-manager: 9.1.2

2. Check for broken or pending packages:

dpkg --audit
apt list --upgradable 2>/dev/null | grep -v WARNING

Both should return nothing. If dpkg --audit reports issues:

apt install -f
dpkg --configure -a

3. Verify core Proxmox tools respond:

qm version
pct version

4. Start a test VM and container:

qm start 100 && qm status 100
pct start 101 && pct status 101

Both should reach running within 30 seconds.

5. Storage health:

pvesm status

All storages should show active. For ZFS pools specifically:

zpool status

state: ONLINE on all pools. A DEGRADED or FAULTED state is unrelated to the upgrade but demands immediate attention.

6. Cluster membership:

pvecm nodes

All nodes should show Online.

Troubleshooting Common Upgrade Problems

Packages Held Back After dist-upgrade

If dist-upgrade finishes but reports held-back packages, a dependency conflict exists. Find it:

apt dist-upgrade --dry-run 2>&1 | grep -E "Conf|Remov|Inst|held"
apt-mark showhold

Common culprits: manually installed packages that pin a specific libvirt or qemu-kvm version, or Ceph packages from a mismatched repository. Remove the conflicting package or correct the repository, then re-run dist-upgrade.

Node Fails to Rejoin Cluster After Reboot

systemctl status corosync
systemctl status pve-cluster
journalctl -u corosync -n 50

If corosync cannot reach peers, check three things in order:

1. IP address is unchanged — ip addr vs /etc/corosync/corosync.conf
2. UDP 5404/5405 is not blocked — verify firewall rules on the management interface
3. Clock drift — chronyc tracking shows offset; skew above 1 second will prevent corosync from forming

apt dist-upgrade Interrupted Midway

Do not reboot with a partially upgraded system. Fix the package state first:

dpkg --configure -a
apt install -f
apt dist-upgrade

Only reboot once dist-upgrade exits cleanly with no errors.

Upgrade Timing Reference

Step	Typical Duration
apt update	10–20 seconds
Package download	2–5 minutes
Package installation	3–6 minutes
Reboot and POST	1–3 minutes
Cluster rejoin	30–60 seconds
Total per node	~10–15 minutes

For a three-node cluster including VM live-migration time, budget 45–60 minutes end-to-end.

Conclusion

The PVE 9.0 to 9.1 upgrade is a standard Debian point-release process: verify repos, run apt dist-upgrade, reboot, validate. The OCI LXC capability is the operationally meaningful change — if you already maintain container images in a registry, pulling them directly into LXC removes the last manual step in that workflow. Once you are on 9.1, try pct create with an oci: source on a test container to confirm the feature works in your environment, then revisit your backup schedule to make sure any new OCI-sourced containers are covered.