Storage for HA in Proxmox VE: an in-depth technical overview#
Table of Contents#
- The Role of Storage in High Availability
- Scenario: single node + HA requirement
- Scenario: two nodes + quorum / witness
- Scenario: three or more nodes with shared storage
- Hyperconverged storage (Ceph, etc.)
- External SAN/NAS solutions
- Storage types not suited for HA
- Recommendations and best practices
- Conclusion
1. The Role of Storage in High Availability#
In a Proxmox VE cluster, the High Availability (HA) mechanism ensures automatic restart of virtual machines or containers on other nodes when one of the servers fails. How well HA works directly depends on which storage is used.
Why the type of storage is critical for HA#
When one node goes down, another must:
- have access to the disks of the virtual machine (VM/CT);
- read the configuration;
- start the instance without data desynchronization.
Storage requirements for an HA cluster#
- Shared access - the VM disk must be accessible from multiple nodes.
- Cluster-wide consistency - support for locking, protection against split-brain, synchronous/asynchronous replication.
- Failover without manual intervention - automatic VM startup on another node.
- Network requirements - a dedicated network for storage, 10GbE minimum recommended.
- Data reliability - RAID, Ceph replication, DRBD, fault-tolerant NAS/SAN.
- Support for the Proxmox HA stack - Ceph, iSCSI, NFS, ZFS replication, GlusterFS (not supported starting with Proxmox VE 9).
2. Scenario: single node + HA requirement#
2.1. Local storage with ZFS RAID or hardware RAID#
- Pros: disk protection, simplicity, performance
- Cons: no failover, HA is impossible
2.2. ZFS replication to a “passive” node#
- Allows for a so-called “cold standby,” but not HA - no automatic VM startup
2.3. Replication to an external NAS/storage array#
- Suitable for backups, but the NAS becomes an SPOF (single point of failure)
2.4. Snapshots + vzdump#
- Data recovery only, failover isn’t possible
Conclusion: HA isn’t possible on a single node - a minimum of 2+ nodes with shared storage is required.
3. Scenario: two nodes + quorum#
3.1. The problem with two nodes: split-brain#
- Without a third vote (QDevice), HA doesn’t work correctly.
Solution: QDevice#
- Options: a separate VM, a mini-PC, PBS, Raspberry Pi
- After adding it → 3 votes, HA works
3.2. Storage for HA in a 2-node cluster#
| Storage | HA Compatibility | SPOF | Requires QDevice | Performance |
|---|---|---|---|---|
| ZFS Local + Replication | Yes (partially) | Node | Yes | Medium |
| NFS / iSCSI / NAS | Yes | NAS | Yes | Medium |
| Ceph | Yes | No | Yes | High |
| DRBD9 | Yes | No | Yes | Medium |
4. Scenario: three or more nodes with shared storage#
4.1. Why things get simpler with 3+ nodes#
- Quorum is supported natively
- No split-brain when one node is lost
- Simple HA management
4.2. Shared storage#
4.2.1 NFS#
- File-based connection, suitable for VM/CT disks and ISOs
- The NAS is an SPOF (single point of failure). Losing connectivity to the NAS - all virtual machines shut down.
4.2.2 iSCSI#
- Block-level access, multipath for HA
- SAN SPOF is possible
4.2.3 SMB/CIFS#
- Only for ISOs/templates, not for VM disks
4.3. Hyperconverged storage#
- Ceph: RBD/FS, 3+ nodes, self-healing, block storage
| Architecture | SPOF | Performance | Requirements | Scalability | HA Compatibility |
|---|---|---|---|---|---|
| NFS | NAS | Medium | Low | Medium | Yes |
| iSCSI | SAN | High | Medium | Medium | Yes |
| Ceph | None | High | High | Excellent | Yes |
5. Hyperconverged storage (Ceph, etc.)#
5.1. Ceph#
- RBD for VM disks
- CephFS for ISOs, templates, backups
- Requires 3+ nodes, a dedicated network, SSD/NVMe
6. External SAN/NAS solutions#
| Type | Description | HA Compatibility | Notes |
|---|---|---|---|
| NAS (NFS/CIFS) | File-based storage | Yes (NFS) | CIFS only for ISOs/templates |
| SAN (iSCSI/Fibre Channel) | Block-level access | Yes | Multipath is mandatory |
| Unified Storage | NFS + iSCSI | Yes | NAS/SAN clustering recommended |
Connection examples:
# NFS
Datacenter -> Storage -> Add -> NFS
Server: IP NAS
Export: /volume1/proxmox
Content: Disk image, ISO, backup
Nodes: all nodes
# iSCSI
Datacenter -> Storage -> Add -> iSCSI
Target: IQN LUN
Portal: IP SAN
Nodes: all nodes
Optionally: LVM over iSCSI7. Storage types not suited for HA#
| Storage | HA Compatibility | Notes |
|---|---|---|
| Local LVM/Directory | none | Single node only |
| CIFS/SMB | none | ISOs/templates only |
| Local ZFS without replication | none | Requires DR/replication |
| Rsync / manual replication | none | No automatic failover |
8. Recommendations and best practices#
At least 3 nodes for quorum
Use shared or hyperconverged storage
A dedicated network for storage (10GbE)
Configure an HA group in Proxmox
Regular backups using Proxmox Backup Server
Test failover and monitor the system
Synchronous replication for critical VMs
Document the architecture (so you don’t forget what you designed)
9. Conclusion#
A single node ≠ HA
Two nodes require a QDevice
Three or more nodes is the optimal configuration
The choice of storage depends on your needs and budget: Ceph, DRBD, NAS/SAN (GlusterFS, as a native storage plugin, is no longer supported starting with Proxmox VE 9, but remains available via directory storage or a third-party plugin)
Always make sure to have backups and network redundancy
Following these principles, a Proxmox VE cluster will be resilient, scalable, and provide automatic failover for critical VMs and containers.
