## Introduction This guide gives daily operators a focused, safe set of Proxmox VE commands with practical, copy-paste-ready examples. It is written for developers, DevOps consultants, and technical startup teams who need a reliable routine for node health checks, VM and container lifecycle, storage and backups, cluster and Ceph validation, and quick troubleshooting. Operate conservatively: prefer read-only checks first, then make reversible, verified changes. Test examples in a lab or during a maintenance window before production. ## Prerequisites and safety - Environment: Proxmox VE 7.x or 8.x on Linux nodes; commands run on the node shell. - Privileges: root on the node (or sudo with equivalent rights). - Network: shell access to each node; cluster commands from any quorum member. - IDs used below are examples (VMID/CTID like 100/101/200/201). Substitute your IDs. - Always confirm storage free space and cluster health before lifecycle or migration actions. - Keep a recent, verified backup before disruptive operations. ## Workflow overview Follow this operator-friendly order in day-to-day work: - Verify node health and services. - Inventory VMs and containers. - Perform safe lifecycle actions (graceful first, verify state). - Check storage capacity and backups. - Validate cluster and Ceph health. - Inspect networking and targeted logs. - Use API helpers for quick reports. Tip: after any change, re-run health and storage checks to confirm steady state. ## Nodes and services Start read-only, then drill down if something looks off. - Proxmox VE version and component versions: pveversion -v Expected: lists pve-manager, pve-kernel, qemu-server, lxc-pve, etc. Mismatched versions across nodes may indicate pending updates or partial upgrades. - Node status summary: pvenode status Expected: CPU load, memory usage, KSM, swap, uptime. High swap with free RAM can point to swappiness or a ballooning guest. - Storage status overview across configured storages: pvesm status Expected: one line per storage with total/used/avail. If a storage shows unknown or error, inspect that backend before doing backups or restores. - Core Proxmox services: systemctl status pvedaemon.service systemctl status pveproxy.service systemctl status pvestatd.service systemctl status pve-cluster.service Expected: Active: active (running). If pve-cluster is degraded or stopped, cluster tasks and the API may fail. - Recent logs for the API proxy (last hour): journalctl -u pveproxy -b --since "1 hour ago" --no-pager Look for listener errors on port 8006 or authentication failures. ### LXC template management (read-only and safe) pveam update pveam available pveam list local Use available to choose a template; list shows what is already downloaded to a storage named local. ## VMs and containers ### Inventory and status - List all QEMU virtual machines: qm list Sample output lines: VMID, NAME, STATUS, MEM, BOOTDISK, PID. A running VM shows status running and a PID. - Check status for a specific VM (replace 100 with your VMID): qm status 100 Expected: status: running or status: stopped. - List all LXC containers: pct list - Check status for a specific container: pct status 101 ### Graceful lifecycle examples Prefer graceful shutdowns. Verify state after each action. - VM: request ACPI shutdown, wait up to 60s, then check state: qm shutdown 100 --timeout 60 qm status 100 If the guest ignores ACPI, shutdown may time out. As a last resort during a maintenance window, use qm stop 100 (hard power-off). Avoid during normal operations. - VM: start and verify: qm start 100 qm status 100 - Container: request shutdown, then verify: pct shutdown 101 --timeout 60 pct status 101 - Container: start and verify: pct start 101 pct status 101 - Inspect configuration (read-only): qm config 100 pct config 101 - Safe in-guest check for a running container: pct exec 101 -- uptime Caution: qm stop and pct stop are hard stops. Prefer shutdown unless you have a clear rollback plan and a window. If a VM is stuck due to a stale lock after a failed task, clear it only after confirming there is no active disk operation: qm unlock 100 Failure mode hints: - Task hangs: check journalctl -xe and /var/log/syslog in-guest if reachable. - Lock persists: ensure no vzdump or snapshot task is running; then unlock. ### Lifecycle command comparison
ActionUse whenEffectRisk/Notes
qm shutdown / pct shutdownNormal maintenanceGuest-initiated graceful shutdownMay time out if guest tools/ACPI not responding
qm stop / pct stopEmergency onlyHard power-offRisk of data loss; prefer only within change window
qm resetGuest unresponsive but must stay onACPI reboot (like power cycle)Can corrupt in-guest FS if buffers not flushed
## Storage, backup, restore ### Storage checks - Proxmox storage view: pvesm status - List volumes in a storage named local: pvesm list local - OS-level free space: df -h Investigate any storage with less than 15% free space before starting backups or restores. ### Create a backup (vzdump) and verify - Backup VM 100 to storage local using snapshot mode and Zstandard compression, with a simple note: vzdump 100 --storage local --mode snapshot --compress zstd \ --notes-template "auto:%{date}" --mailto "" Expected: task log ending with INFO: Finished Backup of VM 100. If snapshot mode is unsupported by the storage, vzdump fails with a message about snapshot capability. - Verify the backup file exists: ls -lh /var/lib/vz/dump/ Expected filename pattern: vzdump-qemu-100-YYYY_MM_DD-HH_MM_SS.vma.zst (for VMs) or vzdump-lxc-101-...tar.zst (for containers). Common failure modes and responses: - Not enough space: free space on the target storage or choose another storage with --storage. - Snapshot not supported: retry during a window with --mode stop. - Guest lock remains: ensure no tasks running, then qm unlock 100 or pct unlock 101 (only if safe). ### Restore a VM backup in a lab to a new VMID Restore to avoid ID or MAC collisions. Example targets VMID 200 on storage local-lvm. qmrestore /var/lib/vz/dump/vzdump-qemu-100-YYYY_MM_DD-HH_MM_SS.vma.zst 200 \ --storage local-lvm --unique 1 --unique 1 regenerates MAC addresses and related identifiers. Then verify: qm list | grep "^\s*200\b" qm start 200 ### Container backup and restore - Backup CT 101: vzdump 101 --storage local --mode snapshot --compress zstd - Restore CT backup to new CTID 201: pct restore 201 /var/lib/vz/dump/vzdump-lxc-101-YYYY_MM_DD-HH_MM_SS.tar.zst \ --storage local-lvm ### Backup mode comparison Mode Guest state Requirement Typical use snapshot Running Storage with snapshot support (LVM-thin, ZFS, Ceph RBD) Minimal downtime backups suspend Pauses guest QEMU guest agent recommended Short I/O quiesce without full stop stop Powered off Maintenance window When snapshot is unsupported or for simplest consistency ## Cluster and Ceph ### Cluster health and membership - Cluster manager status and quorum: pvecm status Expected highlights: Quorum OK, number of nodes, ring status. If Quorum is lost, avoid migrations and disruptive changes. - List cluster nodes: pvecm nodes Shows Nodeid, Votes, Name, and online state. Nodes marked DEAD or with 0 votes should be investigated (network/Corosync). ### Ceph health (if enabled) - High-level Ceph health from Proxmox integration: pveceph status Expected: health HEALTH_OK, osd: up/in counts, pg states. If health is WARN/ERR (e.g., degraded, undersized PGs, full ratio exceeded), postpone migrations and address storage issues first. Operate only when pvecm status shows quorum and pveceph status reports healthy. ## Networking and logs ### Quick network checks - Interfaces (brief): ip -br a - Routing table: ip r - Proxmox web UI listener on port 8006: ss -lntp | grep ":8006" - Basic reachability to a gateway: ping -c 3 192.0.2.1 If UI is unreachable but pveproxy is running, confirm firewall and reverse proxies. ### Targeted logs for recent issues journalctl -u pveproxy -b --since "30 min ago" --no-pager journalctl -u pvedaemon -b --since "30 min ago" --no-pager journalctl -u pve-cluster -b --since "30 min ago" --no-pager If you run the Proxmox firewall, check status without changing rules: pve-firewall status ## API helpers with pvesh Use the built-in shell client to query the API for quick reports and scripts. - All cluster resources (VMs, containers, nodes, storages): pvesh get /cluster/resources - Only VMs: pvesh get /cluster/resources --type vm - VM 100 configuration from node pve1: pvesh get /nodes/pve1/qemu/100/config Tip: explore endpoints with pvesh ls / to discover the API tree. pvesh uses your local node credentials; no external tokens needed for on-node queries. ## Local pilot plan Goal: one safe, measurable run you can inspect locally before any production work. - Baseline checks (read-only) - Run: pveversion -v, pvenode status, pvesm status. - Save outputs to a dated log file for comparison. - Inventory - Run: qm list, pct list. - Identify a noncritical VM (e.g., VMID 100) for lifecycle tests. - Controlled lifecycle - Run: qm shutdown 100 --timeout 60, then qm status 100 (expect status: stopped). - Start: qm start 100, confirm with qm status 100 (expect status: running). - Backup and verify - Run: vzdump 100 --storage local --mode snapshot --compress zstd. - Verify file exists with ls -lh /var/lib/vz/dump/ and note its size. - Test restore in lab - Restore to a new VMID (e.g., 200) with qmrestore ... 200 --storage local-lvm --unique 1. - Confirm with qm list | grep 200 and boot with qm start 200. - Cluster and Ceph checks (if applicable) - Run: pvecm status, pvecm nodes, and pveceph status. - Networking spot-check - Run: ip -br a, ip r, and ss -lntp | grep :8006. Exit criteria: - All commands ran without errors; outputs captured. - Test VM was gracefully cycled and is reachable again. - Backup file exists and a lab restore boots. - Cluster shows quorum; Ceph reports healthy if in use. Rollback considerations: - If a shutdown hangs, avoid hard stops unless you have a maintenance window; try in-guest shutdown first. - If a restore conflicts with IDs or storage, choose a new ID or storage and retry. - If space is low, expand or free storage before retrying backups or restores. ## Conclusion This operator-safe workflow gives you a dependable Proxmox routine: start with read-only health checks, perform one controlled change at a time, verify outcomes, and keep short logs for comparison. With the commands and examples above, you can confidently handle daily operations, produce quick status reports with pvesh, and run tested backups and restores without surprises. Once the pilot is repeatable, standardize it as your runbook and extend it to more workloads as your environment grows.