A local Linux lab is a controlled space on your workstation where you can try ideas, test changes, and troubleshoot issues without risking shared environments. You can reset it to a known-good state in minutes, observe real system behavior, and practice repeatable workflows that transfer to production. This guide focuses on a safe, verifiable setup using Linux virtual machines, concrete commands, expected outputs, and reliable rollback paths. You will build a small two-VM topology (client and server) suitable for service testing, network checks, and hands-on experiments. ## What you will build - Host : Linux with KVM/libvirt - VM 1 (lab-srv) : 2 vCPU, 2 GB RAM, 20 GB disk running Nginx as a test HTTP service - VM 2 (lab-cli) : 1 vCPU, 1 GB RAM, 10 GB disk for client-side testing (curl, ping, traceroute) - Networking : libvirt default NAT network (internet access through host), optional host-only later - Safety : snapshots on both VMs to enable fast rollback ## Why start small A tightly scoped lab is measurable, fast to verify, and easy to reset if you break it. That means you learn faster and spend less time undoing mistakes. ## Version and environment inventory Before building the lab, capture what you are starting with and pick a small, inspectable scope. The first pilot should be narrow and measurable: for example, start one server VM running Nginx and one client VM that can curl it over a private network. ### Record host OS and versions (run on host) cat /etc/os-release uname -r lscpu | grep -E 'Virtualization|Model name' systemd --version | head -1 bash --version | head -1 virsh --version qemu-system-x86_64 --version | head -1 ### Check hardware and resources (run on host) lsmod | grep -E 'kvm_(intel|amd)' free -h lsblk -o NAME,SIZE,TYPE,MOUNTPOINT ip addr ip route ### Decide the initial topology - One NAT network via libvirt (default) so VMs can pull packages from the internet through the host - Two VMs to keep failure domains simple - Snapshots immediately after base install and after service installation ### Example inventory Item Example value Host OS Ubuntu 22.04 LTS, kernel 5.15 CPU 8-core with virtualization support (VT-x/AMD-V) RAM 16 GB total, 3 GB reserved for lab Disk 200 GB SSD, 40 GB free for lab images Virtualization KVM/libvirt 8.x, QEMU 6.x Lab VMs lab-srv (2 vCPU/2 GB/20 GB), lab-cli (1 vCPU/1 GB/10 GB) Network libvirt default NAT 192.168.122.0/24 ## Safe configuration path The design goals: - Isolate experiments from the host - Keep the network scope tight - Enable fast rollback with snapshots The steps below assume a Debian/Ubuntu-like host. Where useful, Fedora/RHEL-like alternatives are shown. ### 1. Install KVM and libvirt (run on host) # Debian/Ubuntu sudo apt update sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients virt-manager # Fedora/RHEL sudo dnf install -y @virtualization virt-install virt-manager sudo systemctl enable --now libvirtd Add your user to the libvirt group and re-evaluate your session: sudo usermod -aG libvirt $USER newgrp libvirt ### 2. Verify virtualization on host # Should return non-empty if supported lsmod | grep -E 'kvm_(intel|amd)' # libvirt default network should be present virsh net-list --all virsh net-start default || true virsh net-autostart default Expected: the default network shows as active. If not, see Failure Modes later. ### 3. Create VM disks (run on host) # Adjust paths and sizes as needed sudo qemu-img create -f qcow2 /var/lib/libvirt/images/lab-srv.qcow2 20G sudo qemu-img create -f qcow2 /var/lib/libvirt/images/lab-cli.qcow2 10G ### 4. Install lab-srv from an ISO # Replace ISO path and OS variant as appropriate sudo virt-install \ --name lab-srv \ --memory 2048 \ --vcpus 2 \ --disk path=/var/lib/libvirt/images/lab-srv.qcow2,format=qcow2,bus=virtio \ --cdrom /var/lib/libvirt/boot/ubuntu-22.04-live-server-amd64.iso \ --network network=default,model=virtio \ --os-variant ubuntu22.04 \ --graphics vnc \ --noautoconsole Complete the installation in the console (virt-manager can attach to the VM if needed) and set: - Hostname: lab-srv - User: labops (constructed), with sudo - Network: DHCP via default NAT ### 5. Install lab-cli similarly sudo virt-install \ --name lab-cli \ --memory 1024 \ --vcpus 1 \ --disk path=/var/lib/libvirt/images/lab-cli.qcow2,format=qcow2,bus=virtio \ --cdrom /var/lib/libvirt/boot/ubuntu-22.04-live-server-amd64.iso \ --network network=default,model=virtio \ --os-variant ubuntu22.04 \ --graphics vnc \ --noautoconsole Set hostname lab-cli , user labops , DHCP network. ### 6. Post-install basics on each VM (run inside each VM) # Update packages sudo apt update && sudo apt -y upgrade # Debian/Ubuntu # or sudo dnf -y update # Fedora/RHEL # Confirm hostname and network hostnamectl ip addr Optionally set predictable hostnames in /etc/hosts on both VMs so the names resolve. Replace the IPs with your actual VM addresses: sudo sh -c 'echo "192.168.122.101 lab-srv" >> /etc/hosts' sudo sh -c 'echo "192.168.122.102 lab-cli" >> /etc/hosts' (You can also rely on plain IPs; this is for convenience.) ### 7. Baseline snapshots (run on host) virsh shutdown lab-srv && virsh shutdown lab-cli # Wait for both to be shut off virsh dominfo lab-srv | grep State virsh dominfo lab-cli | grep State # Create snapshots named baseline virsh snapshot-create-as lab-srv baseline "Post-OS baseline" --disk-only --atomic virsh snapshot-create-as lab-cli baseline "Post-OS baseline" --disk-only --atomic # Start both VMs again virsh start lab-srv virsh start lab-cli Note: If your storage pool does not allow disk-only snapshots, omit --disk-only . Ensure your qcow2 images support snapshots. ### 8. Install a simple test service on lab-srv (inside lab-srv) # Install Nginx sudo apt -y install nginx # Debian/Ubuntu # or sudo dnf -y install nginx # Fedora/RHEL # Start and enable sudo systemctl enable --now nginx # Check status systemctl is-active nginx && systemctl is-enabled nginx If you use a host firewall on the VM: # UFW example (Ubuntu) sudo ufw allow 80/tcp sudo ufw reload # firewalld example (Fedora/RHEL) sudo firewall-cmd --permanent --add-service=http sudo firewall-cmd --reload ### 9. Take a second snapshot after service install (run on host) virsh shutdown lab-srv virsh snapshot-create-as lab-srv svc-nginx "Nginx installed and running" --disk-only --atomic virsh start lab-srv ## Verification and diagnostics Verification means observable evidence: the hypervisor is running, the VMs boot, they can talk on the intended network, and the service responds with expected content. Use these checks and compare to the Expected result notes. ### 1. Host-level checks (run on host) # Libvirt active networks virsh net-list # Running VMs virsh list --all Expected: default network is active; both lab-srv and lab-cli are running. ### 2. Find VM IP addresses Inside each VM, run: ip -4 addr show dev eth0 | awk '/inet /{print $2}' Expected: IPs in the 192.168.122.0/24 range (libvirt default) unless you changed it. ### 3. Basic connectivity (inside lab-cli) # Replace with lab-srv IP LAB_SRV_IP=192.168.122.101 ping -c 3 $LAB_SRV_IP Expected: 0% or low packet loss. ### 4. HTTP service check (inside lab-cli) # If curl is missing: sudo apt -y install curl || sudo dnf -y install curl curl -s -o /dev/null -w "%{http_code}\n" http://$LAB_SRV_IP/ Expected: 200 Get and view a few bytes to prove content: curl -s http://$LAB_SRV_IP/ | head -n 5 Expected: HTML including a reference to nginx or your custom index page. ### 5. Service and logs (inside lab-srv) systemctl status nginx --no-pager journalctl -u nginx --since "-5m" --no-pager Expected: active (running), recent 200 responses in logs if logging is enabled. ### 6. Time and entropy (inside both VMs) timedatectl | sed -n '1,4p' Expected: System clock synchronized: yes. If not, enable systemd-timesyncd or chrony as appropriate. ### Quick reference table
CheckCommandExpected result
Hypervisor OKlsmodkvm_intel or kvm_amd present
Network activevirsh net-listdefault active
VM runningvirsh list --alllab-srv, lab-cli running
IP assignedip -4 addr192.168.122.x addresses
Nginx statussystemctl is-active nginxactive
HTTP reachablecurl -w "%{http_code}"200
## Failure modes and recovery When something fails, identify the symptom fast, confirm with a targeted command, and repair or roll back.
SymptomLikely causeWhat to runFix
virt-install fails to start VMCPU virtualization offlscpuEnable VT-x/AMD-V in BIOS/UEFI
Permission denied to libvirtUser not in libvirt groupidAdd user to libvirt, newgrp or re-login
No VM IP addresslibvirt network downvirsh net-listvirsh net-start default; autostart it
VM cannot reach internetDNS or NAT issuecurl http://example.comCheck /etc/resolv.conf, restart libvirt network
Cannot reach NginxFirewall on lab-srvss -tlnpOpen port 80 in ufw/firewalld
403/404 from NginxWrong docroot or SELinuxjournalctl -u nginxFix config; on SELinux, use proper contexts
Disk full on hostLarge images/snapshotsdf -h /var/lib/libvirt/imagesDelete old snapshots or expand disk
Snapshot revert failsVM runningvirsh dominfoShutdown VM, then snapshot-revert
### Recovery commands and patterns ### Rebuild or restart libvirt default network (run on host) virsh net-destroy default || true virsh net-undefine default || true # Recreate default network from template sudo cp /usr/share/libvirt/networks/default.xml /tmp/default.xml sudo virsh net-define /tmp/default.xml virsh net-start default virsh net-autostart default ### Snapshot rollback (run on host) # Always shutdown first virsh shutdown lab-srv && virsh shutdown lab-cli # Revert to baseline virsh snapshot-revert lab-srv baseline --running virsh snapshot-revert lab-cli baseline --running If you want to keep the current broken state for later study, create a snapshot before reverting: virsh snapshot-create-as lab-srv pre-revert "State before revert" ### Service reset (inside lab-srv) # Restore stock config if you broke nginx.conf sudo cp /etc/nginx/nginx.conf.default /etc/nginx/nginx.conf 2>/dev/null || true sudo nginx -t sudo systemctl restart nginx If the distribution does not provide a .default config, reinstall the package: # Debian/Ubuntu sudo apt -y install --reinstall nginx # Fedora/RHEL sudo dnf -y reinstall nginx ### Time sync fixes (inside any VM) # Debian/Ubuntu sudo systemctl enable --now systemd-timesyncd # Fedora/RHEL sudo dnf -y install chrony sudo systemctl enable --now chronyd ### Free space recovery (host) # List snapshots virsh snapshot-list lab-srv # Delete unneeded snapshots by name virsh snapshot-delete lab-srv svc-nginx # Confirm disk usage sudo du -sh /var/lib/libvirt/images ## Operations checklist Use this checklist every time you run an experiment. ### Before you start - Confirm host resources: at least 3 GB free RAM and 30 GB free disk - Start libvirt and default network: systemctl status libvirtd; virsh net-list - Start VMs: virsh start lab-srv; virsh start lab-cli - Confirm time sync: timedatectl shows synchronized: yes - Verify basic connectivity: ping and curl from lab-cli to lab-srv ### Create a safe point - Shutdown lab VMs cleanly - Create or update a snapshot with a descriptive label - Start VMs again ### Run your changes - Apply one change at a time (config, package, kernel param) - Log what you changed and the commands you ran - Verify with observable checks and save sample outputs ### If something breaks - Check the relevant service status and logs - Use the failure table to identify the probable cause - If repair takes longer than a few minutes, revert to the last snapshot ### When you finish - Decide: keep the new state (take a snapshot) or roll back to baseline - Clean up temporary files and unused snapshots - Note what worked, what failed, and why ### Weekly maintenance - Update packages on both VMs and test that services still start - Create a fresh baseline snapshot after updates - Prune stale snapshots to control disk usage - Verify that your ISO images and VM definitions are still accessible ### Quick checklist table
PhaseAction
PrepStart libvirt, verify default network
BaselineBoot VMs, verify ping/curl
Safe pointCreate snapshot with label
ExecuteChange one thing, observe, record
TriageCheck status/logs, fix or revert
CloseoutSnapshot or revert, cleanup
## Conclusion With a small two-VM Linux lab and reliable rollback, you can safely explore configuration changes, rehearse upgrades, and triage issues without collateral damage. The approach here keeps scope tight, emphasizes observable results, and always preserves a way back. Next steps you can take immediately: - Template your VM creation commands so you can spin up fresh labs quickly - Add a second network (host-only) to test routing and firewalling between segments - Create a reusable Bash script to automate start, snapshot, revert, and verification steps - Expand the lab gradually (for example, add an SSH bastion VM or a database VM) only when your current workflow is repeatable and fast By keeping your experiments measurable and easy to inspect locally, you will iterate faster, reduce rework, and build operational confidence that transfers to larger environments.