A local Linux lab is a controlled space on your workstation where you can try ideas, test changes, and troubleshoot issues without risking shared environments. You can reset it to a known-good state in minutes, observe real system behavior, and practice repeatable workflows that transfer to production. This guide focuses on a safe, verifiable setup using Linux virtual machines, concrete commands, expected outputs, and reliable rollback paths. You will build a small two-VM topology (client and server) suitable for service testing, network checks, and hands-on experiments.
What you will build
- Host: Linux with KVM/libvirt
- VM 1 (lab-srv): 2 vCPU, 2 GB RAM, 20 GB disk running Nginx as a test HTTP service
- VM 2 (lab-cli): 1 vCPU, 1 GB RAM, 10 GB disk for client-side testing (curl, ping, traceroute)
- Networking: libvirt default NAT network (internet access through host), optional host-only later
- Safety: snapshots on both VMs to enable fast rollback
Why start small
A tightly scoped lab is measurable, fast to verify, and easy to reset if you break it. That means you learn faster and spend less time undoing mistakes.
Version and environment inventory
Before building the lab, capture what you are starting with and pick a small, inspectable scope. The first pilot should be narrow and measurable: for example, start one server VM running Nginx and one client VM that can curl it over a private network.
Record host OS and versions (run on host)
cat /etc/os-release
uname -r
lscpu | grep -E 'Virtualization|Model name'
systemd --version | head -1
bash --version | head -1
virsh --version
qemu-system-x86_64 --version | head -1
Check hardware and resources (run on host)
lsmod | grep -E 'kvm_(intel|amd)'
free -h
lsblk -o NAME,SIZE,TYPE,MOUNTPOINT
ip addr
ip route
Decide the initial topology
- One NAT network via libvirt (default) so VMs can pull packages from the internet through the host
- Two VMs to keep failure domains simple
- Snapshots immediately after base install and after service installation
Example inventory
| Item | Example value |
|---|---|
| Host OS | Ubuntu 22.04 LTS, kernel 5.15 |
| CPU | 8-core with virtualization support (VT-x/AMD-V) |
| RAM | 16 GB total, 3 GB reserved for lab |
| Disk | 200 GB SSD, 40 GB free for lab images |
| Virtualization | KVM/libvirt 8.x, QEMU 6.x |
| Lab VMs | lab-srv (2 vCPU/2 GB/20 GB), lab-cli (1 vCPU/1 GB/10 GB) |
| Network | libvirt default NAT 192.168.122.0/24 |
Safe configuration path
The design goals:
- Isolate experiments from the host
- Keep the network scope tight
- Enable fast rollback with snapshots
The steps below assume a Debian/Ubuntu-like host. Where useful, Fedora/RHEL-like alternatives are shown.
1. Install KVM and libvirt (run on host)
# Debian/Ubuntu
sudo apt update
sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients virt-manager
# Fedora/RHEL
sudo dnf install -y @virtualization virt-install virt-manager
sudo systemctl enable --now libvirtd
Add your user to the libvirt group and re-evaluate your session:
sudo usermod -aG libvirt $USER
newgrp libvirt
2. Verify virtualization on host
# Should return non-empty if supported
lsmod | grep -E 'kvm_(intel|amd)'
# libvirt default network should be present
virsh net-list --all
virsh net-start default || true
virsh net-autostart default
Expected: the default network shows as active. If not, see Failure Modes later.
3. Create VM disks (run on host)
# Adjust paths and sizes as needed
sudo qemu-img create -f qcow2 /var/lib/libvirt/images/lab-srv.qcow2 20G
sudo qemu-img create -f qcow2 /var/lib/libvirt/images/lab-cli.qcow2 10G
4. Install lab-srv from an ISO
# Replace ISO path and OS variant as appropriate
sudo virt-install \
--name lab-srv \
--memory 2048 \
--vcpus 2 \
--disk path=/var/lib/libvirt/images/lab-srv.qcow2,format=qcow2,bus=virtio \
--cdrom /var/lib/libvirt/boot/ubuntu-22.04-live-server-amd64.iso \
--network network=default,model=virtio \
--os-variant ubuntu22.04 \
--graphics vnc \
--noautoconsole
Complete the installation in the console (virt-manager can attach to the VM if needed) and set:
- Hostname:
lab-srv - User:
labops(constructed), with sudo - Network: DHCP via default NAT
5. Install lab-cli similarly
sudo virt-install \
--name lab-cli \
--memory 1024 \
--vcpus 1 \
--disk path=/var/lib/libvirt/images/lab-cli.qcow2,format=qcow2,bus=virtio \
--cdrom /var/lib/libvirt/boot/ubuntu-22.04-live-server-amd64.iso \
--network network=default,model=virtio \
--os-variant ubuntu22.04 \
--graphics vnc \
--noautoconsole
Set hostname lab-cli, user labops, DHCP network.
6. Post-install basics on each VM (run inside each VM)
# Update packages
sudo apt update && sudo apt -y upgrade # Debian/Ubuntu
# or
sudo dnf -y update # Fedora/RHEL
# Confirm hostname and network
hostnamectl
ip addr
Optionally set predictable hostnames in /etc/hosts on both VMs so the names resolve. Replace the IPs with your actual VM addresses:
sudo sh -c 'echo "192.168.122.101 lab-srv" >> /etc/hosts'
sudo sh -c 'echo "192.168.122.102 lab-cli" >> /etc/hosts'
(You can also rely on plain IPs; this is for convenience.)
7. Baseline snapshots (run on host)
virsh shutdown lab-srv && virsh shutdown lab-cli
# Wait for both to be shut off
virsh dominfo lab-srv | grep State
virsh dominfo lab-cli | grep State
# Create snapshots named baseline
virsh snapshot-create-as lab-srv baseline "Post-OS baseline" --disk-only --atomic
virsh snapshot-create-as lab-cli baseline "Post-OS baseline" --disk-only --atomic
# Start both VMs again
virsh start lab-srv
virsh start lab-cli
Note: If your storage pool does not allow disk-only snapshots, omit --disk-only. Ensure your qcow2 images support snapshots.
8. Install a simple test service on lab-srv (inside lab-srv)
# Install Nginx
sudo apt -y install nginx # Debian/Ubuntu
# or
sudo dnf -y install nginx # Fedora/RHEL
# Start and enable
sudo systemctl enable --now nginx
# Check status
systemctl is-active nginx && systemctl is-enabled nginx
If you use a host firewall on the VM:
# UFW example (Ubuntu)
sudo ufw allow 80/tcp
sudo ufw reload
# firewalld example (Fedora/RHEL)
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --reload
9. Take a second snapshot after service install (run on host)
virsh shutdown lab-srv
virsh snapshot-create-as lab-srv svc-nginx "Nginx installed and running" --disk-only --atomic
virsh start lab-srv
Verification and diagnostics
Verification means observable evidence: the hypervisor is running, the VMs boot, they can talk on the intended network, and the service responds with expected content. Use these checks and compare to the Expected result notes.
1. Host-level checks (run on host)
# Libvirt active networks
virsh net-list
# Running VMs
virsh list --all
Expected: default network is active; both lab-srv and lab-cli are running.
2. Find VM IP addresses
Inside each VM, run:
ip -4 addr show dev eth0 | awk '/inet /{print $2}'
Expected: IPs in the 192.168.122.0/24 range (libvirt default) unless you changed it.
3. Basic connectivity (inside lab-cli)
# Replace with lab-srv IP
LAB_SRV_IP=192.168.122.101
ping -c 3 $LAB_SRV_IP
Expected: 0% or low packet loss.
4. HTTP service check (inside lab-cli)
# If curl is missing: sudo apt -y install curl || sudo dnf -y install curl
curl -s -o /dev/null -w "%{http_code}\n" http://$LAB_SRV_IP/
Expected: 200
Get and view a few bytes to prove content:
curl -s http://$LAB_SRV_IP/ | head -n 5
Expected: HTML including a reference to nginx or your custom index page.
5. Service and logs (inside lab-srv)
systemctl status nginx --no-pager
journalctl -u nginx --since "-5m" --no-pager
Expected: active (running), recent 200 responses in logs if logging is enabled.
6. Time and entropy (inside both VMs)
timedatectl | sed -n '1,4p'
Expected: System clock synchronized: yes. If not, enable systemd-timesyncd or chrony as appropriate.
Quick reference table
| Check | Command | Expected result |
|---|---|---|
| Hypervisor OK | lsmod | kvm_intel or kvm_amd present |
| Network active | virsh net-list | default active |
| VM running | virsh list --all | lab-srv, lab-cli running |
| IP assigned | ip -4 addr | 192.168.122.x addresses |
| Nginx status | systemctl is-active nginx | active |
| HTTP reachable | curl -w "%{http_code}" | 200 |
Failure modes and recovery
When something fails, identify the symptom fast, confirm with a targeted command, and repair or roll back.
| Symptom | Likely cause | What to run | Fix |
|---|---|---|---|
| virt-install fails to start VM | CPU virtualization off | lscpu | Enable VT-x/AMD-V in BIOS/UEFI |
| Permission denied to libvirt | User not in libvirt group | id | Add user to libvirt, newgrp or re-login |
| No VM IP address | libvirt network down | virsh net-list | virsh net-start default; autostart it |
| VM cannot reach internet | DNS or NAT issue | curl http://example.com | Check /etc/resolv.conf, restart libvirt network |
| Cannot reach Nginx | Firewall on lab-srv | ss -tlnp | Open port 80 in ufw/firewalld |
| 403/404 from Nginx | Wrong docroot or SELinux | journalctl -u nginx | Fix config; on SELinux, use proper contexts |
| Disk full on host | Large images/snapshots | df -h /var/lib/libvirt/images | Delete old snapshots or expand disk |
| Snapshot revert fails | VM running | virsh dominfo | Shutdown VM, then snapshot-revert |
Recovery commands and patterns
Rebuild or restart libvirt default network (run on host)
virsh net-destroy default || true
virsh net-undefine default || true
# Recreate default network from template
sudo cp /usr/share/libvirt/networks/default.xml /tmp/default.xml
sudo virsh net-define /tmp/default.xml
virsh net-start default
virsh net-autostart default
Snapshot rollback (run on host)
# Always shutdown first
virsh shutdown lab-srv && virsh shutdown lab-cli
# Revert to baseline
virsh snapshot-revert lab-srv baseline --running
virsh snapshot-revert lab-cli baseline --running
If you want to keep the current broken state for later study, create a snapshot before reverting:
virsh snapshot-create-as lab-srv pre-revert "State before revert"
Service reset (inside lab-srv)
# Restore stock config if you broke nginx.conf
sudo cp /etc/nginx/nginx.conf.default /etc/nginx/nginx.conf 2>/dev/null || true
sudo nginx -t
sudo systemctl restart nginx
If the distribution does not provide a .default config, reinstall the package:
# Debian/Ubuntu
sudo apt -y install --reinstall nginx
# Fedora/RHEL
sudo dnf -y reinstall nginx
Time sync fixes (inside any VM)
# Debian/Ubuntu
sudo systemctl enable --now systemd-timesyncd
# Fedora/RHEL
sudo dnf -y install chrony
sudo systemctl enable --now chronyd
Free space recovery (host)
# List snapshots
virsh snapshot-list lab-srv
# Delete unneeded snapshots by name
virsh snapshot-delete lab-srv svc-nginx
# Confirm disk usage
sudo du -sh /var/lib/libvirt/images
Operations checklist
Use this checklist every time you run an experiment.
Before you start
- Confirm host resources: at least 3 GB free RAM and 30 GB free disk
- Start libvirt and default network:
systemctl status libvirtd; virsh net-list - Start VMs:
virsh start lab-srv; virsh start lab-cli - Confirm time sync:
timedatectlshows synchronized: yes - Verify basic connectivity: ping and curl from lab-cli to lab-srv
Create a safe point
- Shutdown lab VMs cleanly
- Create or update a snapshot with a descriptive label
- Start VMs again
Run your changes
- Apply one change at a time (config, package, kernel param)
- Log what you changed and the commands you ran
- Verify with observable checks and save sample outputs
If something breaks
- Check the relevant service status and logs
- Use the failure table to identify the probable cause
- If repair takes longer than a few minutes, revert to the last snapshot
When you finish
- Decide: keep the new state (take a snapshot) or roll back to baseline
- Clean up temporary files and unused snapshots
- Note what worked, what failed, and why
Weekly maintenance
- Update packages on both VMs and test that services still start
- Create a fresh baseline snapshot after updates
- Prune stale snapshots to control disk usage
- Verify that your ISO images and VM definitions are still accessible
Quick checklist table
| Phase | Action |
|---|---|
| Prep | Start libvirt, verify default network |
| Baseline | Boot VMs, verify ping/curl |
| Safe point | Create snapshot with label |
| Execute | Change one thing, observe, record |
| Triage | Check status/logs, fix or revert |
| Closeout | Snapshot or revert, cleanup |
Conclusion
With a small two-VM Linux lab and reliable rollback, you can safely explore configuration changes, rehearse upgrades, and triage issues without collateral damage. The approach here keeps scope tight, emphasizes observable results, and always preserves a way back. Next steps you can take immediately:
- Template your VM creation commands so you can spin up fresh labs quickly
- Add a second network (host-only) to test routing and firewalling between segments
- Create a reusable Bash script to automate start, snapshot, revert, and verification steps
- Expand the lab gradually (for example, add an SSH bastion VM or a database VM) only when your current workflow is repeatable and fast
By keeping your experiments measurable and easy to inspect locally, you will iterate faster, reduce rework, and build operational confidence that transfers to larger environments.