Redis Capacity Planning With Practical Examples

Redis capacity planning turns vague estimates into concrete, repeatable steps: measure what matters, size memory and CPU with safety margins, choose persistence and eviction policies that match your data criticality, and verify the results with concrete diagnostics. This guide walks through a version and environment inventory, a safe configuration path, worked sizing examples, verification checks, common failure modes with recovery actions, and an operations checklist you can run monthly or before major launches.

Version and Environment Inventory

Capture a precise snapshot of your current state before making changes. This inventory becomes your baseline and your rollback reference.

Record the following:

• Redis version and build options (e.g., redis-server --version, check for jemalloc vs glibc allocator).
• Topology: standalone, primary-replica, or Redis Cluster; number of primaries and replicas.
• Persistence mode: none, RDB, AOF (appendfsync always|everysec|no), or both.
• Eviction settings: maxmemory, maxmemory-policy.
• Data model: key counts by type, average key and value sizes, TTL coverage (% of keys with TTL), largest keys.
• Traffic profile: read/write mix, operations per second, peak concurrency, P99 latency targets.
• Replication: replica count, network RTT, acceptable replication lag.
• Host resources: RAM, CPU (cores and clock), NIC bandwidth, storage capacity and IOPS/latency.
• Client behavior: timeouts, connection pooling, cluster awareness (for Redis Cluster).

Collect baseline metrics using redis-cli:

Memory and fragmentation:

• redis-cli INFO memory
• redis-cli MEMORY STATS
• redis-cli MEMORY DOCTOR

Workload and latency:

• redis-cli INFO stats
• redis-cli --latency
• redis-cli --latency-history

Keyspace:

• redis-cli INFO keyspace
• Sample key sizes: redis-cli --scan | head -n 1000 | xargs -n1 redis-cli MEMORY USAGE

Replication and persistence:

• redis-cli INFO replication
• redis-cli INFO persistence

Keep these snapshots for at least one peak period and one maintenance window (when persistence rewrites may occur).

Safe Configuration Path

Design for safety first. The following choices reduce surprises and make scaling decisions reversible.

Memory headroom

• Target steady-state used_memory (including allocator overhead) at or below 70% of system RAM dedicated to Redis.
• Account for fragmentation. Multiplying by a factor between 1.2 and 1.5 is a practical starting range; use MEMORY STATS to refine.
• If using fork-based persistence (RDB save, AOF rewrite), leave additional headroom for copy-on-write during background operations. Heavier write rates require more temporary headroom.

maxmemory and eviction policy

• For caches: set maxmemory and use an allkeys policy (allkeys-lru or allkeys-lfu) so Redis can shed load predictably.
• For authoritative datasets: avoid eviction if possible; cap key creation upstream or shard to keep memory under control.
• Set a clear alert near 80-85% of maxmemory and a hard action threshold near 90-95%.

Persistence

• AOF everysec is a balanced starting point for many workloads.
• Schedule RDB bgsave during off-peak windows or disable if AOF already meets durability needs.
• Ensure enough disk capacity for AOF growth and for temporary files during rewrite.

Replication and failover

• Keep at least one replica for HA when the dataset is important.
• Monitor replication lag and link health; add network headroom for bursts.

CPU and concurrency

• Redis command execution is single-threaded; ensure sufficient single-core performance and avoid hot keys that serialize traffic.
• Keep CPU below ~60% on the busiest core during peak.

Client timeouts and backpressure

• Set client timeouts to fail fast during incidents.
• Use reasonable connection pooling and pipelining to reduce per-request overhead.

Capacity dimensions to model

Practical Sizing Examples

These constructed examples show how to build defensible estimates. Replace inputs with your measurements.

Example A: Session cache on a single primary with one replica

Assumptions (constructed):

• 5,000,000 session keys; average key size 32 bytes; values are JSON blobs about 256 bytes.
• TTL: 100% of keys expire in <= 24h.
• Object overhead: estimate 50 bytes per key-value pair (varies by type and encoding).
• Allocator and meta overhead factor: 1.2.
• Fragmentation factor: 1.3 initially, improving after defragmentation to 1.2.
• Replication: 1 replica (2 copies in RAM total).
• Persistence: AOF everysec, periodic rewrite; no RDB snapshots.

Memory per key estimate:

• Data bytes = key(32) + value(256) + overhead(50) = 338 bytes.
• Apply allocator/meta factor: 338 * 1.2 = 405.6 bytes.
• Apply fragmentation: 405.6 * 1.3 = 527.28 bytes.

Dataset memory (primary only):

• 5,000,000 * 527.28 bytes ≈ 2.63 GB.

With one replica:

• Total RAM across both nodes ≈ 5.26 GB.

Safety margin on each node:

• Add 30% headroom for bursts and operational events: 2.63 GB * 1.3 ≈ 3.42 GB per node.
• Choose an instance with at least 8 GB RAM per node if Redis is the only major process, so steady-state remains well under 70%.

maxmemory setting (primary):

• Set maxmemory ≈ 3.5 GB, policy allkeys-lfu (for a cache). This allows controlled evictions before OOM.

Disk for AOF:

• Expect AOF size on the order of the logical dataset with compaction variance. Provision at least 2-3x the expected AOF size to accommodate rewrite (temporary files) and growth during spikes.

Verification goals:

• evicted_keys remains low and stable; hit rate acceptable for your app.
• used_memory < 70% of RAM and mem_fragmentation_ratio declines after warmup.

Example B: Product catalog on Redis Cluster (authoritative, no eviction)

Assumptions (constructed):

• 20,000,000 keys across 6 primaries; average key 24 bytes; average value 512 bytes (hashes with ziplist/hashtable mix).
• No eviction allowed; upstream caps writes.
• Replication: 1 replica per primary.
• Persistence: RDB snapshots nightly; no AOF.
• Write rate: 5,000 ops/sec peak total, mostly reads.

Memory per key estimate (primary):

• Data bytes = 24 + 512 + 70 bytes overhead ≈ 606 bytes.
• Allocator/meta factor: 1.2 -> 727.2 bytes.
• Fragmentation factor: 1.25 -> 909 bytes.

Per-primary key count: ~3.33M. Memory per primary: 3.33M * 909 bytes ≈ 3.03 GB.

Node sizing and headroom:

• With a replica, per-node RAM target: 3.03 GB + 30% headroom ≈ 3.94 GB.
• Choose 8-16 GB RAM per node to allow for growth and snapshot overhead.

RDB snapshot considerations:

• Schedule snapshots off-peak. During bgsave, copy-on-write can temporarily raise memory if many pages are modified; ensure headroom is sufficient for your write rate.

No eviction policy:

• Do not set maxmemory or set policy to noeviction. Add alerts when used_memory exceeds 60-65% of RAM to act before pressure builds.

Throughput:

• With primarily reads and small objects, single-core performance should suffice. Distribute keys evenly to avoid hot shards. Monitor cluster slot balance and migrate if needed.

Quick worksheet for your estimates

Verification and Diagnostics

After any capacity change, verify outcomes with measurable checks. Focus on these areas.

Memory and fragmentation:

• redis-cli INFO memory — Confirm used_memory and used_memory_rss. Keep mem_fragmentation_ratio stable and near your expectation (e.g., 1.1-1.5 depending on allocator and workload).
• redis-cli MEMORY STATS — Inspect allocator fragmentation, active vs resident memory, and peak memory.
• redis-cli MEMORY DOCTOR — Get human-readable suggestions about fragmentation and allocation patterns.

Keyspace and object health:

• redis-cli INFO keyspace — Track key count and expires. Ensure the ratio of keys with TTL matches design.
• Spot big keys (can cause latency and skew memory): redis-cli --scan | xargs -n1 -P4 redis-cli MEMORY USAGE | sort -nr | head -50

Latency and throughput:

• redis-cli --latency --latency-history — Ensure P99 latency remains within SLO during peak.
• redis-cli INFO stats | egrep "instantaneous_ops_per_sec|keyspace_hits|keyspace_misses|evicted_keys|expired_keys" — Watch for unexpected evictions or miss spikes.

Replication and persistence:

• redis-cli INFO replication — Confirm role, replica offsets, and lag are small and stable.
• redis-cli INFO persistence — Check aof_rewrite_in_progress, rdb_bgsave_in_progress, last rewrite/save status and duration.

Resource headroom:

• OS-level checks: free RAM, busiest CPU core, NIC utilization, disk IOPS/latency.
• Ensure Redis remains under the chosen steady-state thresholds (e.g., <70% RAM, <60% busiest-core CPU).

Scaling signals and thresholds (tune for your SLOs)

Failure Modes and Recovery

Prepare responses before they are needed. Below are common issues and practical recovery actions.

1. Memory pressure and evictions

• Symptoms: evicted_keys increasing, OOM errors with noeviction, client timeouts.
• Immediate actions:
• For caches: temporarily raise maxmemory if headroom exists; or switch to allkeys-lfu to improve retention quality.
• For authoritative stores: block new key creation upstream; delete non-essential keys; shard or scale vertically.
• Root-cause checks: large temporary keys, oversized values, TTL regression, fragmentation spike.
• Rollback: revert eviction policy if it degrades hit rate; restore previous maxmemory after cleanup.
• Verification: used_memory returns under threshold; evictions stabilize; P99 latency normal.

1. Latency spikes during RDB or AOF rewrite

• Symptoms: P99 latency elevated; fork time long; increased RSS due to copy-on-write.
• Immediate actions:
• Move snapshot or rewrite to off-peak; reduce frequency temporarily.
• If necessary, disable the less-critical persistence mode for the duration (e.g., CONFIG SET save ""). Re-enable after the window.
• Medium-term:
• Increase RAM headroom; reduce write amplification; tune dataset to smaller objects.
• Verification: fork times acceptable; latency returns to baseline; persistence completes within the window.

1. Replication lag and failover risk

• Symptoms: replica behind primary; link flaps; growing backlog.
• Immediate actions:
• Check network health and bandwidth; ensure replicas are not I/O bound by persistence.
• Reduce load temporarily or add replicas closer to clients.
• Rollback:
• If a topology change caused lag, revert and retry off-peak.
• Verification: replica offset catches up; lag metrics flatten; failover rehearsal succeeds.

1. Disk saturation from AOF growth

• Symptoms: AOF files approach disk capacity; rewrite cannot complete.
• Immediate actions:
• Trigger manual rewrite off-peak; ensure enough free space for temp files.
• Temporarily adjust appendfsync policy to reduce I/O if latency is impacted.
• Medium-term:
• Increase disk capacity; implement key TTLs for non-critical data; compress values upstream if acceptable.
• Verification: aof_current_size stable or reduced; rewrites complete successfully; latency within SLO.

1. Hot key or hot shard

• Symptoms: uneven CPU usage, single core pegged, local latency spikes.
• Actions:
• Identify hot keys via latency monitor or command stats; split or cache hot results at the application tier; add sharding if needed.
• Verification: ops and latency distribute evenly across cores/shards.

Operations Checklist

Use this repeatable checklist monthly and before major launches.

• Inventory and baseline
• Record Redis version, topology, persistence, eviction policy.
• Snapshot INFO memory, stats, keyspace, replication, persistence.
• Keep a 24h metric view with peak markers.

• Memory and safety margins
• Confirm used_memory <= 70% of dedicated RAM.
• Review mem_fragmentation_ratio and allocator stats.
• Validate maxmemory and policy match data criticality.

• Persistence and storage
• Verify rewrite/save windows are off-peak.
• Confirm disk capacity >= 2-3x expected AOF or sufficient for RDB artifacts.
• Check last rewrite/save status is OK.

• Replication and failover
• Ensure replicas present, healthy, and lag low.
• Rehearse failover in a maintenance window and document results.

• Workload and latency
• Compare P99 latency vs SLO during peak.
• Review slowlog for new patterns; optimize or split heavy commands.

• Hotspots and growth
• Scan for big keys or hot keys; mitigate as needed.
• Re-estimate memory with latest key counts and sizes; adjust scaling plan.

• Alerting
• Alerts set for memory, evictions, latency, replication lag, and persistence duration.

• Pilot before rollout
• Test a narrow, measurable change first (e.g., new eviction policy on a canary node), then scale once verified.

Conclusion

Capacity planning for Redis is a cyclical practice: measure, decide, verify, and adjust. Start with a narrow pilot that is easy to inspect, then scale once your estimates hold. Keep steady-state headroom, align eviction and persistence to your data criticality, and verify changes with concrete diagnostics. With these steps and safety margins, you can scale predictably and recover quickly when reality surprises you.