## Intro

Bash capacity planning means estimating and managing CPU, memory, I/O, file descriptors (FDs), and process counts so your scripts run predictably under load. This guide gives you:

- A step-by-step workflow to size and scale safely

- Practical probes and formulas you can run locally

- A small pilot plan to de-risk rollout

Target readers: developers, DevOps consultants, and startup teams who ship Bash-based jobs (ETL, CI steps, data munging, release tasks) on laptops, servers, Docker, or Kubernetes.

## Workflow Overview

Use a simple, repeatable sequence. Keeping the steps distinct reduces rework and clarifies decisions.

### 1. Define the workload and SLOs

- Work profile : how many inputs per unit time, average size, peak size, and acceptance criteria.

- SLO examples : P95 latency per item, hourly throughput, and error budget.

- Failure modes to avoid : OOM kills, runaway forks, FD exhaustion, and long tail latencies.

### 2. Probe the environment

Create a fast inventory of capacity and limits.

#!/usr/bin/env bash
set -Eeuo pipefail

printf "Env probe\n"
cores=$(getconf _NPROCESSORS_ONLN || echo 1)
mem_kb=$(grep -i MemTotal /proc/meminfo | awk '{print $2}')
mem_mb=$((mem_kb/1024))
fd_limit=$(ulimit -n || true)
proc_limit=$(ulimit -u || true)
avail_disk=$(df -h . | awk 'NR==2{print $4}')

printf "cores=%s\nmem_mb=%s\nfd_limit=%s\nproc_limit=%s\navail_disk=%s\n" \
 "$cores" "$mem_mb" "$fd_limit" "$proc_limit" "$avail_disk"

# Current FD usage (use $$ for current shell PID)
printf "fds_now=%s\n" "$(ls -1 /proc/$$/fd | wc -l)" 

 Note : In containers, ulimit values may differ from the host. Set them explicitly when needed (Docker: --ulimit , Kubernetes: container-level ulimit via securityContext if your distro supports it).

</div>

### 3. Measure per-item resource cost

Pick a representative job and measure CPU time, RSS (resident memory), and FD usage.

#!/usr/bin/env bash
# sample_job.sh: replace body with your real work unit
set -Eeuo pipefail
in="$1"
# Example workload: hash then compress to simulate CPU + I/O
sha256sum "$in" >/dev/null
gzip -c "$in" >/dev/null 
 #!/usr/bin/env bash
# measure_cost.sh: run job and capture rough CPU and memory
set -Eeuo pipefail
item="$1"
# Prefer /usr/bin/time -v if available for Max RSS
if command -v /usr/bin/time >/dev/null; then
 /usr/bin/time -v bash sample_job.sh "$item" 2>_time.txt 1>/dev/null || true
 cpu_user=$(awk -F: '/User time/ {gsub(/ /,""); print $2}' _time.txt)
 cpu_sys=$(awk -F: '/System time/ {gsub(/ /,""); print $2}' _time.txt)
 rss_kb=$(awk -F: '/Maximum resident set size/ {gsub(/ /,""); print $2}' _time.txt)
 printf "cpu_user=%s cpu_sys=%s rss_kb=%s\n" "$cpu_user" "$cpu_sys" "$rss_kb"
else
 # Fallback: coarse time and a snapshot of RSS for the current shell
 t0=$(date +%s)
 bash sample_job.sh "$item"
 t1=$(date +%s)
 rss_kb=$(awk '/VmRSS/ {print $2}' /proc/$$/status 2>/dev/null || echo 0)
 printf "elapsed_s=%s rss_kb=%s\n" "$((t1-t0))" "$rss_kb"
fi 
 Run the measure script across 20 to 50 representative items and compute medians, P95s, and maximums. Use conservative numbers (P95 or max) plus safety margins.

### 4. Build a simple capacity model

### Memory sizing

- Let rss_per_job_kb be P95 RSS in KB.

- Let avail_mem_kb be total minus OS/app reserve.

- Safety margin: keep 25% memory free.

- concurrency_mem = floor((avail_mem_kb * 0.75) / rss_per_job_kb)

Example :

- Total mem: 8 GB → 8 × 1024 × 1024 = 8,388,608 KB

- Reserve 25% → 6,291,456 KB usable

- rss_per_job_kb = 80,000

- concurrency_mem = floor(6,291,456 / 80,000) = 78

### CPU sizing

- Let cores = logical cores.

- Let cpu_sec_per_item = user + system seconds per item at 1× concurrency.

- Budget 70% CPU to leave headroom.

- max_items_per_sec_cpu = (cores * 0.7) / cpu_sec_per_item

- concurrency_cpu = floor(cores * 0.7) if each job keeps a core busy.

Example :

- cores = 8 , cpu_sec_per_item = 0.2 s

- max_items_per_sec_cpu = (8 * 0.7) / 0.2 = 28 items/s

### File descriptor sizing

- fd_limit from ulimit -n .

- Each pipeline stage can consume FDs (stdin/stdout/stderr, pipes, open files).

- Measure: before and during the job, run ls /proc/$$/fd | wc -l .

- concurrency_fd = floor((fd_limit * 0.8) / fds_per_job)

### Process and thread limits

- proc_limit from ulimit -u . Count children with ps --ppid .

- Keep 30% headroom: children <= proc_limit * 0.7 .

### Disk and network I/O

dd if=/dev/zero of=./_io_test.bin bs=1M count=512 oflag=direct 2>&1 | tail -1 
 Use dd to sanity-check write speed (rough). For reads, time cat or sha256sum on a large file. Ensure your throughput target stays below 70% of observed steady-state bandwidth.

### 5. Decide scaling and parallelism

- For local parallelism: use xargs -P or GNU parallel to cap concurrency.

- For sharding: split input into N parts and process with N workers.

- Prefer coarse-grained parallelism (process-level) over spawning a subshell for every token.

Example harness with xargs :

#!/usr/bin/env bash
set -Eeuo pipefail
P="${P:-8}" # parallelism cap
LIST="${1:-worklist.txt}
export -f sample_job.sh || true

# Each line in worklist.txt is a path to process
cat "$LIST" | xargs -r -P "$P" -n 1 -I{} bash sample_job.sh {} 

### 6. Add guardrails and safety margins

- set -Eeuo pipefail to fail fast and reduce partial work.

- ulimit settings (example values, tune for your host):

 # Soft caps to avoid host-wide impact
ulimit -S -n 4096 # FDs
ulimit -S -u 2048 # processes
ulimit -S -m $((6*1024*1024)) || true # memory KB, if supported 

- Back-pressure: cap queue depth and parallel workers.

- Retry policy: exponential backoff for transient I/O errors.

### 7. Observe signals that drive scaling decisions

- Latency : moving average and P95 per item rising over time.

- Queue depth : worklist grows faster than completion.

- Resource saturation : vmstat r > cores for long periods, CPU > 90%, I/O wait > 20% steady.

- Errors : non-zero exits, OOM kills, broken pipes.

 Simple latency tracker :

#!/usr/bin/env bash
set -Eeuo pipefail
log=_latency.csv
echo "ts_s, elapsed_ms" > "$log"
while read -r item; do
 t0=$(date +%s%3N)
 bash sample_job.sh "$item"
 t1=$(date +%s%3N)
 echo "$((t0/1000)),$((t1 - t0))" >> "$log"
done < worklist.txt 

### 8. Container and cluster notes (Docker, Compose, Kubernetes)

- Docker : set CPU and memory caps so your math matches reality, e.g. --cpus , --memory , and --ulimit nofile=4096 .

- Compose : replicate those limits in the service definition.

- Kubernetes : requests and limits should reflect your measured budgets; watch for CPU throttling if limits are tight.

- Kubernetes Ingress and Secrets are unrelated to Bash compute limits, but your jobs may pull inputs via HTTP or read secrets; avoid fetching secrets in tight loops.

## Practical Examples: Formulas and Snippets

### Memory-bound concurrency calculator

 #!/usr/bin/env bash
set -Eeuo pipefail
rss_kb=${1:?"rss_kb per job required"}
avail_kb=$(grep -i MemAvailable /proc/meminfo | awk '{print $2}')
# 25% safety margin
usable_kb=$(( (avail_kb * 75) / 100 ))
conc=$(( usable_kb / rss_kb ))
printf "avail_kb=%s usable_kb=%s rss_kb=%s concurrency=%s\n" \
 "$avail_kb" "$usable_kb" "$rss_kb" "$conc" 

### CPU throughput estimator

 #!/usr/bin/env bash
set -Eeuo pipefail
cores=$(getconf _NPROCESSORS_ONLN || echo 1)
cpu_sec_per_item=${1:?"cpu seconds per item"}
budget=$(python3 - <<EOF
cores=$cores
cpi=$cpu_sec_per_item
print(round((cores*0.7)/cpi,2))
EOF
)
printf "cores=%s items_per_sec_cpu_budget=%s\n" "$cores" "$budget" 
 If Python is not available, compute by hand or with bc .

### FD safety check inside a pipeline

#!/usr/bin/env bash
set -Eeuo pipefail
before=$(ls /proc/$$/fd | wc -l)
# Simulate a small fanout
{ find . -maxdepth 1 -type f -print0 | xargs -0 -P 8 -n 1 -I{} bash -c 'cat "{}" >/dev/null'; }
after=$(ls /proc/$$/fd | wc -l)
limit=$(ulimit -n)
printf "fd_before=%s fd_after=%s fd_limit=%s\n" "$before" "$after" "$limit" 

### Safe xargs pattern

 find input/ -type f -print0 | xargs -0 -P "$P" -n 1 -I{} bash -c '
 set -Eeuo pipefail
 f="$1"
 # Avoid reading whole file into memory
 sha256sum "$f" | awk "{print \$1}" >> hashes.txt
' _ {} 

### Growth risk checklist

- Avoid O(n²) loops over growing directories; prefer indexing and single-pass scans.

- Do not slurp whole files into memory; stream with while read -r or awk .

- Avoid unbounded globbing; use find with size and depth limits.

- Use -print0 with xargs -0 to handle spaces safely and reduce parsing bugs.

- Cap retries and parallelism to avoid thundering herds.

## Local Pilot Plan

 Make the first pilot narrow, measurable, and easy to inspect locally before any rollout.

### Scope

- One critical job type, 200 representative items, max file size 64 MB.

- Fixed parallelism P in {1, 4, 8}.

### Pass or fail criteria

- P95 item latency ≤ 300 ms at P=4.

- Zero OOM, zero FD exhaustion, non-zero exits ≤ 0.5%.

- CPU steady-state ≤ 70%, memory free ≥ 25%.

### Pilot files

worklist.txt : list of input paths.

pilot.sh : orchestrates the run and collects metrics.

#!/usr/bin/env bash
set -Eeuo pipefail
P="${P:-4}
LIST="${1:-worklist.txt}
mkdir -p _pilot
: > _pilot/latency.csv
: > _pilot/errors.log

run_item() {
 local item="$1"
 local t0 t1 rc
 t0=$(date +%s%3N)
 if bash sample_job.sh "$item\); then
 t1=$(date +%s%3N)
 echo "$((t0/1000)),$((t1-t0))" >> _pilot/latency.csv
 else
 rc=$?
 echo "$(date -Is) item=$item rc=$rc" >> _pilot/errors.log
 fi
}
export -f run_item

cp "$LIST" _pilot/worklist.run
cat _pilot/worklist.run | xargs -r -P "$P" -n 1 -I{} bash -c 'run_item "$@"' _ {}

# Summaries
items=$(wc -l < _pilot/worklist.run)
errs=$(wc -l < _pilot/errors.log 2>/dev/null || echo 0)
lat_p95=$(awk -F, 'NR>1{a[NR-1]=$2} END{n=asort(a); idx=int(0.95*n); if(idx<1) idx=1; print a[idx]}' _pilot/latency.csv 2>/dev/null || echo 0)
free_mem_mb=$(awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo)

printf "items=%s errors=%s p95_ms=%s free_mem_mb=%s\n" "$items" "$errs" "$lat_p95" "$free_mem_mb" 

### Tuning loop

- If p95_ms is high: lower P or reduce per-item CPU (optimize the job).

- If free_mem_mb is low: reduce P or memory per item.

- If errors spike: inspect _pilot/errors.log and add retries or input validation.

### Safety and rollback

- Default P=1 if any check fails.

- Keep soft ulimits in the pilot to bound impact on shared hosts.

## Conclusion

 You now have a concrete way to size Bash workloads:

- Measure per-item CPU, memory, FDs, and I/O costs.

- Model concurrency with simple formulas and 25% to 30% headroom.

- Enforce guardrails with ulimit and bounded parallelism.

- Watch latency, queue depth, saturation, and errors to decide when to scale.

### Next steps

- Run the local pilot, record P95 latencies and resource use, and set your default P.

- Document steady-state and peak budgets for CPU, memory, FDs, and processes.

- If moving to Docker or Kubernetes, carry over the same limits and headroom in container specs.