Configuration is one of the highest-leverage levers in Node.js operations. Small missteps like leaving NODE_ENV unset, mis-typing a timeout, or mixing trust proxy settings can create outsized reliability, security, or performance issues. This guide shows how to implement safe and reversible changes through centralized config loading, strict startup validation, observable health endpoints, and concrete rollback procedures. You will get concrete examples, commands, and checklists that map to Express APIs and typical data stores like MongoDB and Redis.
Prerequisites & Assumptions
This guide assumes the following baseline:
- Node.js 18+ for native fetch, AbortController, and the built-in test runner
- Express 4.x as the HTTP framework
- Common reverse proxies: nginx, AWS ALB, Cloudflare, or similar TLS terminators
- Container (Docker/Kubernetes) or bare-metal/systemd/PM2 deployment targets
- Secret management via AWS Secrets Manager, Doppler, dotenv-vault, or injected environment variables — never committed to source control
- Familiarity with curl, jq, and basic shell scripting for verification
All constructed examples are clearly labeled. Replace placeholder values (connection strings, ports, versions) with your actual infrastructure details before use.
Architecture Overview
The following ASCII diagram illustrates the configuration and request flow. The trust proxy boundary is the critical security demarcation: only enable TRUST_PROXY when the reverse proxy is trusted and strips inbound X-Forwarded-* headers.
┌──────────────┐ ┌──────────────────┐ ┌─────────────────────────┐
│ Environment │────▶│ config.js │────▶│ server.js │
│ Variables │ │ (Zod validation) │ │ (Express + HTTP server) │
└──────────────┘ └──────────────────┘ └───────────┬─────────────┘
│
┌─────────────────────────────────┼─────────────────────────────────┐
▼ ▼ ▼
┌─────────────┐ ┌─────────────────┐ ┌──────────────────┐
│ /healthz │ │ /configz │ │ /metrics │
│ (liveness) │ │ (redacted view) │ │ (Prometheus) │
└─────────────┘ └─────────────────┘ └──────────────────┘
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────────┐
│ External Dependencies │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ MongoDB │ │ Redis │ │ HTTP APIs │ │ Reverse Proxy │ │
│ │ (mongoose) │ │ (ioredis) │ │ (fetch) │ │ (nginx/ALB) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ └──────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────┘
Complete Config Schema
Define the entire configuration surface as a TypeScript interface with Zod validation. This serves as both runtime guardrails and IDE documentation.
// config.ts
import { z } from 'zod';
const SessionSecureSchema = z.union([
z.literal('auto'),
z.literal('true'),
z.literal('false'),
]);
const ConfigSchema = z.object({
PORT: z.coerce.number().int().min(1).max(65535).default(3000),
NODE_ENV: z.enum(['development', 'production', 'test']).default('production'),
LOG_LEVEL: z.enum(['error', 'warn', 'info', 'debug', 'trace']).default('info'),
REQUEST_TIMEOUT_MS: z.coerce.number().int().min(100).max(120000).default(10000),
MONGODB_URI: z.string().url().startsWith('mongodb://').or(z.string().url().startsWith('mongodb+srv://')),
REDIS_URL: z.string().url().startsWith('redis://').or(z.string().url().startsWith('rediss://')),
DB_POOL_MIN: z.coerce.number().int().min(0).max(1000).default(2),
DB_POOL_MAX: z.coerce.number().int().min(1).max(1000).default(10),
KEEP_ALIVE: z.coerce.boolean().default(true),
TRUST_PROXY: z.coerce.boolean().default(false),
SESSION_SECURE: SessionSecureSchema.default('auto'),
CONFIG_VERSION: z.string().regex(/^\d{4}-\d{2}-\d{2}\.\d+$/).default('2026-08-08.1'),
TZ: z.string().default('UTC'),
});
export type Config = z.infer<typeof ConfigSchema>;
export function loadConfig(env: Record<string, string | undefined> = process.env): Config {
const parsed = ConfigSchema.safeParse(env);
if (!parsed.success) {
const issues = parsed.error.issues.map(i => `${i.path.join('.')}: ${i.message}`).join('; ');
throw new Error(`Configuration validation failed: ${issues}`);
}
return parsed.data;
}
.env.example — commit this file (without secrets) to version control:
# Required: Application
PORT=3000
NODE_ENV=production
LOG_LEVEL=info
REQUEST_TIMEOUT_MS=10000
CONFIG_VERSION=2026-08-08.1
TZ=UTC
# Required: Data stores (use secret manager in production)
MONGODB_URI=mongodb://user:pass@localhost:27017/app
REDIS_URL=redis://localhost:6379/0
# Connection pools
DB_POOL_MIN=2
DB_POOL_MAX=10
# HTTP behavior
KEEP_ALIVE=true
TRUST_PROXY=false
SESSION_SECURE=auto
# Optional: Memory bound (adjust per container limit)
# NODE_OPTIONS=--max-old-space-size=2048
Database Pool Integration
Wire validated pool settings to actual clients. The following shows minimal initialization for Mongoose and ioredis using the config values.
// db.ts
import mongoose from 'mongoose';
import Redis from 'ioredis';
import { Config } from './config';
export function createMongoClient(cfg: Config) {
return mongoose.connect(cfg.MONGODB_URI, {
maxPoolSize: cfg.DB_POOL_MAX,
minPoolSize: cfg.DB_POOL_MIN,
serverSelectionTimeoutMS: 5000,
socketTimeoutMS: 45000,
family: 4, // IPv4 first
});
}
export function createRedisClient(cfg: Config) {
const client = new Redis(cfg.REDIS_URL, {
maxRetriesPerRequest: 3,
retryStrategy: (times) => {
if (times > 3) return null; // stop retrying
return Math.min(times * 200, 2000);
},
connectionName: `app-${process.pid}`,
lazyConnect: true,
});
client.on('error', (err) => console.error('Redis connection error', { error: err.message }));
return client;
}
Tip: Size DB_POOL_MAX using the formula (cpu_cores * 2) + effective_spindle_count. For a 4-core container with SSD storage, start at 10. Monitor db.serverStatus().connections in MongoDB and connected_clients in Redis to tune.
Session/Cookie Security
Implement SESSION_SECURE='auto' logic: secure cookies only when TRUST_PROXY=true and the request arrived over HTTPS (req.secure). This prevents secure cookie failures when TLS terminates at the reverse proxy.
// session.ts
import session from 'express-session';
import cookieParser from 'cookie-parser';
import { Request } from 'express';
import { Config } from './config';
export function createSessionMiddleware(cfg: Config) {
return [
cookieParser(),
session({
name: 'sid',
secret: process.env.SESSION_SECRET ?? 'dev-secret-change-in-production',
resave: false,
saveUninitialized: false,
cookie: {
httpOnly: true,
sameSite: 'lax',
secure: (req: Request) => {
if (cfg.SESSION_SECURE === 'true') return true;
if (cfg.SESSION_SECURE === 'false') return false;
// 'auto': trust proxy + req.secure (set by Express when trust proxy is enabled)
return cfg.TRUST_PROXY && req.secure;
},
maxAge: 24 * 60 * 60 * 1000, // 24 hours
},
}),
];
}
Security: Generate SESSION_SECRET with openssl rand -base64 32 and store in your secret manager. Never use the default in production.
Graceful Shutdown
Handle SIGTERM to stop accepting new connections, drain in-flight requests, close database pools, and exit cleanly. Align the grace period with your orchestrator's terminationGracePeriodSeconds (Kubernetes default 30s).
// shutdown.ts
import { Server } from 'http';
import mongoose from 'mongoose';
import Redis from 'ioredis';
import { Config } from './config';
export function setupGracefulShutdown(
server: Server,
mongo: typeof mongoose,
redis: Redis,
cfg: Config
) {
const GRACE_MS = 25000; // leave 5s buffer before K8s SIGKILL
let shuttingDown = false;
async function shutdown(signal: string) {
if (shuttingDown) return;
shuttingDown = true;
console.log(`Received ${signal}, starting graceful shutdown`);
// 1. Stop accepting new connections
server.close(() => console.log('HTTP server closed'));
// 2. Wait for in-flight requests (simplified: use a timeout)
await new Promise(resolve => setTimeout(resolve, GRACE_MS));
// 3. Close DB pools
await Promise.allSettled([
mongo.connection.close(false).then(() => console.log('MongoDB pool closed')),
redis.quit().then(() => console.log('Redis connection closed')),
]);
console.log('Graceful shutdown complete');
process.exit(0);
}
process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));
// Windows: SIGTERM not emitted on `taskkill /PID`, use SIGINT equivalent
if (process.platform === 'win32') {
process.on('message', (msg) => {
if (msg === 'shutdown') shutdown('windows-shutdown');
});
}
}
Kubernetes preStop hook (add to pod spec):
lifecycle:
preStop:
exec:
command: ["sh", "-c", "sleep 5"] # overlaps with GRACE_MS
Request Timeout Enforcement
req.setTimeout only sets the socket idle timeout — it does not cancel route handlers. Use AbortController (Node 18+) to propagate cancellation through async middleware and route handlers.
// timeout.ts
import { Request, Response, NextFunction } from 'express';
import { Config } from './config';
export function requestTimeoutMiddleware(cfg: Config) {
return (req: Request, res: Response, next: NextFunction) => {
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), cfg.REQUEST_TIMEOUT_MS);
// Attach signal to request for downstream use
(req as any).abortSignal = controller.signal;
res.on('finish', () => clearTimeout(timeoutId));
res.on('close', () => clearTimeout(timeoutId));
controller.signal.addEventListener('abort', () => {
if (!res.headersSent) {
res.status(503).json({ error: 'Request timeout', timeoutMs: cfg.REQUEST_TIMEOUT_MS });
}
});
next();
};
}
Usage in routes:
app.get('/slow', requestTimeoutMiddleware(cfg), async (req, res) => {
await doWork(req.abortSignal); // pass signal to cancellable operations
res.json({ ok: true });
});
/configz Endpoint with Secret Redaction
Expose a runtime config view that redacts secrets and includes a SHA-256 hash for drift detection.
// configz.ts
import { Request, Response } from 'express';
import { Config } from './config';
import { createHash } from 'crypto';
const SECRET_KEY_PATTERN = /(SECRET|KEY|TOKEN|PASSWORD|PASS)/i;
const URI_PASSWORD_PATTERN = /:\/\/([^:]+):([^@]+)@/;
function redactConfig(cfg: Config): Record<string, unknown> {
const clone = JSON.parse(JSON.stringify(cfg));
for (const key of Object.keys(clone)) {
if (SECRET_KEY_PATTERN.test(key)) {
clone[key] = '***REDACTED***';
}
}
// Redact passwords in connection strings
if (clone.MONGODB_URI && typeof clone.MONGODB_URI === 'string') {
clone.MONGODB_URI = clone.MONGODB_URI.replace(URI_PASSWORD_PATTERN, '://$1:***@');
}
if (clone.REDIS_URL && typeof clone.REDIS_URL === 'string') {
clone.REDIS_URL = clone.REDIS_URL.replace(URI_PASSWORD_PATTERN, '://$1:***@');
}
return clone;
}
export function createConfigzEndpoint(cfg: Config) {
return (req: Request, res: Response) => {
const redacted = redactConfig(cfg);
const hash = createHash('sha256').update(JSON.stringify(redacted)).digest('hex').slice(0, 16);
res.json({ config: redacted, configHash: hash, timestamp: new Date().toISOString() });
};
}
Expected /configz output (constructed example):
{
"config": {
"PORT": 3000,
"NODE_ENV": "production",
"LOG_LEVEL": "info",
"REQUEST_TIMEOUT_MS": 10000,
"MONGODB_URI": "mongodb://user:***@localhost:27017/app",
"REDIS_URL": "redis://:***@localhost:6379/0",
"DB_POOL_MIN": 2,
"DB_POOL_MAX": 10,
"KEEP_ALIVE": true,
"TRUST_PROXY": true,
"SESSION_SECURE": "auto",
"CONFIG_VERSION": "2026-08-08.1",
"TZ": "UTC"
},
"configHash": "a1b2c3d4e5f67890",
"timestamp": "2026-08-08T12:34:56.789Z"
}
Observability Hooks
Integrate structured logging with Pino, request-ID correlation, and a Prometheus metrics endpoint.
// observability.ts
import pino from 'pino';
import { Request, Response, NextFunction } from 'express';
import { Config } from './config';
import client from 'prom-client';
export function createLogger(cfg: Config) {
return pino({
level: cfg.LOG_LEVEL,
base: { pid: process.pid, hostname: require('os').hostname() },
timestamp: () => `,"time":"${new Date().toISOString()}"`,
});
}
export function requestIdMiddleware(req: Request, res: Response, next: NextFunction) {
const id = req.headers['x-request-id'] as string || crypto.randomUUID();
req.id = id;
res.setHeader('X-Request-Id', id);
next();
}
export function createMetricsEndpoint() {
const register = new client.Registry();
client.collectDefaultMetrics({ register, prefix: 'nodejs_' });
const httpRequests = new client.Counter({
name: 'http_requests_total',
help: 'Total HTTP requests',
labelNames: ['method', 'route', 'status'],
registers: [register],
});
return (req: Request, res: Response) => {
res.set('Content-Type', register.contentType);
res.send(register.metrics());
};
}
Usage in server.ts:
app.use(requestIdMiddleware);
app.use((req, res, next) => {
const start = process.hrtime.bigint();
res.on('finish', () => {
const durationMs = Number(process.hrtime.bigint() - start) / 1e6;
logger.info({ reqId: req.id, method: req.method, url: req.url, status: res.statusCode, durationMs });
httpRequests.inc({ method: req.method, route: req.route?.path || req.path, status: res.statusCode });
});
next();
});
app.get('/metrics', createMetricsEndpoint());
CI/CD Validation
GitHub Actions workflow that validates config schema, runs type checks, and smoke-tests /healthz in staging.
# .github/workflows/ci.yml
name: CI
on: [push, pull_request]
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- run: npm ci
- run: npm run lint
- run: npm run typecheck
- run: npm test
- name: Config validation
env:
MONGODB_URI: mongodb://dummy:dummy@localhost:27017/test
REDIS_URL: redis://dummy@localhost:6379/0
run: node -e "require('./dist/config').loadConfig(process.env)"
deploy-staging:
needs: validate
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy to staging
run: ./deploy.sh staging
- name: Smoke test /healthz
run: |
sleep 10
curl -sf https://staging.example.com/healthz | jq -e '.ok == true and .env == "production"'
Docker/Process Manager Examples
Dockerfile (Multi-stage, non-root, dumb-init)
# Dockerfile
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY . .
RUN npm run build
FROM node:20-alpine AS runner
WORKDIR /app
RUN apk add --no-cache dumb-init
ENV NODE_ENV=production \
NODE_OPTIONS=--max-old-space-size=2048 \
TZ=UTC
COPY --from=builder --chown=node:node /app/node_modules ./node_modules
COPY --from=builder --chown=node:node /app/dist ./dist
COPY --from=builder --chown=node:node /app/package.json ./
USER node
EXPOSE 3000
ENTRYPOINT ["dumb-init", "--"]
CMD ["node", "dist/server.js"]
Warning: Set --max-old-space-size to ≤ 75% of your container memory limit (Node.js docs). For a 2 GiB limit, use 1536 or 2048. The 1024 MB example in earlier drafts is dangerously low for production workloads.
docker-compose.yml (Local dev stack)
# docker-compose.yml
version: '3.8'
services:
app:
build: .
ports: ["3000:3000"]
environment:
- NODE_ENV=development
- MONGODB_URI=mongodb://mongo:27017/app
- REDIS_URL=redis://redis:6379/0
- TRUST_PROXY=true
depends_on:
mongo:
condition: service_healthy
redis:
condition: service_healthy
healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://localhost:3000/healthz"]
interval: 10s
timeout: 5s
retries: 5
mongo:
image: mongo:6
ports: ["27017:27017"]
healthcheck:
test: echo 'db.runCommand("ping").ok' | mongosh localhost:27017/test --quiet
interval: 10s
timeout: 5s
retries: 5
redis:
image: redis:7-alpine
ports: ["6379:6379"]
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
nginx:
image: nginx:alpine
ports: ["80:80", "443:443"]
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ./certs:/etc/nginx/certs:ro
depends_on: [app]
nginx.conf (Reverse proxy with trust proxy headers)
# nginx.conf
events { worker_connections 1024; }
http {
upstream app {
server app:3000;
keepalive 32;
}
server {
listen 80;
listen 443 ssl http2;
ssl_certificate /etc/nginx/certs/fullchain.pem;
ssl_certificate_key /etc/nginx/certs/privkey.pem;
location / {
proxy_pass http://app;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 10s; # match REQUEST_TIMEOUT_MS
proxy_send_timeout 10s;
}
location /healthz {
proxy_pass http://app;
access_log off;
health_check interval=10s fails=3 passes=2 uri=/healthz;
}
}
}
systemd unit (Bare-metal production)
# /etc/systemd/system/node-app.service
[Unit]
Description=Node.js Application
After=network.target mongod.service redis.service
[Service]
Type=simple
User=nodeapp
Group=nodeapp
WorkingDirectory=/opt/nodeapp
EnvironmentFile=/opt/nodeapp/.env
ExecStart=/usr/bin/node --max-old-space-size=2048 dist/server.js
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
StandardOutput=journal
StandardError=journal
SyslogIdentifier=nodeapp
[Install]
WantedBy=multi-user.target
PM2 ecosystem.config.js
// ecosystem.config.js
module.exports = {
apps: [{
name: 'nodeapp',
script: 'dist/server.js',
instances: 'max',
exec_mode: 'cluster',
env: {
NODE_ENV: 'production',
NODE_OPTIONS: '--max-old-space-size=2048',
},
env_file: '.env',
error_file: '/var/log/nodeapp/error.log',
out_file: '/var/log/nodeapp/out.log',
log_date_format: 'YYYY-MM-DD HH:mm:ss Z',
kill_timeout: 25000,
wait_ready: true,
listen_timeout: 10000,
}],
};
Troubleshooting Decision Tree
Follow this flowchart when /healthz fails or behaves unexpectedly.
Health check fails?
│
├─▶ Check logs for validation errors at startup
│ └─▶ Run: journalctl -u nodeapp -n 100 --no-pager
│ └─▶ Look for: "Configuration validation failed", "Invalid boolean", "Invalid integer"
│
├─▶ Check PORT binding
│ ├─▶ Linux/macOS: lsof -iTCP:3000 -sTCP:LISTEN
│ ├─▶ Windows: Get-NetTCPConnection -LocalPort 3000 -State Listen
│ └─▶ If EADDRINUSE: kill conflicting process or change PORT
│
├─▶ Check DB connectivity
│ ├─▶ MongoDB: mongosh "mongodb://user:pass@host:27017/app" --eval "db.runCommand({ping:1})"
│ ├─▶ Redis: redis-cli -u redis://host:6379 ping
│ └─▶ Verify security groups / firewall rules allow outbound
│
├─▶ Check reverse proxy → app connectivity
│ ├─▶ curl -v http://localhost:3000/healthz (from proxy host)
│ ├─▶ Verify TRUST_PROXY=true and X-Forwarded-Proto=https headers arrive
│ └─▶ Check nginx error.log for upstream timeouts
│
└─▶ Check resource exhaustion
├─▶ Memory: process.memoryUsage() in logs or `docker stats`
├─▶ CPU: top -p $(pgrep -f node)
└─▶ File descriptors: lsof -p $(pgrep -f node) | wc -l
Performance Tuning Guide
| Parameter | When to Adjust | Guidance |
|---|---|---|
| DB_POOL_MAX | DB saturation, connection timeouts | Formula: (cpu_cores * 2) + effective_spindle_count. For 4-core SSD: 10. Monitor db.serverStatus().connections.current. |
| keepAliveTimeout / headersTimeout | High latency, connection churn | Set keepAliveTimeout = proxy_read_timeout + 1000-2000ms. headersTimeout = keepAliveTimeout + 1000ms. |
| --max-old-space-size | OOM kills, high GC pause | ≤ 75% of container memory limit. For 2 GiB cgroup limit: 1536-2048. |
| REQUEST_TIMEOUT_MS | Upstream latency spikes | Set to p99 latency + 20% buffer. Never exceed reverse proxy proxy_read_timeout. |
| LOG_LEVEL | Log volume costs | info for production. debug only during incident investigation. |
Security Hardening
Layer defenses at the application and infrastructure level.
// security.ts
import helmet from 'helmet';
import rateLimit from 'express-rate-limit';
import cors from 'cors';
import { Config } from './config';
export function createSecurityMiddleware(cfg: Config) {
return [
helmet({
contentSecurityPolicy: false, // configure per-app
hsts: cfg.TRUST_PROXY, // let reverse proxy handle HSTS
referrerPolicy: { policy: 'strict-origin-when-cross-origin' },
}),
cors({
origin: process.env.CORS_ORIGIN?.split(',') || false, // false = reflect request origin
credentials: true,
methods: ['GET', 'HEAD', 'PUT', 'PATCH', 'POST', 'DELETE'],
}),
rateLimit({
windowMs: 60 * 1000,
max: 1000,
standardHeaders: true,
legacyHeaders: false,
keyGenerator: (req) => req.ip,
skip: (req) => req.path === '/healthz' || req.path === '/metrics',
}),
];
}
TLS for MongoDB/Redis: Use mongodb+srv:// (enforces TLS) and rediss:// URLs. Verify certificates in production:
// mongoose TLS options (when not using mongodb+srv)
mongoose.connect(uri, {
tls: true,
tlsCAFile: '/etc/ssl/certs/ca-certificates.crt',
// tlsCertificateKeyFile: '/path/to/client.pem', // if mutual TLS
});
Load Test Script
Verify timeout enforcement, keep-alive behavior, and graceful degradation under load.
// load-test.js (run with: node load-test.js)
import autocannon from 'autocannon';
const BASE = process.env.TARGET_URL || 'http://localhost:3000';
async function run() {
// 1. Health check baseline
console.log('=== Health check ===');
await autocannon({ url: `${BASE}/healthz`, connections: 10, duration: 10 });
// 2. Slow route timeout verification (assumes /slow sleeps 15s, timeout=10s)
console.log('=== Timeout verification (expect 503) ===');
const timeoutResult = await autocannon({
url: `${BASE}/slow`,
connections: 20,
duration: 15,
timeout: 20000,
});
console.log('Non-2xx:', timeoutResult.non2xx);
// 3. Keep-alive connection reuse
console.log('=== Keep-alive reuse ===');
await autocannon({
url: `${BASE}/healthz`,
connections: 50,
duration: 30,
pipelining: 10,
});
}
run().catch(console.error);
Expected: /slow returns 503 after ~10s (REQUEST_TIMEOUT_MS). Connection count stays low due to keep-alive reuse.
Rollback Verification Script
Automate rollback validation by comparing CONFIG_VERSION from /healthz against git tags.
#!/usr/bin/env bash
# rollback-verify.sh
set -euo pipefail
CURRENT_VERSION=$(curl -sf http://localhost:3000/healthz | jq -r .version)
echo "Current CONFIG_VERSION: $CURRENT_VERSION"
# Find previous git tag matching pattern
PREV_TAG=$(git tag --list 'config-*' --sort=-v:refname | head -n 2 | tail -n 1)
if [ -z "$PREV_TAG" ]; then
echo "No previous config tag found"
exit 1
fi
echo "Previous config tag: $PREV_TAG"
# Extract version from tag (assumes tag format: config-YYYY-MM-DD.N)
PREV_VERSION=${PREV_TAG#config-}
echo "Rolling back to version: $PREV_VERSION"
# Restore env from tag (assumes you store .env per tag or use git show)
git show "$PREV_TAG:.env" > .env.rollback
source .env.rollback
export CONFIG_VERSION="$PREV_VERSION"
# Restart (systemd example)
sudo systemctl restart nodeapp
# Verify
sleep 5
NEW_VERSION=$(curl -sf http://localhost:3000/healthz | jq -r .version)
if [ "$NEW_VERSION" == "$PREV_VERSION" ]; then
echo "✅ Rollback verified: $NEW_VERSION"
exit 0
else
echo "❌ Rollback failed: expected $PREV_VERSION, got $NEW_VERSION"
exit 1
fi
Operations Checklist
Use this as a repeatable runbook for every configuration change.
Pre-change
- Capture Node, npm, and dependency versions (
node -v,npm ls --depth=0) - Confirm topology: reverse proxy type, TLS termination point, current TRUST_PROXY setting
- Define narrow pilot scope (single instance, canary route, or feature flag)
- Specify verification command and expected output
Implement
- Centralize configuration in config.ts with Zod schema validation
- Set safe defaults: NODE_ENV=production, LOG_LEVEL=info, KEEP_ALIVE=true, TRUST_PROXY per topology
- Redact secrets in logs and /configz; never log connection strings
- Tag CONFIG_VERSION with monotonic label (YYYY-MM-DD.N)
Verify
- Start service and confirm listening on expected PORT (
lsof -iTCP:$PORT -sTCP:LISTEN) - Probe /healthz and validate env and CONFIG_VERSION fields
- Confirm timeouts: hit slow route, expect 503 within REQUEST_TIMEOUT_MS
- Confirm keep-alive:
curl -vshows Connection: keep-alive - Confirm proxy semantics: req.ip matches client IP when TRUST_PROXY=true
- Check for warnings:
NODE_OPTIONS="--trace-warnings" node server.js
Operate
- Monitor latency (p50/p95/p99), error rate, and resource usage (CPU, RSS, heap)
- Watch DB connection counts: db.serverStatus().connections (Mongo), connected_clients (Redis)
- Check log volume against baseline when changing LOG_LEVEL
Rollback
- If any verification fails, restore prior CONFIG_VERSION and its env values
- Restart process and re-verify /healthz shows previous version
- Document root cause and add validation rule to prevent recurrence
Review (weekly or per release)
- Re-run inventory and diff against deployed config
- Prune unused keys and remove dead feature flags
- Refresh defaults to match observed safe values
Conclusion
Configuration mistakes in Node.js are common but highly preventable. Centralize and validate inputs at startup using a schema like Zod, tag and observe your configuration at runtime through /healthz and /configz, and keep changes small and reversible with CONFIG_VERSION tagging and toggle-based feature flags. Start with a narrow pilot: enforce NODE_ENV=production, validate numeric environment variables, enable HTTP keep-alive with explicit timeouts derived from REQUEST_TIMEOUT_MS, and set TRUST_PROXY correctly when fronted by a reverse proxy. Tie each change to a clear verification step — curl commands, log assertions, metric checks — and keep a rollback path ready with scripted verification. With these habits in place, you will ship safer changes, diagnose issues faster, and reduce the operational risk of your Node.js services.