Capacity and Load Planning

Know your demand

Capacity planning starts with an honest estimate of load. How many requests per second at a normal time, and at peak? What drives the peaks — a daily batch, end-of-month processing, a marketing event? Averages mislead here: a system sized for the daily mean will fall over during the spike that actually matters, so plan around the peak, not the typical.

You rarely need perfect numbers, just the right order of magnitude and a sense of the shape. Knowing that load triples at month-end, or arrives in a morning burst, tells you far more about what to provision than a single average request rate ever could.

Find the bottleneck

Every system has one constraint that caps its throughput before the others — a database connection pool, a single-threaded stage, an external dependency's own limit, or memory. Total capacity is set by that bottleneck, not by your most abundant resource, so adding capacity elsewhere achieves nothing until the real constraint is addressed.

Identify it deliberately rather than guessing. Under increasing load, watch which resource saturates first; that is your bottleneck. Spending effort optimising anything else is wasted until the binding constraint moves. As you raise one limit, the bottleneck shifts, so this is an ongoing exercise, not a one-off.

Test before users do

Assumptions about capacity are worth little until tested. Load testing drives realistic, scaled traffic at a staging environment to find where performance degrades and where it breaks. The aim is to discover your limits in a controlled setting, on your schedule, rather than during a real spike with users watching.

# ramp load until something gives
for rps in [50, 100, 200, 400, 800]:
  run_load(rps, duration='5m')
  record(latency_p99, error_rate)

Watch where latency climbs and errors appear; that knee in the curve is your practical ceiling. Use traffic that resembles production — realistic payload sizes, a representative mix of endpoints, and warm caches — because a test that exercises only the cheapest path flatters you and hides the limit that will actually bite. Run the test against an environment that mirrors production as closely as you can, since a bottleneck found in a tiny staging box may not be the one that constrains you live.

Headroom and graceful overload

Run with headroom — deliberate spare capacity above your expected peak — so a surprise surge does not immediately tip you over. A system running at the edge has nowhere to go when demand jumps; one with margin absorbs the spike and gives you time to react.

Plan, too, for what happens when you are overwhelmed despite the margin. Shedding or queuing excess load so the system slows rather than collapses is far better than an uncontrolled failure. This connects to graceful degradation: an overloaded system should bend, not break.