status
this chapter is in active development
expect live edits and rapid iteration (except for when i am really busy with other stuff) while this material is written.
what it means for a system to handle growth, and why it is harder than adding servers.
“the wise engineer does not scale until the system demands it”
needing to scale is a sign that something worked. it means you have users, traffic, real demand. it is also the point where your original design assumptions get tested against reality.
scalability is not "add more machines and watch it go faster." it is the question of whether the work can be divided up at all, and whether your current design makes that possible.
stateless computation scales easily. take a request, do work, return a response, nothing persisted between calls. run ten copies behind a load balancer. done.
state does not scale easily. if your application needs to know what happened before, shopping cart contents, session tokens, which records a user has already seen, you now have a coordination problem. either every replica has access to the same state store, or you have to route requests to the right replica, or you accept that replicas can disagree.
"just add more servers" breaks down here. the bottleneck is almost never computation. it is shared state.
vertical scaling means adding resources to one machine: more cpu cores, more ram, faster storage. it is simple, requires no code changes, and works until it does not. hardware has limits. a single machine can only grow so large before cost becomes prohibitive or the ceiling is hit entirely. more importantly, one machine is one failure domain.
horizontal scaling means adding more machines to the pool. it is cheaper at scale, eliminates the single-failure-domain problem, and lets you scale specific components independently. the cost is coordination: requests need routing, shared state needs a shared store, and any statefulness in the application becomes a design problem rather than a deployment footnote.
most production systems end up with both: vertically scale individual nodes to a sensible size, then scale horizontally across multiple nodes.
the scale cube from the art of scalability is useful here. you can scale a system along three axes:
most scaling problems are solved by a combination of these. the question is which combination your current architecture supports.
if any part of your system cannot be parallelized, that part limits your maximum speedup regardless of how many machines you add. if 5% of a workload is sequential, you top out at 20x improvement no matter how many cores you throw at it.
practical implication: find the bottleneck before scaling. throwing capacity at the wrong layer is expensive and achieves nothing. a single slow database query will cap your throughput long before you run out of application servers.
most of what makes scaling genuinely hard is stateful services: databases, caches, queues, anything that holds data. techniques here include:
every one of these involves a tradeoff. read replicas trade consistency. caching trades freshness. sharding trades simplicity. scaling is choosing which tradeoff you can live with.