availability
what the nines actually cost, and why your dependencies spend your budget for you.
what the nines actually cost, and why your dependencies spend your budget for you.
availability is the fraction of time a system is operational and accepting requests. it gets quoted as a number of nines, and the gap between two adjacent nines sounds trivial until it is converted into time.
availability downtime/year downtime/month downtime/week
99.000% 3.65 days 7.30 hrs 1.68 hrs
99.900% 8.76 hrs 43.8 min 10.1 min
99.990% 52.6 min 4.4 min 1.0 min
99.999% 5.3 min 0.4 min 0.1 min99% sounds respectable. 3.65 days a year does not.
the weekly column is the one worth internalizing, because that is the interval you actually operate on. at three nines you can absorb a ten-minute incident a week. at four nines, a single bad deploy that takes a minute to roll back has spent your entire weekly budget.
each nine costs roughly an order of magnitude more than the last. 99% to 99.9% might be one replica and a health check. 99.99% to 99.999% means no single points of failure anywhere, automated failover that is tested rather than assumed, and deploys that cannot take the system down, which is an organizational change more than a technical one.
three terms that get used interchangeably and should not be. the google SRE workbook is the reference here.
an sli is a measurement: p99 latency, request success rate, the raw number your monitoring emits.
an slo is a target on that measurement. "99.9% of requests succeed over 28 days". it is internal, and it is a decision about how much unreliability you are willing to ship.
an sla is a contract with money attached to missing the slo. slas are external and should always be looser than the slo, so that missing your internal target is a signal rather than an invoice.
the ordering matters: define the sli first, because a target on an unmeasured quantity is a wish. the practical consequence of an slo is the error budget. at 99.9% over 28 days you have about 40 minutes of failure to spend. that budget is what makes reliability negotiable instead of aspirational: if it is unspent, you are shipping too slowly; if it is gone, you stop shipping features and fix things.
components in series all have to work, so their availabilities multiply. that compounds faster than intuition suggests.
99.9900%99.9900%99.9000%99.8800%10.5 hrsa four-nines service with four dependencies:
+ your service 99.9900% composite 99.9900% budget 52.6 min/yr
+ primary database 99.9900% composite 99.9800% budget 1.75 hrs/yr
+ cache 99.9000% composite 99.8800% budget 10.51 hrs/yr
+ auth service 99.9500% composite 99.8301% budget 14.88 hrs/yr
+ object storage 99.9990% composite 99.8291% budget 14.97 hrs/yrthe service that was designed for 52 minutes of downtime a year is now looking at 15 hours, and it never became less reliable itself. the cache did most of the damage. one dependency at three nines dragged the composite below what any amount of work on your own code could recover.
two things follow. the composite is always worse than your weakest dependency, so a single three-nines component sets a ceiling nothing upstream can beat. and an slo published without adding up your dependencies is a number you have no mechanism to keep.
redundancy runs the math the other way. two replicas that fail independently at 0.1% each are down together with probability 0.000001. six nines. the catch is the word independently. a bad deploy hits both replicas. a zone outage takes every node in the zone. a shared dependency fails and both replicas fail with it. correlated failure is why "we have two copies" reliably delivers less than the arithmetic promises, and why the interesting question about redundancy is always what the copies share.
availability math does not care why you were down. a maintenance window is downtime, and if your deploy process requires one, your deploy cadence is capped by your availability target.
this is the mechanical reason zero-downtime deployment techniques exist. rolling updates, blue-green, canaries. they are not deployment hygiene, they are how you decouple release frequency from the error budget.
availability says nothing about correctness. a service returning 200 OK with wrong data for every request is 100% available and useless. that is the distinction reliability covers.
it also says nothing about latency unless you build latency into the sli. a service answering in 30 seconds is available by any naive definition and unusable by any real one. this is why serious slos are almost always phrased as availability and responsiveness together. "99.9% of requests succeed within 300ms", because the failure mode users actually experience is a request that technically completed.
and it is an aggregate, which hides things. 99.9% overall can mean everyone saw slightly degraded service, or it can mean 0.1% of your users had a completely broken day, every day. those need different responses and the single number distinguishes neither.