Designing for high availability

What availability means

Availability is the percentage of time a system does its job for users. If a results website is down for 1 hour in a 100-hour period, its availability is 99%. Engineers talk about "nines": 99.9% is "three nines", 99.99% is "four nines". A service level agreement (SLA) is a promise in a contract, often with a refund (service credit) if it is broken. A service level objective (SLO) is the internal target a team aims for.

Every extra nine costs much more money and effort. A college notice board may be fine at 99.5%. A payment switch handling UPI traffic needs far more. Choose the target from what downtime really costs the business, then design for it.

Single points of failure

A single point of failure (SPOF) is any part whose failure takes the whole system down. The classic beginner setup has many: one EC2 instance running both the web app and the database, in one Availability Zone, with files on its own disk. If that instance, its disk or its zone has a problem, everything stops. Other SPOFs hide in plain sight: one NAT gateway, one DNS provider, or one engineer who alone knows the root password.

Multi-AZ design

An Availability Zone (AZ) is one or more data centres in a region with their own power, cooling and network, close enough to other AZs for fast connections. Zones are built to fail independently. So the basic rule of cloud HA is: run at least two of everything in at least two AZs. Put app servers in two or three AZs behind a load balancer, use Amazon RDS Multi-AZ so a standby database in another zone takes over automatically, and use services such as S3 that already store data across several zones.

Stateless app servers

State is data that must be remembered between requests: a login session, a shopping cart, an uploaded file. If a server keeps state on its own memory or disk, users lose it when that server dies, and the load balancer cannot freely send them to another server. A stateless server keeps nothing important locally. Sessions go to a cache such as Amazon ElastiCache or a table in DynamoDB, files go to S3, and records go to the database. Then any server can answer any request, and you can add, remove or replace servers at any time.

Health checks and graceful degradation

A health check is a regular test, such as the load balancer calling GET /health every 15 seconds. A server that fails a few checks in a row is taken out of rotation, and an Auto Scaling group can replace it. Keep the health endpoint honest but simple: if it checks the database and the database blips, every server can be marked unhealthy at once.

Graceful degradation means that when one part fails, the rest keeps working in a reduced way. If the recommendation service is down, the shopping page shows without recommendations. If the database is overloaded, the results site can serve cached pages or switch to read-only mode instead of showing errors to everyone.

Availability maths: series and parallel

Components in series all have to work, like a chain. Their availabilities multiply, so the total is lower than the weakest part. A web tier at 99.9% that needs a database at 99.95% gives 0.999 × 0.9995 = 0.9985, about 99.85%.

Redundant components in parallel only fail if all of them fail at once. With n copies each of availability a, the chance that all are down is (1 − a) to the power n, so the availability is 1 − (1 − a)^n. Two servers at 99% each: 1 − 0.01 × 0.01 = 0.9999, which is 99.99%. Redundancy turns two ordinary servers into four nines, as long as they really fail independently.

lb = 0.9999                       # one managed load balancer
app = 1 - (1 - 0.995) ** 2        # two app servers at 99.5% in parallel: 0.999975
db = 0.9995                       # Multi-AZ database
total = lb * app * db             # the three tiers in series
print(round(total * 100, 3))      # 99.938