Backups and disaster recovery

Three different jobs

HA is not backup. If someone runs DELETE FROM students by mistake, a Multi-AZ standby deletes the same rows a millisecond later, because it copies every change. Only a backup from before the mistake brings the data back.

RPO and RTO

The recovery point objective (RPO) is the maximum amount of data, measured in time, you can afford to lose. If backups run nightly at 2 am and the disaster hits at 2 pm, you lose 12 hours of data, so your RPO with that design is up to 24 hours. The recovery time objective (RTO) is the maximum time the service may be down before it is working again. If restoring and switching over takes 4 hours, your RTO must be at least 4 hours.

The four DR strategies

Pick the cheapest strategy that meets the RPO and RTO the business has agreed. A college library catalogue can live with backup and restore. A stock broker's order system may need multi-site.

AWS Backup

AWS Backup is a central service that backs up many AWS services (such as EBS volumes, EC2, RDS and Aurora, DynamoDB, EFS and S3) from one place. A backup plan holds rules: how often (for example daily at 1 am), how long to keep each backup, and when to move older ones to cheaper cold storage. Resources are assigned to a plan by tag (for example backup=daily) or by ID. Backups are stored in a backup vault, an encrypted container with its own access policy. Copy rules can copy backups to a vault in another region and, with AWS Organizations, to another account.

aws backup list-backup-plans
aws backup start-backup-job --backup-vault-name results-vault \
  --resource-arn arn:aws:rds:ap-south-1:111122223333:db:results-db \
  --iam-role-arn arn:aws:iam::111122223333:role/service-role/AWSBackupDefaultServiceRole

Snapshots, versioning and replication

A snapshot is a point-in-time copy of a disk or database. EBS snapshots are incremental (only changed blocks are stored after the first one) and can be copied to another region. RDS automated backups let you restore a database to any second within the retention period (1 to 35 days); a restore always creates a new database instance, so the application must be pointed at it.

For S3, versioning keeps every old version of an object, so an overwrite or delete can be undone. Replication copies new objects to another bucket, for example Cross-Region Replication from Mumbai to Hyderabad; it needs versioning on both buckets. Check how your replication rules handle deletes, so a mistaken delete is not simply copied too.

The 3-2-1 rule and ransomware

The classic 3-2-1 rule says: keep 3 copies of important data, on 2 different kinds of storage, with 1 copy off-site. In the cloud, "off-site" means another region and, ideally, another account. Ransomware attackers who steal admin credentials try to delete backups before they encrypt your data. Defend with immutable backups, which cannot be changed or deleted until their retention ends. AWS Backup Vault Lock locks a vault so that, in compliance mode, nobody, not even the root user, can delete backups early or shorten retention. S3 Object Lock does the same for objects in a versioned bucket (write once, read many). Add a separate backup account with very few people allowed in, MFA, and alerts on backup deletion.

Test your restores

A backup you have never restored is only a hope. Schedule restore drills: restore last night's database into a test account, check the row counts, time how long it took, and compare that with your RTO. AWS Backup has a restore testing feature that automates part of this. Once or twice a year, run a "game day" where the team practises failing over to the recovery region.