High availability and fault tolerance

Assume anything can fail. Spread each tier across Availability Zones, remove single points of failure, automate recovery, and pick a disaster recovery strategy whose RPO and RTO match what the business will pay for.

Availability building blocks

• Run every tier in at least two Availability Zones: an Auto Scaling group across AZs behind a load balancer replaces failed instances on its own.
• Amazon RDS Multi-AZ: A synchronous standby in another AZ with automatic failover. With a Multi-AZ DB instance the standby serves no reads; a Multi-AZ DB cluster has two readable standbys.
• Amazon Aurora: Keeps six copies of data across three AZs; an Aurora Replica is promoted automatically if the writer fails.
• Regional services: S3 (except One Zone classes), DynamoDB, SQS, and Lambda already spread across AZs in a Region with no design work.
• Zonal resources: An EBS volume, a single EC2 instance, and a zonal NAT gateway each live in one AZ. Protect them with snapshots, Auto Scaling, and one NAT gateway per AZ (or a regional NAT gateway).

Amazon Route 53 and failover

• Health checks: Route 53 checks endpoints and stops returning unhealthy ones.
• Routing policies: Simple, weighted (split traffic), latency (closest Region by latency), failover (active-passive), geolocation (by user location), geoproximity (by distance, with bias), multivalue answer, and IP-based.
• Alias records: Point a name, including the zone apex, at a load balancer, CloudFront distribution, or S3 website.
• AWS Global Accelerator: Two static anycast IP addresses that route users over the AWS network to the nearest healthy Regional endpoint, and fail over between Regions in seconds without DNS caching delays.

Disaster recovery strategies

• RPO and RTO: Recovery point objective: how much data you can lose, measured in time. Recovery time objective: how long you can be down.
• Backup and restore: Back up to another Region and rebuild when needed. Lowest cost; RPO and RTO in hours.
• Pilot light: Data is replicated continuously and core infrastructure is defined, but application servers are off or minimal until a disaster. RTO in tens of minutes to hours.
• Warm standby: A scaled-down but fully working copy runs all the time and is scaled up on failover. RTO in minutes.
• Multi-site active-active: Full capacity serves traffic in two or more Regions. Near-zero RTO and RPO, and the highest cost.
• Tools: AWS Backup cross-Region copies, S3 Cross-Region Replication, Aurora Global Database (cross-Region replication with typical lag under a second), DynamoDB global tables (multi-Region, multi-active), AMIs copied between Regions, and infrastructure as code to rebuild.

Automation and immutable infrastructure

• Immutable infrastructure: Replace servers instead of changing them: bake a new AMI or image and roll it out. Nothing drifts, and rollback means redeploying the previous version.
• AWS CloudFormation: Declares the whole environment as code so it can be rebuilt the same way in any Region; drift detection shows manual changes.
• AWS Config records configuration changes and checks them against rules; Systems Manager automates patching and runbooks.
• Blue/green deployments run the new version next to the old one and switch traffic, so a bad release is a quick switch back.

Visibility, proxies, and quotas

• Amazon CloudWatch: Metrics, logs, and alarms. Alarm on the metrics the business cares about (error rate, latency, queue age), not just CPU.
• AWS X-Ray: Traces requests through distributed services to show where latency and errors come from.
• Amazon RDS Proxy: Pools and shares database connections, absorbs connection storms from Lambda, and shortens failover time because the application keeps its connections to the proxy.
• Service quotas and throttling: Every account has limits. Raise quotas in a standby Region in advance, or a failover can't scale. Clients should retry throttled calls with exponential backoff and jitter.

Making legacy applications more reliable

• When the code can't change, improve what's around it: put the app behind a load balancer with health checks, run it in an Auto Scaling group of one so a failed instance is replaced, and keep its data on EFS or in RDS Multi-AZ.
• EC2 automatic recovery moves an instance to healthy hardware after a host failure, keeping its instance ID, private IP addresses, and Elastic IP.
• Route 53 failover records can send users to a static maintenance page in S3 if the application fails its health check.

Purpose-built managed services

• AWS AI services such as Amazon Comprehend (text analysis), Amazon Polly (text to speech), Amazon Transcribe (speech to text), Amazon Translate, Amazon Rekognition (images and video), Amazon Textract (documents), and Amazon Lex (chatbots) are managed and highly available, so you call an API instead of running models yourself.
• Storage durability and availability differ: S3 Standard stores data across at least three AZs; S3 One Zone classes and EFS One Zone keep it in one AZ for a lower price.