High-performing, elastic compute

Pick the compute model and size that fits the work, scale it on a metric that tracks demand, and decouple components so each can scale on its own.

Amazon EC2 instance choices

• General purpose (M, T): Balanced CPU and memory. T instances are burstable: cheap for workloads that are usually idle and sometimes busy.
• Compute optimized (C): High CPU per GB of memory: batch processing, media encoding, high-traffic web servers, scientific modeling.
• Memory optimized (R, X): Large memory for in-memory databases, caches, and real-time analytics.
• Accelerated computing (P, G, Inf, Trn): GPUs or AWS chips for machine learning and graphics.
• Storage optimized (I, D): High-speed local NVMe or dense HDD storage for databases and data warehouses that need fast local disk.
• AWS Graviton (Arm) instances usually give better price performance when the software supports Arm.

Placement and networking

• Cluster placement group: Instances packed close together in one AZ for the lowest latency and highest throughput between them (HPC, tightly coupled jobs).
• Spread placement group: Each instance on distinct hardware to reduce correlated failures. Limited to seven running instances per AZ per group.
• Partition placement group: Groups of instances on separate racks, for large distributed systems such as Hadoop, Cassandra, and Kafka.
• Enhanced networking (ENA) raises bandwidth and lowers latency; Elastic Fabric Adapter (EFA) adds low-latency networking for HPC and machine learning jobs.

Scaling

• Amazon EC2 Auto Scaling: Keeps a group of instances at the right size and replaces unhealthy ones.
• Target tracking: Keeps a metric near a value (for example, average CPU at 50% or requests per target at 1,000). The default choice.
• Step and simple scaling: Add or remove capacity in steps when a CloudWatch alarm fires.
• Scheduled and predictive scaling: Scale ahead of known patterns (business hours) or forecasted daily and weekly cycles.
• Choosing the metric: Scale on what tracks demand: CPU for compute-bound work, ALB requests per target for web tiers, SQS backlog per instance for queue workers.
• AWS Auto Scaling and Application Auto Scaling: Scale other resources the same way: ECS services, DynamoDB capacity, Aurora replicas, Lambda provisioned concurrency.
• Warm pools keep pre-initialized instances ready, cutting scale-out time for apps with slow startup.

Serverless and container sizing

• Lambda memory: Set from 128 MB to 10,240 MB. CPU is allocated in proportion to memory, so more memory often makes a CPU-bound function finish faster and cost about the same.
• Lambda concurrency: Provisioned concurrency keeps functions initialized to avoid cold starts; reserved concurrency guarantees and caps a function's concurrency.
• Lambda runs up to 15 minutes per invocation. Longer jobs belong on Fargate, ECS, AWS Batch, or Step Functions.
• Fargate tasks are sized by vCPU and memory per task; ECS service auto scaling adds tasks on CPU, memory, or request metrics.

Batch, big data, and edge compute

• AWS Batch: Runs batch jobs from queues on managed compute (EC2, Spot, or Fargate) and handles scheduling and scaling.
• Amazon EMR: Managed big data frameworks such as Apache Spark and Hadoop, on clusters or serverless.
• AWS Elastic Beanstalk: Upload application code and Beanstalk provisions and scales the environment for you.
• Edge and distributed compute: AWS Local Zones and AWS Wavelength put compute closer to users and 5G devices; AWS Outposts runs AWS infrastructure on premises; Lambda@Edge and CloudFront Functions run code at edge locations.

Decoupling so components scale independently

• An SQS queue between a web tier and workers lets each tier scale on its own signal: the web tier on requests, the workers on queue depth.
• SNS or EventBridge fan-out lets new consumers be added without touching the producer.