Performance Testing on AWS: EC2, RDS, and Lambda Considerations

Performance testing on AWS introduces considerations that don’t exist in traditional on-premises environments. Instance types, auto-scaling behaviour, network topology, and managed service limitations all affect how you design and interpret your tests. Here’s what I’ve learned from testing applications across various AWS services.

EC2 Instance Considerations

Instance Type Selection

The instance type directly impacts your baseline performance. Different instance families are optimised for different workloads:

Family	Optimised For	Performance Testing Notes
T3/T3a	Burstable	CPU credits affect sustained load tests
M5/M6i	General purpose	Consistent baseline, good for most testing
C5/C6i	Compute	CPU-bound applications
R5/R6i	Memory	In-memory caching, large datasets
I3	Storage	Database servers, high IOPS workloads

Burstable instances (T-series) are problematic for performance testing. They accumulate CPU credits when idle and spend them under load. A T3.medium might handle your load test initially, then throttle dramatically when credits run out.

For performance testing, I recommend using fixed-performance instances (M5, C5) rather than burstable types to get consistent, repeatable results.

Network Performance

Instance size affects network bandwidth. A t3.micro has significantly less network capacity than a c5.4xlarge. If your test shows network-related bottlenecks, check whether you’re hitting instance-level network limits rather than application issues.

# Check network performance on the instance
ethtool -S eth0 | grep -i error
sar -n DEV 1 10

RDS Database Testing

Connection Pool Sizing

RDS instances have maximum connection limits based on instance size:

Instance Class	Max Connections (MySQL)	Max Connections (PostgreSQL)
db.t3.micro	66	112
db.t3.small	150	225
db.r5.large	1365	1600
db.r5.xlarge	2730	3200

Your application’s connection pool must stay within these limits. During load tests, monitor:

-- MySQL
SHOW STATUS LIKE 'Threads_connected';
SHOW STATUS LIKE 'Max_used_connections';

-- PostgreSQL
SELECT count(*) FROM pg_stat_activity WHERE state = 'active';

Read Replica Lag

If your application uses read replicas, test for replication lag under load:

-- MySQL
SHOW SLAVE STATUS\G
-- Look for Seconds_Behind_Master

-- PostgreSQL
SELECT EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))::INT;

I’ve seen applications behave correctly under normal load but show data inconsistencies when replica lag increases during peak traffic.

Storage IOPS

RDS storage performance depends on your provisioned IOPS or volume type. For performance testing:

1. Monitor CloudWatch metrics: ReadIOPS, WriteIOPS, ReadLatency, WriteLatency
2. Look for IOPS throttling if using gp2 volumes
3. Consider provisioned IOPS (io1/io2) for consistent performance

Lambda Cold Starts

Lambda functions introduce cold start latency that can significantly impact user-facing response times.

Measuring Cold Start Impact

Cold starts occur when:

• A function hasn’t been invoked recently (typically 15-30 minutes)
• Concurrent invocations exceed warm instances
• You deploy a new version

To measure cold start impact in your load test:

// k6 example - detect cold starts via response time
import http from 'k6/http';
import { Trend } from 'k6/metrics';

const coldStartTrend = new Trend('cold_start_duration');

export default function () {
  const res = http.get('https://your-api-gateway.execute-api.eu-west-1.amazonaws.com/prod/endpoint');

  // Cold starts typically >1000ms, warm invocations <100ms
  if (res.timings.duration > 1000) {
    coldStartTrend.add(res.timings.duration);
  }
}

Cold Start by Runtime

Cold start duration varies significantly by runtime:

Runtime	Typical Cold Start	Notes
Python	200-500ms	Fast, small package sizes help
Node.js	200-500ms	Similar to Python
Java	3-10 seconds	JVM initialisation overhead
.NET	1-3 seconds	CLR initialisation
Go	100-200ms	Compiled, minimal runtime

For latency-sensitive applications using Java or .NET, consider Provisioned Concurrency to eliminate cold starts.

Testing Concurrency Scaling

Lambda scales automatically, but there are limits:

# Check your account's concurrent execution limit
aws lambda get-account-settings --query 'AccountLimit.ConcurrentExecutions'

Design your load test to verify scaling behaviour:

export const options = {
  scenarios: {
    burst: {
      executor: 'constant-arrival-rate',
      rate: 100,           // 100 requests per second
      timeUnit: '1s',
      duration: '2m',
      preAllocatedVUs: 100,
    },
  },
};

Monitor CloudWatch metrics for throttling: Throttles, ConcurrentExecutions.

Auto Scaling Behaviour

Testing Scale-Out Latency

Auto Scaling groups don’t add capacity instantly. When testing applications behind an ASG:

1. Start with baseline load that the current capacity handles
2. Ramp up to trigger scaling events
3. Measure response time degradation during scaling
4. Verify new instances join and start serving traffic

Typical EC2 scaling takes 2-5 minutes from trigger to serving traffic. Container-based scaling (ECS, EKS) is faster but still not instant.

Target Tracking vs Step Scaling

Test how different scaling policies respond to your load profile:

# Monitor scaling activities during the test
aws autoscaling describe-scaling-activities \
  --auto-scaling-group-name my-asg \
  --query 'Activities[*].[StartTime,StatusCode,Description]' \
  --output table

Load Test Infrastructure Placement

Where you run your load generators matters:

Location	Pros	Cons
Same VPC	No data transfer costs, low latency	May not represent real user experience
Different region	Realistic latency	Data transfer costs, network variability
On-premises	Matches actual user path	Internet variability affects results

For AWS applications, I typically run initial tests from within AWS (same region, different AZ) for consistent baselines, then validate with tests from external locations.

Monitoring During Tests

Essential CloudWatch metrics to capture:

EC2:

• CPUUtilization, NetworkIn/Out, DiskReadOps/WriteOps

RDS:

• CPUUtilization, DatabaseConnections, ReadIOPS/WriteIOPS, ReadLatency/WriteLatency

Lambda:

• Invocations, Duration, Errors, Throttles, ConcurrentExecutions

Application Load Balancer:

• RequestCount, TargetResponseTime, HTTPCode_Target_2XX_Count, HTTPCode_Target_5XX_Count

Set up a CloudWatch dashboard before testing to correlate application metrics with load test results.

Cost Considerations

Performance testing on AWS incurs costs. To manage expenses:

1. Use spot instances for load generators (acceptable for most test scenarios)
2. Schedule tests during off-peak hours if using production infrastructure
3. Clean up resources immediately after testing
4. Set billing alerts before starting large-scale tests

A 1-hour load test with 10 c5.xlarge instances costs roughly $17, but forgetting to terminate them costs $1,224/month.

Performance testing on AWS requires understanding both your application and the underlying platform behaviours. The cloud adds complexity, but also provides flexibility and scalability that’s difficult to achieve with traditional infrastructure.