Load balancing and observability on AWS

The load balancer is where traffic enters, and CloudWatch is where the evidence of what happened to it ends up. Both are the AWS spellings of things earlier courses covered generically — the system-design course's load-balancing lesson, the observability course's metrics and alerting — and this lesson is the specifics: what an ALB actually does to a request, what a health check must be, how the metrics and logs land in CloudWatch, which alarms are worth having, and where the bill comes from.

#The Application Load Balancer

An ALB is a layer-7 (HTTP) load balancer: it terminates TLS with a certificate from ACM (free, auto-renewed), listens on 443, evaluates rules — by host, path, header, query — and forwards to a target group, which is a set of targets (EC2 instances, ECS tasks, IP addresses, a Lambda) with a health check. It is nginx's routing from the CI/CD course as a managed service, with the ingress controller's job on EKS, and it adds the forwarded headers the same way: X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Port, so Spring's forwarded-headers strategy applies unchanged.

What it does that nginx on one host could not: it is itself multi-AZ and scales with traffic; it does connection draining (deregistration delay) so a task being replaced finishes its requests; it can do sticky sessions, which the Kubernetes lesson said not to need; and it integrates with WAF for the rate-based and signature rules, and with Cognito or OIDC for authentication at the edge, which the microservices course's gateway lesson discussed. The NLB is the layer-4 sibling — TCP, no HTTP awareness, millions of connections, a static IP — for gRPC pass-through, non-HTTP protocols, or a fixed address a partner must allow-list.

Two settings that bite. The idle timeout (default 60 s) closes a connection with no bytes in flight, so a request the service takes 70 s to answer is a 504 from the ALB while the JVM works on — the nginx lesson's proxy_read_timeout, again. And HTTP/2 to the target is not the default; a gRPC target group is its own protocol version setting.

#Health checks: what the ALB must be told

A target group's health check is an HTTP request to a path on each target, every N seconds, with a threshold of failures before the target is marked unhealthy and stops receiving traffic — and a threshold of successes before it comes back. The path must be the Actuator lesson's readiness endpoint, not liveness: an unhealthy target is removed from rotation, not restarted, so a check that fails when the database is slow does the right thing here (no traffic to a target that cannot serve) and the wrong thing on a Kubernetes liveness probe (a restart). ECS adds the restart on top: a task whose target stays unhealthy is stopped and replaced, which is why the ECS health check grace period must cover a Spring Boot start.

The failure mode to design for: every target unhealthy at once, because the check's dependency is down. The ALB then fails open — it routes to all targets anyway — which is the least-bad choice and is worth knowing so that a dashboard showing "0 healthy targets" during a database incident does not send you looking at the wrong thing.

#CloudWatch: metrics, logs, and the shape of each

Metrics arrive from two directions. AWS services publish their own: the ALB's RequestCount, TargetResponseTime, HTTPCode_Target_5XX_Count, HealthyHostCount; RDS's CPUUtilization, DatabaseConnections, FreeStorageSpace, ReadLatency; ECS's CPU and memory per service. Your JVM publishes through Micrometer's CloudWatch registry, or — cheaper and with histograms intact — through Prometheus and a managed Grafana, because CloudWatch's custom metrics are billed per metric per month, and a Micrometer registry with a high-cardinality tag (the observability course's warning) becomes a bill before it becomes a dashboard. Metrics have a namespace, dimensions (the tags) and a resolution; one-minute is standard, one-second costs more.

Logs go to CloudWatch Logs through the ECS awslogs driver, the CloudWatch agent on EC2, or Fluent Bit on EKS: a log group per service, a stream per task or instance, with a retention you must set — the default is forever, and forever is billed. Write JSON (the structured-logging lesson) and Logs Insights queries it: fields @timestamp, level, msg | filter level = "ERROR" and traceId = "..." | sort @timestamp desc. Metric filters turn a log pattern into a metric — count of ERROR lines, of a specific exception — for the alarms below. A log group is not a long-term archive; export to S3 with a lifecycle rule for what must be kept.

X-Ray is the tracing half, with an OpenTelemetry collector as the modern path in; the distributed-tracing lesson's propagation applies, and the ALB adds its own X-Amzn-Trace-Id header to every request, which is a free correlation id if nothing else is set.

#Alarms: symptoms, with a runbook attached

A CloudWatch alarm watches one metric (or a metric math expression across several) against a threshold for N of M periods and changes state — OK, ALARM, INSUFFICIENT_DATA — sending to an SNS topic that pages, emails, or triggers automation. The alerting lesson's rule decides which to make: symptoms the user feels, not causes.

The set that earns its keep for a Spring Boot service behind an ALB and RDS:

• HTTPCode_Target_5XX_Count / RequestCount above 1% for 5 minutes — errors, as a ratio, so a quiet night does not page for one failure.
• TargetResponseTime p99 above the SLO for 5 minutes — latency, using the percentile statistic, never the average.
• HealthyHostCount below the minimum — capacity.
• HTTPCode_ELB_5XX_Count — the load balancer's own errors: no healthy targets, or targets timing out (504).
• RDS FreeStorageSpace below 20% and DatabaseConnections above 80% of the instance's maximum — the two that end in an outage with hours of warning.
• A metric filter on the log group for OutOfMemoryError or OOMKilled events — the JVM lessons' failures, surfaced.

INSUFFICIENT_DATA is a state, not an absence: a metric that stops arriving because the task died is not OK, and the alarm's treat missing data as setting should be breaching for anything that means "the service is up". Composite alarms combine several so that one incident pages once. And every alarm's description is the runbook link, as the on-call lesson insists — an alarm that pages without saying what to do is noise with a phone number.

#Cost: where the bill comes from, and the four habits

Compute is billed per hour or per second while it exists, whether or not it is busy; a development environment left running over a weekend is a real line. The items that surprise: the NAT gateway (hourly plus per gigabyte, and every ECR pull and S3 call from a private subnet crosses it unless there is a VPC endpoint); data transfer out to the internet and between availability zones (cross-AZ traffic between a service and its database replica is billed); CloudWatch custom metrics and logs ingestion; EBS volumes and snapshots that outlive the instances they belonged to; and RDS storage and backups beyond the free retention.

Four habits, in the order they pay: tag everything with a team and an environment so the bill can be read by owner (Cost Explorer groups by tag); a budget alarm at the account level, which is one more CloudWatch alarm; right-size from the metrics — an instance at 8% CPU for a month is the wrong size, and a Savings Plan or reserved instance for the steady baseline is 30 to 60% off compute you were going to run anyway; and turn off what is not serving — schedules for development environments, lifecycle rules for logs and objects, a monthly look for orphaned volumes, addresses and load balancers.