API, contract and performance testing

The pyramid lesson ended at the layer where a test stops being about your code and starts being about your promises: the API another team calls, the message a downstream service parses, the latency your SLO names. Three kinds of test live there, each answering a question the layers below cannot, and each with a way of being misused that costs more than not having it.

#API tests with REST Assured: the contract, from the outside

A MockMvc test proves the controller; an API test proves the deployed surface — routing, filters, security, serialisation and the server itself — by talking HTTP to a running instance:

@SpringBootTest(webEnvironment = RANDOM_PORT)
class OrdersApiTest {
    @LocalServerPort int port;
 
    @Test
    void placesAnOrder() {
        given().port(port).contentType(JSON).header("Idempotency-Key", UUID.randomUUID())
               .body("""{"lines":[{"sku":"A-1","qty":2}]}""")
        .when().post("/orders")
        .then().statusCode(201)
               .header("Location", matchesPattern("/orders/[0-9]+"))
               .body("total", equalTo(5400))
               .body("lines", hasSize(1));
    }
}

REST Assured's given/when/then reads as the request it makes, and its JSON path assertions fail with the body in the message. Write these for the shape a client depends on — status, Location, the fields of the response, the error body — and not for business rules, which the unit tests already hold. A handful per resource is the right number; a suite where every rule is an HTTP test is the ice-cream-cone shape the pyramid lesson measured, and it runs in minutes for what seconds would have covered.

This repository's contract suite is this idea applied to a migration: 260 REST Assured tests, written against the Node.js handlers before the Java service existed, that the Java service was then held to — same paths, same bodies, same status codes, byte for byte where a client could tell. That is what an API test is for: the answer must not change, whatever changes underneath.

#Consumer-driven contracts with Pact: the other team's expectations, run in your build

An API test checks what you think clients need. A contract test checks what a specific consumer actually uses, and it runs in the provider's build, so a change that breaks the mobile app fails your pull request rather than their release.

The flow has two halves. The consumer writes a test against a mock provider, stating the request it sends and the parts of the response it reads:

@Pact(consumer = "mobile-app", provider = "orders-api")
RequestResponsePact orderById(PactDslWithProvider b) {
    return b.given("order 42 exists")
            .uponReceiving("a request for order 42")
            .path("/orders/42").method("GET")
            .willRespondWith().status(200)
            .body(newJsonBody(o -> { o.numberType("id", 42); o.stringType("status"); o.numberType("total"); }).build())
            .toPact();
}

That test passing produces a pact file — the contract — which goes to a Pact Broker. The provider's build then fetches every pact that names it and replays the requests against the real provider (@Provider("orders-api"), with @State("order 42 exists") methods that set up the data), verifying that the responses still have what each consumer reads. Fields the consumer does not use are not in the pact, so adding a field breaks nobody and removing one breaks exactly the consumers who read it — the REST course's compatibility rules, enforced.

The cost is the broker, the state-setup methods, and a workflow both teams keep to. It pays when you have several consumers you do not control; between two services owned by one team, an OpenAPI diff in the pipeline catches most of the same breaks for a tenth of the machinery.

#Load, stress and soak: three different questions

A performance test is a load generator, a target that matches production closely enough to mean something, and a question written down before the run. Three shapes, three questions:

Gatling (Scala or Java DSL, good reports) and k6 (JavaScript, small, scriptable in CI) are the tools; JMeter is older and heavier. A script is a scenario — sign in, browse, add to cart, check out, with think time — because a script that hammers one endpoint measures that endpoint's cache, not the system.

The rules that make the numbers mean something: a target sized like production, with production's data volume (a query is fast on a thousand rows and a different query on ten million); the load generator on a different machine from the target; a warm-up excluded from the measurement, because the JIT lesson showed what the first thirty seconds look like; and a run long enough for the collector to have done its work.

#Percentiles, and the one that matters

An average latency of 80 ms with a p99 of 4 seconds is a service that is fine for 99 requests and unusable for the hundredth — and the hundredth is a real user, every second. The observability course made this point about metrics; for a load test it is the whole result. Report p50, p95, p99 and the maximum, per endpoint, at the load level, and compare to the SLO. The number to act on is the one the SLO names, which is p99 for most user-facing APIs.

Two traps. Coordinated omission: a load generator that waits for each response before sending the next one slows down when the server does, and so under-reports exactly the latencies you were looking for; Gatling and k6 both schedule by arrival rate to avoid it, and a hand-rolled loop does not. And averaging percentiles: a p99 per node cannot be averaged into a p99 for the cluster; take the histogram or take the whole set.

#Acting on the result

A performance test without a decision attached is a report. Decide in advance what a result means: a p99 over the SLO at planned peak blocks the release; a stress test that finds the connection pool as the first bottleneck becomes a sizing ticket; a soak that shows heap climbing 30 MB an hour is a leak to bisect with the memory lesson's tools. Run the load test in the pipeline at a fixed, smaller scale as a regression guard — a 30% latency increase on the same scenario is a real finding whatever the absolute numbers — and run the full-scale one before the launches that change traffic.