Replica sets, transactions and consistency

A single MongoDB node is a development setup. Production is a replica set: several copies of the data with one accepting writes, and everything about durability and consistency is a setting on the write or the read that says how many of those copies must be involved. The defaults are safer than they were; the settings still decide whether a confirmed write can vanish and whether a read can see it.

#Replica set basics

A replica set is one primary and, typically, two secondaries, plus an optional arbiter that votes but holds no data. Every write goes to the primary and is recorded in its oplog, an ordered log of operations; secondaries tail the oplog and apply it — the same shape as the SQL course's replication lesson, with the same lag. When the primary stops answering heartbeats (the default timeout is ten seconds), the remaining members hold an election and a secondary with a majority of votes becomes primary; the old primary, when it returns, steps down and rolls back any writes the new primary never received, writing them to a file for a human to look at.

Two consequences follow for a Java service. The driver takes the seed list and discovers the topology itself, so the connection string names the set (replicaSet=rs0) and the driver follows the primary through an election, retrying a write once (retryWrites=true, the default) so that a failover in the middle of an insertOne is not an error the application sees. And an odd number of voting members matters: two data nodes cannot elect anyone when one is gone, which is why the arbiter exists and why a three-member set is the minimum that survives a node.

#Write concern: how many copies before "done"

collection.withWriteConcern(WriteConcern.MAJORITY).insertOne(order);

The write concern says how many members must have the write before the driver returns. w: 1 is the primary only — fast, and a write the primary acknowledged can be rolled back if it fails before a secondary copied it. w: "majority" (the default since 5.0) waits for a majority of voting members to have it in their journal, which is the point at which an election cannot lose it; it costs a round trip to the secondaries per write. w: 0 does not wait for anything and is for metrics you can lose. j: true adds a journal flush on the members counted. Set it per operation for the writes that are money and per collection for the rest; and set wtimeout, because a majority write with a secondary down waits forever otherwise.

The SQL course's "commit" was one machine's fsync. Here "committed" has a number in it, and the number is what a durability review asks for.

#Read preference and read concern

Read preference is which member answers: primary (the default; every read sees the latest write), primaryPreferred, secondary, secondaryPreferred, nearest. Reading from secondaries scales reads and offloads the primary, at the cost the replication lesson named: lag, so a read may return a state from seconds ago, and a client that writes then reads its own write from a secondary may not find it. Route analytical and report queries to secondaries; keep anything a user just changed on the primary.

Read concern is how settled the data must be: local returns whatever the member has, including writes that may still be rolled back; majority returns only data acknowledged by a majority, so a read can never show a write that later disappears; linearizable on the primary additionally confirms the member still is the primary, for the read-then-decide case where a stale primary would be catastrophic (a lease check, a uniqueness check outside a unique index). majority reads and majority writes together give the guarantee most services want: what you read was durable, what you wrote will be seen.

Causal consistency ties the two across a session: with a ClientSession, reads after a write in the same session are guaranteed to see that write, even from a secondary, because the driver sends the write's timestamp along and the secondary waits until it has caught up. That is read-your-own-writes as a session property rather than a routing rule.

#Multi-document transactions, and their cost

A single-document write is atomic on its own — every field, every embedded array, one document, all or nothing — and the document-modelling lesson's advice to put what changes together in one document is what makes most applications need no transaction at all. When two documents must change together (a ledger and a balance in separate collections, a transfer between two accounts), MongoDB 4.0+ has multi-document transactions on a replica set:

try (ClientSession session = client.startSession()) {
    session.withTransaction(() -> {
        accounts.updateOne(session, eq("_id", from), inc("balancePaise", -amount));
        accounts.updateOne(session, eq("_id", to), inc("balancePaise", amount));
        ledger.insertOne(session, entry);
        return null;
    }, TransactionOptions.builder().writeConcern(WriteConcern.MAJORITY).build());
}

They are real ACID transactions, and they are not free: snapshot isolation across the documents, locks on the documents written (a concurrent write to the same document aborts with a transient error that withTransaction retries), a default 60-second lifetime, and a throughput cost that makes a transactional workload on MongoDB slower than the same workload on PostgreSQL. Spring Data's @Transactional with a MongoTransactionManager wraps the same session. Use them for the few writes that genuinely span documents, keep them short, and treat "we need transactions everywhere" as the signal that the model is relational and the store choice should be revisited — the data-store lesson's question, asked late.

#Sharded clusters, in one section

When a replica set's data or write load outgrows one primary, a sharded cluster splits a collection across several replica sets by a shard key, with mongos routers in front and config servers holding the map. Everything the replication-and-sharding lesson in the system-design course said applies: the shard key decides whether a query goes to one shard (targeted, fast) or all of them (scatter-gather, slow); a monotonically increasing key (ObjectId, a timestamp) sends every insert to one shard; a key with low cardinality cannot spread; and the key cannot be changed without rewriting the collection (5.0 allows resharding, expensively). Choose it from the access patterns — the field every frequent query filters on — hash it if the queries are equality-only and the writes must spread, and know that transactions across shards and $lookup across shards are possible and slower still. Most services never need it; a replica set with a well-modelled schema serves far more than teams expect.