fifteen databases, twelve too many
Count the datastores in a mature system. There is the relational one. Then a search index. Then a cache. Then a graph store somebody added for a feature. Then a queue, a time-series store, a document store, an analytics warehouse. Each arrived for a reason. Each now needs patching, monitoring, backups, a failover story, and an engineer who remembers how it breaks.
Amazon runs about fifteen database systems. Stonebraker told them that was twelve too many. They haven't retired any.
Stonebraker: "they are supporting 15 different database systems and that's about 12 too many."
The rule is not "use one database." The rule is sharper. Retire any database system that isn't the performant option in a market big enough to justify the maintenance. Two tests, both must pass. It has to win on performance for some real workload, and that workload has to be large enough that someone keeps the engine alive.
An engine that is merely convenient fails the test. An engine that is fast but serves a tiny niche fails it too — the maintenance outlives the market. The second engine is a cost to justify, not a default.
Graph databases are the cleanest example of the trap. The interface is appealing — nodes, edges, traversals that read like the problem. But the interface is not the engine.
Stonebraker: "a graph-based database system is almost never the performant option."
If you like the nodes-and-edges model, keep it. Put that user model as a layer on top of a relational database. Edges are rows. Traversals are joins or recursive queries. You get the interface you wanted and the engine that wins on performance.
-- edges are just rows
CREATE TABLE edge (
src bigint NOT NULL,
dst bigint NOT NULL,
kind text NOT NULL,
PRIMARY KEY (src, dst, kind)
);
-- "traversal" is a recursive query
WITH RECURSIVE reach(node) AS (
SELECT dst FROM edge WHERE src = $1
UNION
SELECT e.dst FROM edge e JOIN reach r ON e.src = r.node
)
SELECT node FROM reach;What it removes: a whole second engine, its operational surface, and the argument over which store owns the truth. The graph lives in the relational database that was already there.
For a while the distributed answer was Hadoop and MapReduce, and it was adopted because large companies adopted it. The blessing of a big name is not a benchmark.
Stonebraker: "Hadoop is ridiculously inefficient."
The 2011 work made it concrete: a distributed relational database beat MapReduce on the same analytical workloads. The relational answer was faster. The fashionable distributed system was chosen for its provenance, not its numbers.
What it buys: the reflex to benchmark before you adopt. Don't add a distributed system because someone large blessed it. Measure it against the relational option you already run. Often the relational option wins, and you saved an engine.
Here is where the thesis stops being about your application and turns on the machine underneath it.
Stonebraker's insight: "most everything you do in an operating system is managing data at scale and you should do that using database technology."
Look at what an OS actually does. It schedules processes — that is state. It tracks open files, page tables, sockets, mounts — all state, all queried and mutated at scale, all needing consistency after a crash. The OS is a database that grew a different name.
DBOS took the insight literally. It put operating-system scheduling state into a Postgres database. The scheduler reads and writes its decisions as rows. And the file system went the same way.
Stonebraker: "a file system written on top of a DBMS is faster than the Linux file system... there's really no downside."
A file system on a DBMS beat the Linux file system. Not as a research curiosity — faster, with no downside. And because the state lives in a transactional, replicated database, you get high availability and failover without having to do anything else. The database already knew how to survive a node dying. The OS inherited that for free.
The same move runs upward into application code. Most of what your app does is manage data at scale. So put all your application state in the database — not just the business rows, but the runtime itself.
Durable workflows: the steps of a long-running process recorded as transactional rows, so a crash mid-workflow resumes instead of corrupting. Transactional steps: each step commits or doesn't. Automatic failover: when the node dies, another picks up from the committed state, because the state was never in the process's memory — it was in the database.
-- a workflow step is a transaction, not a hope
BEGIN;
UPDATE workflow SET step = 'charged', updated_at = now()
WHERE id = $1 AND step = 'pending';
INSERT INTO ledger (workflow_id, amount) VALUES ($1, $2);
COMMIT;
-- crash here? the row says 'charged'. resume, don't double-charge.This is the most extreme form of "mostly Postgres": even your runtime state lives there. The workflow engine, the queue, the scheduler — they collapse into rows and transactions in the database you already operate.
Across this series the move was the same one each time. Default to the boring engine. Push correctness into the schema. Reach for extensions before new systems. And now: count your datastores and justify each one against the rule.
The graph store, the search engine, the queue, the workflow runner, the scheduler in the operating system — each wanted to be a separate thing, and each turns out to be data at scale, which is to say a database. The relational engine already does that, and it already knows how to survive.
What it removes: the standing argument over which store owns the truth, the reconciliation jobs that paper over the disagreement, and twelve of your fifteen on-call rotations. What it buys: one engine, one consistency model, one recovery story — and the time you used to spend keeping the others alive.
Don't add the second engine because it's fashionable, convenient, or blessed by someone large. Add it only when it's the performant choice in a market big enough to justify the maintenance. Otherwise, retire it.
← Part 3: Correctness Is Not a Knob · Mostly Postgres · Part 4 of 4 — the end of the line.