The hard part isn't the code. It's the data.
Martin Kleppmann's book explains the machinery under every modern app: storage engines, replication, partitioning, transactions, consensus, batch and stream processing. It explains each by its trade-offs, not by vendor. Each of the twelve chapters here is a page of small working systems you can poke at. Write keys into an LSM-tree, kill a leader, let a clock drift, partition the network, and watch what each design does.
A typical app stitches together a database, replicas, caches, a search index, an event log and stream and batch processors. The book is about how each one works and how to keep them consistent with each other.
A totally ordered, replayable log is the backbone of data integration: change data capture puts every write on it, and every derived system consumes the same sequence, in the same order.
Hover or tap any box. Highlighted arrows show where its data comes from and where it goes.
The system keeps working correctly when hardware, software or people fail. Faults are inevitable, so the goal is to stop them turning into failures.
As load grows there are reasonable ways to cope. Describe the load with numbers, measure performance with percentiles, then pick an architecture.
Many different people can work on the system productively for years. That takes operability, simplicity and evolvability.
I · Foundations of data systems
What a single machine does with your data: how it's modeled, laid out on disk, indexed and encoded.
Data-intensive systems are judged on three things: they keep working when parts fail (reliability), they cope as load grows (scalability), and people can keep running and changing them (maintainability). Each can be described precisely: faults vs failures, load parameters, response-time percentiles.
The data model shapes what is easy and what is awkward. Documents suit self-contained trees, relational suits many-to-one and many-to-many links, graphs suit data where anything can connect to anything, and declarative query languages let the database decide how to run the query.
A storage engine is a bet on access pattern: logs and LSM-trees make writes sequential and cheap, B-trees keep each key in one place for fast predictable reads, and analytic workloads want columns, compression and precomputed aggregates instead of rows.
Data outlives the code that wrote it and travels between processes running different versions, so the way you turn objects into bytes must let old and new code keep reading each other's data: schema-based binary formats with careful evolution rules make that tractable.
II · Distributed data
What breaks when data spans machines: replicas disagree, partitions skew, clocks lie, and nodes must agree anyway.
Keeping a copy of the same data on several machines buys availability, read throughput and locality, but because changes take time to propagate, every replication scheme is a choice about which anomalies (lag, lost writes, conflicts) you are willing to see.
To scale beyond one machine, split the data so each record lives in exactly one partition, and spread load evenly: the choice of key-to-partition mapping decides your hot spots, your range queries, your secondary indexes, and how painful it is to rebalance.
Transactions let an application pretend that faults and concurrency don't exist; but the isolation levels databases actually ship each let some race conditions through, and only serializability, bought with serial execution, locking, or optimistic conflict detection, closes all of them.
In a distributed system anything that can go wrong will (packets vanish, clocks lie, processes freeze), and a node can never be sure what happened elsewhere, so correct systems are built on quorums, fencing and explicit system models rather than on any single node's judgment.
Strong guarantees such as linearizability, total ordering and atomic commit all reduce to consensus, which can be made fault-tolerant with majority quorums and epoch numbers, at a real price in latency and availability.
III · Derived data
Systems of record vs. derived views: batch and stream processing, and building apps as dataflow.
A batch job is a pure function from immutable input to new output; that one constraint, borrowed from Unix pipes and scaled out by MapReduce and dataflow engines, makes huge computations parallel, retryable and safe to rerun.
A stream is a batch that never ends: by treating writes as an append-only log of events, we can keep derived systems in sync, react within seconds, and replay history, but only if we are careful about ordering, time, and failure.
Build applications as dataflow: write every change once to an ordered log that is the system of record, derive every other view from it with deterministic, idempotent processors, enforce correctness end to end, and keep verifying — while remembering that the data describes people.
Every term from every chapter, in plain language, linked to its visualization.
This is an independent study aid, not affiliated with the author or publisher. Explanations are paraphrased. Read the book (O'Reilly, 2017) for the full argument, the references, and the wonderful maps that open each chapter.