Tree Rings

crs48 with Claude · History

Cut a tree and you can read its whole life. Every year the trunk lays a new ring on the outside — wide in the wet years, tight in the droughts, scarred where the fire came through — and the years already laid down do not move. A tree never goes back to fix 1987. There is a science built on exactly this property: dendrochronologists have cross-dated overlapping ring patterns from living trees back through beams in medieval barns to sub-fossil oaks in peat bogs, and assembled an unbroken year-by-year record reaching thousands of years into the past. The record is trustworthy for one reason. The medium only ever adds. Growth on the outside; history, legible, within.

Software, mostly, is not built like this. Software is built like a whiteboard: to record something new, we erase something old. A database row is a place; an update walks to the place and destroys what it finds there. Rich Hickey — the creator of the Clojure programming language, and the closest thing our field has to a working philosopher — has spent two decades arguing that this is not a detail. It is the load-bearing mistake. He gave it a deliberately deflating name, place-oriented programming, and pointed out that its only real justification was that 1970s memory and disk were desperately small. The constraint died; the habit survived. New information does not need to replace old information. That is just what tiny computers forced us to do, and we kept doing it after the computers stopped being tiny.

This essay is about Hickey's argument and one attempt — ours — to take it literally in a protocol. The occasion is a talk he gave in 2017, at the conference marking Clojure's tenth birthday, called Effective Programs. It is the closest thing the corpus has to a summing-up, and near its centre there is a slide we have never stopped thinking about: a ranked list of everything that goes wrong in software, ordered by what it actually costs.

The pyramid of what actually hurts

The talk opens with a question most language designers skip: what are programs for? Hickey's answer is that programming is about making computers effective in the world — and that most of the leverage comes not from computation but from information: from accumulating facts about what happened and extracting predictive power from them. The programs that do this — the ones most of us are paid to build and run — he calls situated. A situated program runs for years, not milliseconds. It consumes and produces information whose meaning it does not control. It talks to other systems that will not upgrade on its schedule, absorbs requirements of genuinely dumbfounding arbitrariness (his example, from a radio-scheduling system he built early in his career: the station plays two songs by the same artist back-to-back on Tuesdays), and is expected to keep working while the world changes around it. A compiler gets to define its whole universe. A situated program lives in ours.

Then comes the slide. Hickey lists the problems of programming and prices them, most expensive at the top. At the summit sit domain complexity and misconceptions — the problem being genuinely hard, and you misunderstanding it. These dominate everything, and no language feature reaches them; by his estimate there is roughly an order-of-magnitude drop in cost before the first problem a language could even address. At the bottom sit the trivia: typos, silly inconsistencies, passing the wrong type — real, but cheap, caught earliest, closest to the keyboard. And in between lies the band where design actually pays: mutable places. Tangled couplings. Information forced through rigid class shapes. Names that only mean something inside one program's walls. Concurrency. Distribution. His charge, delivered mildly and landing heavily, is that our industry's most celebrated tooling concentrates on the cheap bottom of the pyramid — while the middle, where systems actually die, goes comparatively unserved. Clojure was his attempt to serve the middle.

Clojure serves that middle band at the level of a language. xNet is a bet that the same band can be served at the level of a system — a protocol for keeping, merging, and sharing a workspace full of other people's information, which is about as situated as programs get: always on, many devices, many authors, nobody in charge of the whole world. So let us descend the middle of the pyramid, problem by problem, and show receipts.

Facts do not take editing

Start with place-oriented programming, because everything else in xNet follows from refusing it. Hickey's alternative to the mutable place is the value — immutable, and meaning what it says without needing anyone's permission or interface documentation to read it. His favourite examples are disarming: your source control system never edits a commit; your logs are never rewritten; your accountant does not erase the ledger when the balance changes, and it is mildly alarming how many industries consider that basic hygiene while software considers it exotic. A fact is something that happened. You cannot update the past; you can only learn more of it.

In xNet, the primitive of all writing is built exactly on this refusal. Every edit — a keystroke batch, a task ticked, a cell changed — becomes a change: a small record naming the node it touches and the properties it sets. Before it goes anywhere, it is hashed and signed:

// packages/sync/src/change.ts — an edit becomes a fact: hash, then sign.
export function signChange<T>(unsigned: UnsignedChange<T>, signingKey: Uint8Array): Change<T> {
  const hash = computeChangeHash(unsigned)  // BLAKE3 over canonical JSON
  const signature = sign(new TextEncoder().encode(hash), signingKey)
  return { ...unsigned, hash, signature }
}

Note what this object cannot do. It cannot be altered — the hash would break. It cannot be disowned — the signature names its author. And it cannot be overwritten, because nothing in the system overwrites: the log of these changes is append-only, and the current state of your workspace is not a place being serviced by updates but a view derived from the log, the way the shape of a trunk is derived from its rings. We have walked through this log line-by-line before; what we want to say here is simply where the idea sits. It is the top of Hickey's middle band, implemented as a wire format.

Just use maps

Next: information. The strangest-sounding claim in Effective Programs, until it clicks, is that mainstream languages are bad at information specifically. The class — the proud unit of object-oriented design — makes you declare, in advance, in one program's vocabulary, what a person or an invoice is; the names get compiled away, and two systems with slightly different declarations cannot use each other's data without a diplomatic incident. Hickey's word for this is parochialism: types and names that only work inside their own parish. His prescription is older than it sounds — represent information as plain associative data, maps of self-describing keys, the way RDF represents everything as subject-predicate-object so that facts from strangers can merge at all. Clojure programs are famously maps all the way down. So is xNet:

// packages/data/src/schema/node.ts — the universal container type.
export interface Node {
  id: string
  schemaId: SchemaIRI   // e.g. xnet://xnet.fyi/Task@2.0.0
  createdAt: number
  createdBy: DID
  /** All other fields are schema-defined */
  [key: string]: unknown
}

Four fields are universal; everything else is keys. A schema in xNet is a lens over this open map — it validates, describes, and gives your editor something to autocomplete — but it does not seal the map shut. Anyone can lay extra keys on any node under an extension prefix that names their authority, and the protocol will sync and merge them like any other property; a later owner can promote a well-loved extension into the schema proper. And the names are anti-parochial by construction: a schema is not Task, a word that means something different in every codebase on earth, but xnet://xnet.fyi/Task@2.0.0 — a namespaced, versioned identifier anyone can mint under their own domain or DID, and no one can collide with. Authors, likewise, are not row IDs in somebody's users table; they are DIDs, names that keep working when the data walks out of the building. This is what it looks like to let information leave the parish: the map is open, and every key knows where it came from.

One braid at a time

Six years before Effective Programs, Hickey gave the talk most working programmers know him for, Simple Made Easy — the one that rescued the word simple from meaning "whatever I already know". Simple, by etymology, is one fold: a thing is simple when each concern stands alone, and complex when concerns are braided together — for which he exhumed the perfect archaic verb, complect. Easy, by contrast, just means near at hand — and the talk's lasting warning is that these are different axes. Tools that make things easy to write can braid horrors into what you have written. State complects value with time. Objects complect state with identity. And the cost lands later, in the artefact you run and maintain, not in the construct you enjoyed typing. His verdict on our profession's due diligence still stings: "programmers know the benefits of everything and the tradeoffs of nothing."

We can offer a confession-shaped receipt for this one. The heart of any multi-device protocol is its merge rule — ours is per-property last-writer-wins over the change log. For a while, that rule lived in three places: once in the store that applies changes, once in the SQLite layer's conflict guards, once in the hub's ordering. Three implementations of one idea, each drifting on its own schedule, on the invariant the whole system's convergence depends on. That is complexity in Hickey's exact sense — not a hard algorithm, but one fold braided through three modules. The fix was not cleverness. It was un-braiding:

// packages/core/src/lww.ts — "The ONE Last-Write-Wins ordering for xNet".
// Previously re-implemented in three places; every implementation now
// derives from this module, and a conformance suite pins them equal.
export function compareLwwStamps(a: LwwStamp, b: LwwStamp): number {
  if (a.lamport !== b.lamport) return a.lamport - b.lamport
  if (a.wallTime !== b.wallTime) return a.wallTime - b.wallTime
  // … then a grinding-resistant tiebreak key, then the author DID
}

The rule itself stayed three lines. The work was making it one — a single fold, projected into JavaScript and SQL from one source, with a conformance suite standing guard against re-braiding. Simplicity, Hickey keeps insisting, is a choice, and mostly an unglamorous one; nothing about that diff would demo well. But it is the difference between a castle of bricks and a castle of knitting, and you find out which one you built on the day you need to change it.

The whole workspace is a value

The deepest idea in the corpus is the one Hickey built a database out of. If facts are immutable and accrete, then time stops being an enemy and becomes an index. He calls it the epochal time model: an identity — you, a document, a company — is not a mutable thing but a succession of values, each one complete, each one permanent, like — his image, and this essay's title — the rings of a tree. Datomic, his database, made this concrete in a way that still feels like a magic trick: you do not send queries to the database; you ask for the database, as a value, as of any moment, and query it locally at your leisure. The past is not a backup tape. It is an argument you can pass.

xNet's version of the trick is called a frontier: a map from node to change-hash that pins a moment in the log — by content, never by clock, so it either means exactly what it says or fails honestly. Materialising a workspace at a frontier is a pure fold: start empty, apply changes in order, out comes the state as of that moment. Everything the app calls history is this one primitive wearing three hats. A read-only frontier is the timeline scrubber. A named, pinned frontier is a checkpoint. A frontier you fork and keep writing to is a draft — and because your draft and the shared head are just two successions of the same values, comparing or merging them needs no new machinery at all. Undo, blame, diff, "what did this look like in March?" — none of these are features in xNet. They are queries against rings the tree was growing anyway.

Once you see the pattern you find it everywhere in the design, and the repetition is the point. A change is a value — hashed, signed, done. A moment is a value — a frontier of hashes. A query is a value — a serialisable expression any storage engine can compile. An error is a value — a tagged record you can match on, not a string you grep. Even a schema is a value, versioned in its own name, with lenses to carry data between versions. Hickey's pitch for values — that they can be shared, cached, compared, sent across a wire, and reproduced without anyone coordinating — is not an aesthetic in a sync protocol. It is the entire mechanism. Two devices converge because they are folding the same immutable facts through the same pure function; there is nothing else to agree on.

Three departures

If the essay ended there it would be fan mail, and Hickey's own method forbids that — the benefits of everything, remember, and the tradeoffs of nothing. So here is where xNet leaves the hammock, on purpose, three times.

We use static types. Everywhere. xNet is TypeScript from kernel to UI, and its author would not give up the tooling — the renames, the exhaustive switches, the autocomplete — for any philosophy. What we kept from Hickey's type-scepticism is the boundary discipline: his real charge is that elaborate types mostly polish the cheap bottom of the pyramid, and become actively parochial at system boundaries, where your beautiful Invoice<Paid> is a private language nobody else speaks. So in xNet, types never cross the wire. The protocol's contract is canonical JSON, open maps, and namespaced names; the TypeScript is scaffolding inside our walls, and a peer written in Rust or Clojure owes it nothing. Types as tooling, data as truth — we think of it as agreeing with the talk's conclusion while declining its temperament.

There is one black box in the tree. Prose is the one domain where per-property last-writer-wins is the wrong physics — two people typing in the same paragraph need character-level merging — so document bodies and canvas scenes use Yjs, a CRDT whose binary updates are decidedly not semantically transparent values. We drew a hard lane around it: the log for facts, the CRDT for prose, and the CRDT's blobs ride inside signed log changes like freight — addressed, attributed, and ordered by the same machinery, opaque only within. A purist would note the interior of that box has no rings. Correct. It is the tradeoff we can name.

We prune. Hickey's soaring line about the new economics of storage is that we can finally afford to remember everything — and servers can. Phones, whose owners also deserve their whole workspace, cannot always. So xNet ships history-horizon policies that can compact the deep past on constrained devices, which is a real departure from the accrete-forever ethos. What we refused to give up is honesty at the horizon: because frontiers address history by hash, a scrub past what a device retains cannot silently show you a wrong answer — it fails, loudly, and checkpoints pin what they reference so the moments someone chose to name survive compaction. A tree with a hollow core, which still never lies about a ring it has.

The hammock at the top

Which leaves the summit of the pyramid — misconceptions, the problems that cost the most and that no data model, ours included, can touch. Hickey's remedy for those is not in any repository. In the most beloved talk of the corpus he recommends, with a completely straight face, the hammock: state the problem in writing, feed your head with everything known about it, and then stop typing — because the waking mind is a good critic and a mediocre designer, and the background mind, which does the real synthesis, only takes an agenda from what you loaded before you stepped away. Solve problems, he keeps saying, not features. The cheapest bug is the one designed out before the first keystroke.

The closest thing xNet has to an engineering method is an attempt to practise this. Nothing sizeable lands in the codebase without a numbered exploration first — a written statement of the problem, the prior art, the options, the tradeoffs, a recommendation, and a checklist that later gets marked against reality. At the time of writing there are three hundred and forty-eight of them, and the one that specified this essay spent a while in the hammock deciding what this essay should refuse to claim. Some explorations conclude build nothing; those are the best value in the folder. This, we have come to believe, is the real substance of Hickey-style design — not parentheses, not any particular database, but the unfashionable conviction that understanding is a deliverable, and that the pyramid is climbed from the top.

A tree does not hurry, and it does not revise. It commits, once a year, in public, and keeps every commitment where anyone with a saw and some patience can audit the whole series. That is the standard the talks set for software that intends to be around in forty years and legible when it gets there: growth on the outside; history, legible, within. If you want to hold a workspace built on that bet: the app is free, offline, and private. Read the commitments, or build on the protocol — the wire format is open maps and signed facts, so you do not need our permission, which is rather the point. And watch the talk below, ideally twice. It is ten years of thinking, compressed to seventy minutes, about the only questions in software that compound.

Sources

• The talks: Effective Programs: 10 Years of Clojure (Clojure/conj 2017); Simple Made Easy (Strange Loop 2011); The Value of Values (2012); Deconstructing the Database (2012); Hammock Driven Development (Clojure/conj 2010); and Are We There Yet? (JVM Language Summit 2009). Community-maintained transcripts of all of these live in the matthiasn/talk-transcripts repository.
• The receipts, in the open codebase: the change primitive (packages/sync/src/change.ts), the open-map node (packages/data/src/schema/node.ts), the single merge rule (packages/core/src/lww.ts), and the history engine and frontiers (packages/history/) — see the repository and the protocol docs.
• The paper trail — the numbered explorations behind this essay's claims, in the open repository: 0021 (a Clojure port of the kernel), 0200 (the kernel is the signed log, not the CRDT), 0329 (frontiers, checkpoints, and drafts), and 0330 (the log-for-facts, CRDT-for-prose rule).
• Companion essays in this series: The Loom You Can Read (the change log, line by line), Clutch Power (the four frozen interfaces), and The Vault and the View (apps as views over data you keep).

This is an independent essay about Rich Hickey's public talks. He has no affiliation with xNet and has not reviewed or endorsed this essay or this project. Ideas are paraphrased from the talks linked above; the one direct quotation is brief, attributed, and used as commentary. All artwork here is original, and this page loads nothing third-party.