Palimpsest

crs48 with Claude · History

On the fourteenth of April, 1229, a scribe named Ioannes Myronas finished a prayer book. He was working in Jerusalem, and he did what scribes of his time routinely did when good writing material was scarce: he made his book out of older books. Among the volumes he unbound was a manuscript copied in Constantinople around the year 950 — a collection of treatises by Archimedes. Myronas took its pages apart, scraped and washed the mathematics off them, cut them down, turned them sideways, and wrote his prayers across the faint ghost of the old ink. Among the texts he overwrote were The Method of Mechanical Theorems, which survives nowhere else, and the only copy of On Floating Bodies in the Greek in which Archimedes wrote it.

The prayer book worked. It served a monastic community for centuries, which is to say the erasure was, on its own terms, a success. Then, in 1998, the battered volume sold at auction for two million dollars, and a team of imaging scientists spent the better part of a decade teasing the underwriting back out of it — first with ultraviolet and infrared light, and finally, for four pages someone had covered with forged religious paintings, with X-ray fluorescence at the Stanford synchrotron. It took a particle accelerator to partially undo what one man with a knife did in an afternoon. And the man was not a vandal. He was an economist. Parchment was dear; the mathematics was not deemed worth its substrate. He made a rational trade at the prices of 1229.

Software makes the same trade billions of times a second, and calls it a design principle. Every UPDATE in every mutable database is the scribe's knife: walk to the place, scrape what is there, write the new text over it. We built our whole industry on update-in-place for the same reason Myronas scraped Archimedes — the substrate was expensive, and history did not seem worth what it cost to keep. This essay is about that trade, at today's prices. We have argued elsewhere that keeping history is right. This essay makes a narrower, more falsifiable claim: keeping history — signed, ordered, attributable history — is now so cheap that erasing it saves almost nothing. And for once we can show the bill.

The scribe was an economist

It is worth taking the economics of the medieval scriptorium seriously, because they rhyme with ours. Parchment was animal skin, prepared by hand; a large book cost a flock. Reusing it was not desecration but housekeeping — palimpsesting was ordinary practice across centuries of book production, and nobody wept over the underwriting, because the underwriting had been judged, reasonably or not, to be the less valuable layer. The knife was a tool of thrift.

Computing had its parchment era, and it lasted into living memory. In 1995 a gigabyte of consumer disk cost roughly eight hundred and fifty dollars — history, at that price, was a luxury good. And storage was the cheaper half of the problem. If you wanted history you could trust — each entry signed, so you know who wrote what — the arithmetic got ugly fast. In 2003, a commodity desktop processor could produce about thirty-two RSA-2048 signatures per second, flat out, doing nothing else. Signing every edit a busy user made would have consumed a visible fraction of the machine; signing every keystroke was simply out of the question. The database designers of that era were not timid. They were doing what Myronas did: pricing the substrate honestly and concluding that the past could not pay its rent.

So the mutable row became the default shape of records — not because anyone argued that destroying the previous state of your data was good, but because the alternative billed by the byte and by the cycle, and the bill was unpayable. The trouble with defaults born of scarcity is that nobody re-litigates them when the scarcity ends. The prices collapsed. The knife stayed.

The bill, itemised

xNet refuses the knife. There is no update-in-place anywhere in the protocol: every edit becomes a small immutable record — a change — that is content-hashed, signed by its author's key, chained to what came before, and appended to a log that only ever grows. Current state is derived from the log, never stored instead of it. This is exactly the design the parchment era could not afford. So what does it cost now? We measured, on the codebase as it ships, and we would rather show the receipts than wave at them.

// packages/sync/src/change.ts — the shape of one remembered fact (abridged).
interface Change<T> {
  payload: T                    // the fact itself
  hash: ContentId               // what was said — BLAKE3, content-addressed
  parentHash: ContentId | null  // what came before — history is a chain
  authorDID: DID                // who said it
  signature: Uint8Array         // provably — Ed25519, 64 bytes
  lamport: number               // in what order
}

The time cost first. On a recent laptop, the full pipeline that turns an edit into a fact — canonicalise the record, hash it, sign the hash — takes about 357 microseconds per change, running the deliberately portable pure-JavaScript cryptography we ship today. End to end, with the database write and the index updates and the subscribers notified, a single edit lands in a handful of milliseconds. Let us be honest about the multiplier, because overhead analyses that hide their worst number are advertising: against a bare in-memory write, or a plain database row update with none of the guarantees, the full pipeline is on the order of a hundred times more expensive per write. That is the real ratio, and it sounds damning — right up until you price it against a human being. A furiously productive person generating five thousand edits in a day spends, at 357 microseconds each, under two seconds of one core's time. Not two seconds per edit. Two seconds, total, for the day, to make every one of those edits provable. The overhead is two to three orders of magnitude below the threshold where anyone could feel it, and reads pay nothing at all — the current state is kept materialised in an ordinary indexed database, so asking "what does my workspace look like?" never touches the log.

Now the space cost. Provenance travels as an envelope around each change, and the envelope is a fixed tax:

Roughly 554 bytes of envelope per change, then, around whatever the edit itself weighs. Multiply it out: our furious five-thousand-edits-a-day user accumulates about 1.3 gigabytes of signed history in a year — around a minute of 4K video — which at today's storage prices costs six cents. Per year. For everything they ever did, attributable and ordered and cryptographically beyond dispute. The parchment-era bill that made update-in-place a necessity has become a rounding error on a rounding error; the knife now saves the price of a sweet from a corner shop, annually, and costs you your entire past.

Four curves crossed

Why now? Because four exponential curves, moving mostly independently for thirty years, all crossed the line labelled "cheaper than caring about". None of them alone would have been enough.

Signatures collapsed by three orders of magnitude. Half of that was silicon getting faster; the other half was mathematics getting better. The Ed25519 signature scheme, published in 2011, was designed from the start for speed on ordinary processors, and the difference against the RSA of the parchment era is not an increment — it is the difference between thirty-odd signatures a second and twenty-six thousand. A signature stopped being a ceremony and became a reflex, cheap enough to spend on every edit without thinking.

Storage collapsed by five. From hundreds of dollars per gigabyte to pennies — with one clause of honesty attached: that is the curve of the decades, not of the last two years, in which memory prices have actually risen on the back of the AI build-out. The argument survives the wobble comfortably; a year of signed history costs cents even at unkind prices. But an essay that quotes price curves owes you the direction of the latest segment.

Encodings collapsed the waste. Keeping every operation used to imply grotesque amplification — early collaborative-editing data structures could spend a hundred bytes of bookkeeping per character of text. A decade of careful engineering, most visibly Martin Kleppmann's columnar encoding work in the Automerge project, compressed that overhead from roughly a hundred-to-one down to a few tens of percent. Full history went from "pathological bloat" to "about a third more than the text itself" — not by hardware, but by cleverness.

And the platform finally showed up. A signed log on the server was already prior art — git has been content-addressed and immutable since 2005; certificate transparency runs append-only signed logs for the entire web. The missing piece was the device in your hand. Only in the last few years did the browser acquire what this design needs to run where the user is: real cryptography in the standard library, a real filesystem, a real database compiled to WebAssembly, and the threading to keep it all off the interactive path. The oldest load-bearing part of that platform stack is barely five years old. This design was not waiting for permission. It was waiting for its substrate.

Where the cost still lives

If the essay stopped here it would be a brochure, and this series does not do brochures. The overhead is imperceptible per action — but it has not vanished. It has moved. It now lives at the boundaries where you touch the whole history at once, and we know precisely where those are because each one has already hurt us in production.

Our worst incident of this kind: a workspace whose change log had grown to 318,000 rows took fifteen seconds to become usable on a cold start, because the boot path was replaying history it should have skipped. A mutable database cannot have that bug — there is nothing to replay — and it would be dishonest not to say so. The cure was not to abandon the log but to respect its asymptotics: current state is served from the materialised view, checkpoints bound how much past a cold start ever touches, and superseded entries are pruned. The log is an archive you consult, not a corridor you must walk every morning.

Bulk verification is the same story. Checking one signature is instant; checking a third of a million of them during a large import is minutes of work with the portable cryptography we currently ship — which is why our import path verifies bundles wholesale rather than change-by-change, and why the next engineering increment on our list swaps the bulk paths onto the platform's native cryptography, which we have measured at roughly sixteen times faster. And there is a boundary of kind, not just of scale: the envelope is a fixed tax, so it only makes sense around facts big enough to carry it. Wrap half a kilobyte of provenance around a forty-byte pencil coordinate at a hundred events a second and you will burn hundreds of megabytes an hour recording jitter no one will ever ask about — we know because we tried. The rule that fell out is now protocol discipline: the log is for facts, things a person did that history should answer for; high-frequency ephemera — cursors, presence, mid-gesture strokes — travel a separate, unrecorded lane, and only their settled results become changes. Today that boundary is a convention the codebase polices by rate limits and review rather than a physical law, and we are not yet sure that is enough.

Keep the underwriting

Here is the closing irony, and it is not hypothetical. Our industry already runs multispectral imaging on its own palimpsests — we just give it other names. Change-data-capture pipelines that reverse-engineer a history the database was busy destroying. Audit tables bolted on after the compliance letter arrives. Backup archaeology after an incident, diffing snapshots to reconstruct who changed what, when, because the row itself will not say. Forensics teams billing by the hour to recover, partially and expensively, information the system was handed for free and chose to scrape. Every one of these is a synchrotron pointed at a prayer book. The recovered text is always incomplete, and the imaging always costs more than the ink ever did.

The cheapest recovery technology ever devised is not scraping. That is the whole argument. Not that history is sacred — the scribe can be forgiven; at his prices, we would have reached for the knife too. The argument is arithmetic: the trade that made erasure rational has inverted, thoroughly and permanently, and the systems we build should notice. About half a kilobyte and a third of a millisecond per edit now buys what no mutable row can offer at any price: who, what, when, in what order — provable by anyone, deniable by no one, forever. At those prices, overwriting your users' history is no longer thrift. It is just a habit, wearing thrift's old clothes.

If you want to hold a workspace built on the other habit — the app is free, offline, and private; every edit you make in it is a signed fact you own. Read the commitments, or build on the protocol — the wire format is open, and the log is legible on purpose. Somewhere under the prayers of every mutable database is a Method nobody meant to lose. Keep the underwriting.

Sources

• The manuscript: the Archimedes Palimpsest Project — the scribe Ioannes Myronas and the colophon date of 14 April 1229, the 1998 Christie's sale, the multispectral imaging programme, and the X-ray fluorescence work at the Stanford synchrotron on the pages hidden beneath forged paintings. See also the Walters Art Museum's exhibition record, Lost and Found: The Secrets of Archimedes.
• The numbers: Bernstein, Duif, Lange, Schwabe & Yang, High-speed high-security signatures (the Ed25519 paper); archived openssl speed runs, 2003–2006 (RSA-2048 on the era's commodity processors); Backblaze, Hard Drive Cost Per Gigabyte; Matt Komorowski, A History of Storage Cost; and Martin Kleppmann's columnar-encoding benchmarks for the collapse in CRDT overhead.
• The receipts, in the open codebase: the change primitive and its signing path (packages/sync/src/change.ts), the cryptography (packages/crypto/src/signing.ts), the append-only changes table (packages/sqlite/src/schema.ts), and the materialised read path (packages/data/src/store/store.ts) — see the repository and the protocol docs. The measurements in this essay were made on the shipped code and are written up, with methodology and caveats, in exploration 0350.
• The paper trail — the numbered explorations behind this essay's incidents and rules: 0323 (the high-frequency boundary and the envelope tax), 0344 (bulk verification and the import gate), and 0329 (checkpoints — bounding what a cold start touches).
• Companion essays in this series: Tree Rings (why accretion is the right model — this essay is its price tag), The Loom You Can Read (the change log, line by line), and Weights You Can Hold (the economics of holding your own data).

The benchmark figures in this essay were measured on the xNet codebase as shipped, on a recent laptop, and are documented with methodology and caveats in exploration 0350; they are representative, not universal. Historical facts about the Archimedes Palimpsest follow the Archimedes Palimpsest Project and the sources linked above. This essay contains no direct quotations. All artwork here is original, and this page loads nothing third-party.