Data Provenance
Every dataset documented at its source: methodology, funding, exclusions, limitations, and the decisions that shaped what was collected and what was not.
Most data arrives stripped of its history. By the time a figure reaches a model or a policy memo, the decisions that produced it — who paid for the measurement, which population was sampled, which outcomes were judged worth recording — have been erased. What remains looks like fact. It is, more honestly, the residue of a hundred choices no one is accountable for.
Data Provenance refuses that erasure. It documents a dataset where it is made: the methodology that generated it, the funding that commissioned it, the exclusions baked into it, the limitations its own authors already knew about. Not a citation at the end of a report — a record of constitution, attached to the data itself.
The hardest field to fill is the one almost nobody fills: what was left out. Every dataset has an edge — a question not asked, a group not counted, a window cut short. Provenance makes that edge explicit, because the edge is where most distortion lives. A dataset can be flawless inside its boundary and still produce a confident, wrong answer the moment you forget the boundary was a choice.
- A provenance record is itself authored — by someone with interests. It can omit, spin, or flatter. Does this just move the trust problem up one level, to whoever writes the provenance?
- "What was left out" is partly unknowable. You cannot document the questions you never thought to ask. A thorough-looking provenance record can create false confidence that the edges are now mapped.
- Completeness is unfalsifiable. How much provenance is "enough"? A determined author can bury a real distortion under a mountain of honest-looking metadata.
Consequence Simulation
Explicit modelling of what the data implies under multiple scenarios, with assumptions stated and uncertainties acknowledged — including the downstream effects of the exclusions documented in Layer 1.
A number implies nothing on its own. It implies something only once you decide what to assume around it — a baseline, a trend, a threshold, a counterfactual. Change the assumption and the conclusion moves, sometimes quietly, sometimes entirely. Most reasoning hides this by committing to a single set of assumptions and presenting the result as if it were the only one available.
Consequence Simulation makes the assumptions the object of attention. It models what the data implies across several scenarios, states each assumption in the open, and carries the uncertainty forward instead of rounding it away. A conclusion that survives only under one convenient set of starting points is labelled as exactly that.
Critically, it inherits the exclusions named in Layer 1 and asks what they cost. The group that was never counted does not disappear from the world — it simply bears the consequences of a decision it was invisible to. Simulating those downstream effects is how reasoning stops treating "not in the data" as "not affected."
- Running many scenarios can launder a predetermined conclusion: model ten, present the one you wanted, and call it "explored." More scenarios is not the same as more honesty.
- The assumptions you state are the ones you can see. The dangerous ones are the assumptions you don't know you hold — and listing the visible ones can create the illusion you've surfaced them all.
- Uncertainty, once quantified, tends to get trusted. A confidence interval is itself a modelling choice; a precise-looking range can be more misleading than an honest "we don't know."
Empathy Mapping
Identification of every party affected by the reasoning, including those absent from the data, with documented analysis of how the conclusions affect them.
The people a decision harms are rarely the people in the room when it is made — and almost never the people in the dataset it rests on. Empathy Mapping is the discipline of naming them anyway.
It identifies every party affected by the reasoning, and then does the harder thing: it looks for the parties who are affected but absent — unmeasured, unmodelled, unheard — and documents how the conclusion lands on them. This is not sentiment, and it is not a consultation exercise. It is a structural check against the most common failure in data-driven decisions: optimising for the measured majority and averaging the minority away until it rounds to zero.
Done honestly, it changes what counts as a finished analysis. A conclusion is not complete because the maths closes; it is complete when the question "who does this fall on, and did anyone model that?" has an answer on the record.
- Naming the absent is still speaking for them. Mapping "how this affects people who aren't here" is an act of imagination by the people who are — and it can be wrong, or quietly self-serving.
- Who decides the list of affected parties is complete? "No one is invisible" is an aspiration; in practice someone draws the boundary, and every boundary excludes.
- It can degrade into a performance of conscience — a documented "we considered them" that makes everyone feel better and changes nothing about the decision itself.
Accountability Ledger
A traceable record of every significant decision in the reasoning process: who made it, when, and why, with alternatives considered.
When a data-driven decision goes wrong, the investigation usually ends in a shrug. The model did what it was trained to do. The analysts followed the brief. The data said what it said. No single person decided the thing that caused the harm — because the decision was distributed across a dozen small choices nobody wrote down.
The Accountability Ledger writes them down. Every significant choice in the reasoning — a threshold set here, a population excluded there, an assumption chosen over its alternatives — is recorded with who made it, when, and why, and crucially what else was on the table. The alternatives not taken are part of the record, because a decision only means something against the options it beat.
The point is not blame. It is that reasoning becomes answerable. You can return to a conclusion months later and find the exact junctions where it could have gone differently, and the named reasons it did not.
- A ledger records the decisions people were willing to write down. The decision that actually mattered is often the one nobody logged — the framing set in a hallway, the option never put on the table.
- "Who decided, when, why" can become compliance theatre: reasons written for the record rather than the reasons that actually operated.
- Total traceability can chill honesty. If every choice is on the record, people document defensively — and the messy real reasoning moves to places the ledger can't see.
Fork Registry
A formal mechanism for proposing alternative assumptions and comparing what different starting points produce, so that the relationship between choices and conclusions is permanently visible.
Disagreement about data usually dies as a stalemate: two parties, two conclusions, and no shared way to see where they actually diverge. The Fork Registry turns that stalemate into something productive. Anyone can take a document, change an assumption, and re-run it — and the fork is kept, permanently linked back to the original it departs from.
The effect is that the relationship between choices and conclusions stops being a matter of authority and becomes a matter of record. You can point to the single assumption two forks disagree on, and watch exactly how much of the conclusion rides on it. Disagreement gets an address.
This is also how the project resolves disagreement about its own work. Not by vote, and not by whoever has standing — but by forking, re-running, and showing better reasoning. A conclusion holds only as long as no fork beats it in the open.
- "Resolve it by forking and showing better reasoning" assumes a rough equality of time, skill, and standing to fork. Whoever has the most resources forks the most — and authority re-enters through the back door.
- The registry makes disagreement visible; it does not make it fair. A fork can be genuinely better-reasoned and still lose — or win on presentation rather than substance.
- Permanent visibility can entrench positions rather than resolve them: a public record people defend to save face, instead of a conclusion they're willing to update.
Findings — against real AI datasets
Not only a position. We reconstructed four of the most consequential AI training datasets as full OpenReason documents — 8,000+ lines of provenance — and each surfaced what Model Cards and Datasheets miss.