Measured, not asserted
A design reading well proves less than it used to, so a claim needs an artifact someone else can check: a benchmark committed with the hardware, the date and the competing tool versions; a run journal; a count over transcripts with the recipe to reproduce it. Weigh anything without one as experience, and say so in the sentence that makes the claim.
Measurement cuts both ways, and the useful half is the one that says no. A conformance check that fires on four deliberate designs out of four is noise; an advisory that fires on 0.8% of edits, all of them matcher artifacts, is a toll. The rows in A record of refusals that record a measurement killing an idea somebody wanted are the reason the rest of that list is worth anything.
A measurement also says where the tool falls short. The same journals that show what the cache saved show what it did not, and a page that only publishes the good half is marketing.
Enforced by
- Benchmarks publishes every run beside the environment that produced it, controls first.
- A conformance threshold is a relative bound with a committed ratchet
(
hack/policy/conformance.jsonl), so a rule's floor is a number in the tree. - A record of refusals names the measurement and its date beside each refusal.
Good
A refusal row that carries its measurement:
an advisory when an agent edits a file it never looked up | not built: 0.8% of 3,560
first edits were never anchored, and all 27 cases read by hand were matcher artifacts
Bad
The graph makes agents 3x faster.
No run, no control, no date, no recipe. Nobody can check it, and it outlives whatever it was ever true of.