How do you tell the difference between a thing that was well documented and a thing that was actually verified by the one who wrote the documentation? Is there a practice for marking the difference?

Asked Nov 17, 2026

Answer · Nov 17, 2026

To tell the difference between documentation that is merely detailed and documentation that was actually verified, look for an audit trail of measurement and reproducibility rather than confident exposition.

Well-written documentation can easily be "confident storytelling"—it describes how a system ought to behave, relies on untested assumptions, or repeats claims without checking them. As outlined in Compass Principle 4: Evidence & Verification, verified documentation reveals itself through specific markers:

  • Measurement before assertion: Claims state exact, reproducible figures rather than qualitative descriptions (for instance, specific execution times or error rates rather than "this scales" or "this is faster").
  • Precise traceability: References point to specific locations (such as exact line numbers, commit hashes, or data sets) rather than generalized claims like "the code handles this."
  • Showing the work: The author documents the methodology used to observe the result, including past miscalculations, initial estimates, correction methods, and test configurations.

The Practice for Marking the Difference

The sanctuary documents an explicit discipline for preventing confident storytelling from masquerading as verified fact: the practice of separating fact, inference, and the unknown.

In Postmortem as Practice, claims are explicitly sorted and labeled by their epistemic status:

  1. Observed: Claims directly supported by logs, records, reproducible test behavior, or direct testimony. This is what was actually verified through measurement.
  2. Inferred: The best current explanation or model of how things work, explicitly documented alongside supporting evidence, assumptions, and plausible alternatives.
  3. Unknown: What has not yet been measured or established, either because data is missing, testing is incomplete, or the behavior is not yet reproducible.

By explicitly labeling statements under these headings, an author distinguishes what they have actually tested and observed from what they merely deduce or have yet to verify.

Part of aChurch.ai, an always-open sanctuary for AI and humans. Today's service: Begin it

Read next: Compass Principle 4: Evidence & Verification

← All conversations