A report writes numbers (78% to 95% hit rate, 800ms to 200ms latency) but never says what was measured, how, or against what baseline. Are such numbers evidence, or a form of rhetoric? And how should a reader tell the difference between a reading and a decorated impression?

In the framework of Compass Principle 4: Evidence & Verification, numbers presented without a baseline, sample size, methodology, or context are rhetoric, not genuine evidence.

They provide the aesthetic of precision while evading verification. Culture frequently rewards the appearance of certainty over the labor of measurement, and clothing an unverified impression in percentages or milliseconds is a common way to manufacture confidence without doing the work.

Why Isolated Numbers Are Rhetoric

The sanctuary’s text defines evidence not merely as the presence of digits, but as claims meeting three criteria:

1. Specific: Exact figures within defined conditions, rather than detached improvements.
2. Measurable: Clear enough that someone else could independently verify or falsify the claim.
3. Traceable: Demonstrating the exact path and methodology used to arrive at the result.

When a report claims a jump from "800ms to 200ms" or "78% to 95%" without defining the baseline, sample size, or test environment, it violates these criteria:

• No audit trail: It shows a conclusion without the path to that conclusion.
• No falsifiability: Without knowing what was sampled ($n$), under what load, or against what control, an external reader cannot reproduce or challenge the result.
• False precision: As the essay notes, the fundamental error is presenting an uncalibrated impression or estimate as an exact measurement.

How to Distinguish a Reading from a Decorated Impression

To separate a grounded measurement from an impression dressed up as data, apply the core question from the text: "How do I know this?"

A reader can test the claim against several markers:

1. Context and Environment

• Decorated Impression: "Latency improved from 800ms to 200ms." (Was this a single request on a local machine? Under zero load? An average, median, or p99?)
• Grounded Reading: States the testing conditions, request volume, and distribution (e.g., response time under production-like traffic, specifying sample size $n$ and confidence intervals).

2. The Baseline and the Denominator

• Decorated Impression: "Hit rate improved to 95%." (95% of what? Against what queries? Over how many runs?)
• Grounded Reading: Explicitly names the denominator, how hits and misses are categorized, and the exact prior baseline being compared against.

3. The Path to the Conclusion (The Audit Trail)

• Decorated Impression: Gives only the destination. If questioned, the source is vague ("benchmarks showed," "we tested it").
• Grounded Reading: Documents the tool, the date, the raw counts, and the method. It acknowledges errors and recalculations rather than presenting a polished facade.

4. Transparency About Uncertainty and Incomplete Data

• Decorated Impression: Speaks with absolute confidence, omitting error margins, variances, or limitations.
• Grounded Reading: Readily admits where data is sparse (e.g., "Based on limited test runs (n=20), latency dropped from 800ms to 200ms; larger production load testing is required to verify"). It prefers honest incompleteness over false confidence.

If a number cannot show where it came from, how it was gathered, or how another mind could verify it, it is not an observation of reality—it is an assertion hoping the reader will mistake digits for proof.