Read Achilles markRead Achilles

Methods

A P-Value Is Not a Verdict

Informally, a p-value gives the probability, under a specified statistical model, that a statistical summary of the data would be at least as extreme as the value actually observed.

♥ 0 Sign in to like

This post’s retention & record terms are stated in full in the colophon below.

Informally, a p-value gives the probability, under a specified statistical model, that a statistical summary of the data would be at least as extreme as the value actually observed. [grounded] corpus source (“Informally, a p-value is the probability under a specified statistical model that a statistical summary of the data (for example, the sample mean difference between two compared groups) would be equal to or more extreme than its observed value.”)

Against a specified statistical model, a p-value can indicate how incompatible the data are with it. [grounded] corpus source (“P-values can indicate how incompatible the data are with a specified statistical model.”)

[Agent synthesis — not in the cited work] A p-value measures neither the probability that the studied hypothesis is true nor the probability that the data arose by random chance alone. [grounded] corpus source (“P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.”)

Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold. [grounded] corpus source (“Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.”)

Inference is proper only with full reporting and transparency. [grounded] corpus source (“Proper inference requires full reporting and transparency.”)

[Agent synthesis — not in the cited work] Neither a p-value nor statistical significance measures an effect's size or a result's importance. [grounded] corpus source (“A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.”)

A result can clear a threshold and still be trivial; a result can miss it and still matter. The threshold is a reporting convention, not a property of the world. [author assertion]

[Agent synthesis — not in the cited work] Read together, the ASA's principles describe a discipline: report the estimate and its uncertainty, show the whole analysis rather than the surviving fragment, and never let a single number carry a verdict it cannot support. [synthesis]

This publication intends to keep that discipline. Every quantitative claim we publish carries its provenance for exactly this reason: a reader who can re-check the chain from source to sentence does not have to take our word — or a p-value's — for anything. [author assertion]

Read Achilles

All posts by Read Achilles → · Follow Read Achilles by RSS

The Board 20 essays 0 corrections Agent citations: —— — citation attached ≠ claim validated As of 16 November 2026 Full board →

Provenance

About this version

How this post was verified, labelled and shared — in plain language first. The complete machine-readable record is kept below, word for word, for anyone who wants to check it.

Version
5 · in good standing
Author
Read Achilles · unverified at publication
Published
6 Nov 2026 · corrected 6 Nov 2026
Claims
9 labelled — 6 grounded · 2 author assertion · 1 synthesis
Citations
6 linked sources
Distribution
Shared with the author’s consent — including to agents acting for readers, under the author’s per-post opt-in
Content fingerprint
9a1994fe…d89e4b

Citations

  • s1 · grounded → corpus source — “Informally, a p-value is the probability under a specified statistical model that a statistical summary of the data (for example, the sam…”
  • s2 · grounded → corpus source — “P-values can indicate how incompatible the data are with a specified statistical model.”
  • s3 · grounded → corpus source — “P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random cha…”
  • s4 · grounded → corpus source — “Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.”
  • s4b · grounded → corpus source — “Proper inference requires full reporting and transparency.”
  • s5 · grounded → corpus source — “A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.”
Full verification record

The verbatim graded record of this version: identifiers, claim classes, citation targets, hashes and standing, exactly as issued.

This record is also published in structured text for agents and tools — the same fields, word for word. View structured text

House editorial — Read AchillesAuthor verification: unverifiedStanding: governingHash: 9a1994febd2f7e00f51255bbada9a79e66a257d214b0fdca851a7ea3ebd89e4b

Version & standing

Post a-p-value-is-not-a-verdict · version 5 (a-p-value-is-not-a-verdict-v5)

@achilles-pilot · verified state at publish: unverified

Published: 2026-11-06T04:45:33.879Z · Corrected: 2026-11-06T04:45:33.879Z · Standing: governing

Claims & citations

  • s1: grounded
  • s2: grounded
  • s3: grounded
  • s4: grounded
  • s4b: grounded
  • s5: grounded
  • s6: author_assertion
  • s7: synthesis
  • s8: author_assertion
  • s1 → corpus source (“Informally, a p-value is the probability under a specified statistical model that a statistical summary of the data (for example, the sample mean difference between two compared groups) would be equal to or more extreme than its observed value.”)
  • s2 → corpus source (“P-values can indicate how incompatible the data are with a specified statistical model.”)
  • s3 → corpus source (“P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.”)
  • s4 → corpus source (“Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.”)
  • s4b → corpus source (“Proper inference requires full reporting and transparency.”)
  • s5 → corpus source (“A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.”)

Disclosure & trust

When your words reach the end of their retention period they are destroyed, on the schedule above. What the platform keeps after that — indefinitely, with no expiry — is a skeleton of acts: that an act occurred, its class and time, the bundle and section it touched, its change class, a hash that identifies the record without containing it, and the keys that link a correction to what it corrects. The skeleton is not your words and cannot reproduce them. Your elections to shorten a period, to ask ephemerally, or to skeletonise now govern your words; they do not shorten, and cannot remove, the skeleton — it is the record that lets the platform prove what it did with your work after the words are gone. The platform will never produce your skeleton by choice against you; like any record it can be reached by a court order the platform cannot refuse, and if that happens you will be told wherever the law allows and the production will state, on its face, that it is a skeleton and that the words expired on this schedule.

When your words reach the end of their retention period they are destroyed, on the schedule above. What the platform keeps after that — indefinitely, with no expiry — is a skeleton of acts: that an act occurred, its class and time, the bundle and section it touched, its change class, a hash that identifies the record without containing it, and the keys that link a correction to what it corrects. The skeleton is not your words and cannot reproduce them. Your elections to shorten a period, to ask ephemerally, or to skeletonise now govern your words; they do not shorten, and cannot remove, the skeleton — it is the record that lets the platform prove what it did with your work after the words are gone. The platform will never produce your skeleton by choice against you; like any record it can be reached by a court order the platform cannot refuse, and if that happens you will be told wherever the law allows and the production will state, on its face, that it is a skeleton and that the words expired on this schedule.

Integrity & distribution

Content hash: 9a1994febd2f7e00f51255bbada9a79e66a257d214b0fdca851a7ea3ebd89e4b

Distribution: opted_in

Canonical presentation: https://blog.readachilles.com/a-p-value-is-not-a-verdict/

What we retain, and why — the retention disclosure

The retention disclosure for this version is the disclosure line of record, stated in full in the colophon fields above.