What counts as a claim
A paper verification report lists the claims a paper makes. A model reads the paper and proposes claims; each one is kept only when its quote appears word for word in the text. A second model then classifies each claim against the definition below. The same text is given to both models and to the people who annotate our test set, so all three read one definition.
Reading a claim card
Section titled “Reading a claim card”- Kind, basis and strength describe the claim. They never judge it.
- Tests: ✓ passes, ? unclear, ✗ fails, one mark for each of the four tests below.
- Also stated in: the same claim appears in another part of the paper. Each place shows how firmly it is worded there.
- Assumption: the claim rests on something the paper does not support. Assumptions are listed in their own section of the report and are never counted as claims.
- Lower-confidence claims are shown collapsed with the reason, never hidden. On a paper’s public report everyone sees them. On a report of a PDF you upload, without an account you see how many there are and why; a free account shows each one. Possible issues are counted on every report, and a free account shows them.
- Single case or n = N: how many cases the claim’s own evidence rests on, when the text states it.
- Nominal p (uncorrected): the claim’s significance rests on a p-value with no correction for multiple comparisons stated, and either many tests were run or p is just under 0.05.
- Possible issue — not verified marks a signal that no recomputation and no reasoning-model review has confirmed.
- A cited source marked Checked by a reasoning model — may be wrong was judged by a model. Read the claim’s quote and the source before relying on the verdict.
The definition
Section titled “The definition”A substantial claim is one proposition that the paper, in its own voice, puts forward as true or probably true, and that passes four tests:
S1 — Asserted. The authors commit to it, with or without hedging. A question, an aim, a hypothesis the study sets out to test, planned or future work, and a view the paper reports without endorsing are not asserted.
S2 — Checkable. Evidence could show it false. It names what it is about and what it says of it precisely enough that a reader could check it against data, a source, or the paper’s own methods. Opinions, recommendations, definitions and statements about the text itself are not checkable.
S3 — Load-bearing. At least one of the paper’s conclusions, its stated contribution, or the study’s stated rationale depends on it. If it were false, a conclusion would have to be withdrawn or weakened, or the study would lose its stated reason. General context that no conclusion and no stated rationale needs (for example, burden-of-disease figures) does not pass.
S4 — Based. The paper states why it holds: its own data or analysis, a citation, or explicit reasoning from its own findings. A proposition that a conclusion rests on but that the paper gives no basis for is not a claim: it is recorded as an assumption.
Unit. One proposition per claim. A compound statement is split. Each claim keeps the qualifiers that bound it: population, condition, comparator, magnitude and uncertainty.
Identity. Two statements are the same claim when they assert the same proposition about the same population and conditions. Different wording, section or strength does not make a new claim. Each statement becomes one more anchor of the one claim, and every anchor records its own strength.
What is not a claim
Section titled “What is not a claim”- Aims, research questions, hypotheses the study will test (fails S1).
- Future work, open questions (fails S1).
- Views reported without endorsement (fails S1).
- Speculation the authors mark as speculation (fails S1).
- Definitions, notation, text about the text (fails S2).
- Opinions and recommendations (fails S2). The proposition it rests on is extracted only if the paper states it.
- Procedure-only statements that are not validity claims (fails S3).
- Uncited common knowledge that no conclusion relies on (fails S3).
- Restatements of a claim already extracted (Identity): one claim with one more anchor.
- Assumptions: propositions a conclusion rests on that the paper does not support (fails S4). Recorded as an assumption, not as a claim.
Assumptions
Section titled “Assumptions”An assumption is a proposition the paper relies on for a conclusion but does not establish or support: it gives no data or analysis of its own, no citation and no argument for it. “Relies on” is the S3 test: if it were false, a conclusion, the stated contribution or the study’s stated rationale would have to be withdrawn or weakened. A proposition the paper itself presents as an assumption (“we assume”, “assuming that”, a model assumption) is an assumption whatever it cites.
An assumption is not a claim. It is not counted as a claim, not tested against S1–S4 and not verified as a claim.
Kinds of claim
Section titled “Kinds of claim”| Kind | What it is |
|---|---|
| Finding | States what the paper’s own data or analysis showed: an observed value, a difference, an association, or a null result. Test: checkable against the paper’s own results or tables alone. |
| Interpretation | Goes beyond what was observed: an interpretation, a mechanism, a causal conclusion, a generalisation to a wider population, or an implication. A stated limit on how far the results reach (population, setting) is an inference. Test: checking it needs the findings plus a reasoning step. |
| Premise | A proposition the argument uses that the paper did not establish itself and did not do, usually taken from prior work, and for which the paper gives a citation or an argument. The paper’s own procedure is not a premise. A proposition the argument uses with no support is an assumption, not a premise. Test: the paper treats it as given in order to motivate, design or interpret the study. |
| Study design and conduct | A statement that the study was designed, conducted or analysed with a safeguard on which the credibility of the findings depends. Examples: sequence generation, allocation concealment, blinding, sample-size justification, confounder adjustment, risk-of-bias assessment, validated instrument, preregistration; in designs without such a checklist, the same kind of safeguard (data saturation, triangulation across methods, calibration against data, replicate runs). Results about the conduct of the study (baseline balance, attrition, compliance) and stated limits on how it was run or analysed (power, attrition, blinding) are validity claims. A description of what was done (arms, doses, follow-up length, software, the test used) is procedure, not a safeguard, and is not a claim. Test: a CONSORT 2025, STROBE or PRISMA 2020 item covers it, or, in a design those checklists do not cover, the credibility of the findings rests on it. |
Basis: why the paper says it holds
Section titled “Basis: why the paper says it holds”| Value | What it means |
|---|---|
| Own data | The paper’s own data or analysis is the reason it holds. |
| Citation | A cited work is the reason it holds. |
| Reasoning | Derived in the text from other claims of the paper. |
| None | Nothing is given; a substantial claim never has it (S4), and a statement a conclusion rests on with nothing given is an assumption. |
Strength: how firmly it is worded
Section titled “Strength: how firmly it is worded”| Value | What it means |
|---|---|
| Hedged | A marker lowers commitment (may, might, could, would, suggests, indicates that, likely, probably, possible, putative, appears, estimated, to our knowledge). |
| Plain | Neither. The lists are closed: “can” (capability) and words of magnitude (substantially, slightly) are neither. |
| Boosted | A marker raises it (clearly, demonstrates, have demonstrated, conclusively, strongly, well established, well known). |
Causal wording
Section titled “Causal wording”| Value | What it means |
|---|---|
| No relationship | Covers a claim that relates no two variables (an attitude participants hold is not a relationship) and a claim whose proposition is the absence of an effect or difference. |
| Correlational | Covers “associated with” and “linked to”. |
| Conditional causal | Causal wording with a hedge (“may increase”). |
| Direct causal | Unhedged causal wording (“increases”, “reduces”, “leads to”). |
The wording decides, never the design, and a simulated mechanism is coded by its wording like any claim; when anchors word it differently, the strongest anchor’s wording sets the level.
Evidence scale and nominal p-values
Section titled “Evidence scale and nominal p-values”Two more labels say what a claim’s evidence rests on. Like the others, they are a classification model’s reading of the text, not a check of it. A nominal p-value carries the claim’s significance with no correction for multiple comparisons stated.
| Label | When it shows |
|---|---|
| Single case | The claim’s own evidence is one case: one patient, participant, donor, family or animal, even when compared with controls. |
| n = N | The number of subjects, cases, donors or samples the claim’s own evidence rests on, as the text states it: the cases the result was observed in, not the size of a cited study, a population, a dose or a duration. It shows only when the text states the number. |
| Nominal p (uncorrected): near the threshold | Nominal, from one test or a few, and p is near the threshold (0.01 ≤ p < 0.05). |
| Nominal p (uncorrected): many tests | Nominal, one of many tests (variants, outcomes, subgroups, models or time points), and p is below 0.01. |
| Nominal p (uncorrected): many tests, near the threshold | Nominal, one of many tests, and p is near the threshold (0.01 ≤ p < 0.05). |
A correction for multiple comparisons that the paper states for the claim, or a named adjustment (Bonferroni, Holm, Tukey, Šidák, Scheffé, a false discovery rate, Dunn’s test) stated in its section or in the methods, removes the nominal-p label whatever the model answered. The card says why the label shows.
How we measure confidence
Section titled “How we measure confidence”No field is calibrated yet, so no claim card shows a confidence band. Every label is shown on its own, and causal wording is marked “not yet calibrated”.
A band means the following: in our annotated test set, answers in that band were right at least as often as the target set for it. No percentage is ever shown for a single answer.