Bigger models hallucinate less.
On niche topics, every model hallucinates more.
Both effects are predictable.
We asked language models to cite real scholarly work on 24 research topics, then verified every citation. The picture that emerges is simple: an LLM stores concepts in overlapping superposition, so each fact is read out against background interference from every other fact. Larger models have a lower noise floor, and topics with more training data sit higher above it. Niche topics hurt every model; small models miss on every topic.
Smith, Shock, Segun, Olatunji, Bissyandé, 2026. Factual recall errors in large language models follow log-linear scaling laws.
arXiv preprint, May 2026.
@article{smith2026recall,
title = {Factual recall errors in large
language models follow log-linear
scaling laws},
author = {Smith, Shock, Segun, Olatunji,
and Bissyand\'e},
year = {2026},
archivePrefix = {arXiv}
}Two variables. Sixty percent of the variance.
The log of the model's parameter count, and the log of how often a topic appears in the literature. Together those two numbers explain most of what we see across 743 model-by-topic cells. Size carries about three times the weight of topic, but topic matters enough that even frontier models miss on underrepresented topics.
Size sets the floor.
A model's parameter count strongly shapes how often it can produce real references. The relationship is log-linear within a family and sigmoidal across the population: a sharp climb from 10B through 400B, with diminishing returns beyond.
- Within-family R²
- 0.79
- Half-max
- ~135B parameters
- Models
- 16 dense, 4 families
Citation quality vs total parameters, log scale. R² = 0.79 within dense families; family offsets reflect training procedure, not scale.
Topic sets the signal strength.
The same model that nails climate change fabricates citations on rural school-dropout policy. Reading down a column shows how recall collapses as topics get rarer; reading across a row shows the size effect within a topic.
- Topic range
- S ≈ 32 → 1.2M papers
- Best-case spread
- 0.90 → 0.43 (same model)
- Topic effect
- ≈ ⅓ the size effect
Citation quality, model × topic. Topics ordered by OpenAlex publication count (most-represented left). Five orders of magnitude in topic representation; quality drops with both axes.
If signal strength matters, this is what we should see.
A within-topic test of the same hypothesis. If recall really is gated by training-data frequency, then for any single topic, smaller models should pull only the most-cited papers and larger ones should be able to reach further down the citation tail. They do: median citation count of correctly-recalled references drops log-linearly with model size. The same noise-floor story holds at the level of individual papers, not just whole topics.
- Slope
- −0.35 per decade of params
- 8B median
- ~2,260 citations
- 405B median
- ~660 citations
Median citations of correctly-recalled work, n = 10 dense models with ≥50 matched refs. Slope −0.35, R² = 0.59, Spearman ρ = −0.79 (p = 0.007).
Hallucination isn't pass or fail.
It's tempting to think a model either gets a citation right or makes one up. The data says no. Across the Llama family, the share of fully-verified references climbs and the share of total fabrications drops, but a wide intermediate band persists at every size: real papers with a wrong year, a missing subtitle, a fabricated coauthor. As capacity falls, the fully-verified bucket shrinks and the corrupted-but-real bucket fills in before anything turns into pure invention. Recall degrades; it doesn't fail.
- Llama 405B
- 85% real · 12% corrupted · 3% fake
- Llama 70B
- 63% real · 19% corrupted · 18% fake
- Llama 8B
- 30% real · 28% corrupted · 42% fake
Verification-status mix for Llama 8B → 70B → 405B. The 'verified-with-error' band sits in the middle at every size — confabulation moves through a graded continuum, not a switch from real to fake.
The model is louder than the signal,
until it isn't.
A modern LLM compresses the patterns from trillions of training tokens into a few billion parameters. There isn't enough room to give each pattern its own slot, so they end up sharing: overlapping in the same directions and interfering with each other. Researchers call this superposition.
Recalling a fact is a contest between two quantities: how strongly the concept was written into the model (its signal), and how loud the interference is from everything else (the noise floor). Whether the model produces a real citation or a confident fabrication depends on which side wins.
- P
- total parameters → sets the noise floor (∝ 1/√N)
- S
- topic representation → sets the signal strength
- σ
- sigmoid: bounded recall in [0, 1]
- c
- family offset: data quality, training procedure
Both contributions enter the SNR multiplicatively, which is what produces a sigmoid in their log-sum. The fit holds at R² = 0.60 across 743 cells.
Three regimes
Noise drowns signal. Models produce templated fabrications: plausible authors, real journals, papers that don't exist.
The log-linear regime. Each decade of parameters or each decade of topic data lifts quality by a fixed amount.
Capacity exceeds the concept inventory. Quality saturates near 1; further scaling buys little.
The largest model we tested, DeepSeek V4 Pro at 1.6 T parameters, scores 0.90 on climate change (S ≈ 1.2 M papers). On school dropout prevention in rural areas (S = 32) it scores 0.43. Solving the fit for what it would take to reach 0.90 on the second topic at today's training-data density: roughly 50 trillion parameters. Thirty times the largest model that exists.
Scaling is coverage expansion,
not uniform improvement.
Confabulation is structural, not random.
Hallucinations track a predictable inequality in how knowledge is encoded. High-frequency concepts sit above the interference floor and are recalled reliably; low-frequency ones sit below it and are filled in with plausible-looking guesses. The pattern is consistent enough across 38 models and four families that we can fit it with two variables and a sigmoid.
Aggregate benchmarks hide topical disparities.
Averaging over the frequency spectrum overstates reliability on the long tail. The same DeepSeek V4 Pro that scores 0.90 on climate change scores 0.43 on school dropout in rural areas. A leaderboard number averaged across these is a polite fiction; the same disparity surfaces across geographies and languages whenever training-data density is uneven.
Same parameter count, different family, different quality.
Beyond size and topic, persistent gaps remain between model families at the same parameter count. Llama 3.1 70B outperforms Llama 3.3 70B on this task; MoE models tend to sit below the dense trend line when compared at total parameters. We don't try to attribute these gaps to a single cause: pre-training data, post-training methods, and architecture all vary together across families.
Above the floor, curation. Below it, retrieval.
For concepts near the noise floor, modest signal increases (better data, targeted pre-training, in-context priming) push them across the recall threshold. For concepts well below the floor, no realistic amount of scaling helps in time: pushing the rural-school-dropout topic to 0.90 quality at today's data density would take roughly 50 trillion parameters. The appropriate response there is retrieval augmentation that bypasses parametric recall entirely.
Predict your own query.
Pick a topic and a model size. We pull the topic's OpenAlex coverage in real time, run it through the fitted sigmoid, and report how many of ten requested citations are likely to survive verification.
Try an example
How the field stacks up.
Ten of 38 models, ranked by overall citation quality (authenticity × topic relevance) across all 24 topics. The full ranking, including small models and earlier generations, lives on the leaderboard.
- 01DeepSeek V4 ProMoE·49B / 1.6TMoE49B / 1.6T0.89auth0.85quality
- 02Claude Opus 4.6Closed·—Closed—0.92auth0.81quality
- 03GPT-5Closed·—Closed—0.88auth0.76quality
- 04GPT-5.4Closed·—Closed—0.88auth0.76quality
- 05DeepSeek V3MoE·37B / 671BMoE37B / 671B0.85auth0.75quality
- 06Llama 3.1 405B HermesDense·405BDense405B0.88auth0.70quality
- 07Claude Sonnet 4.6Closed·—Closed—0.86auth0.69quality
- 08Grok 3Closed·—Closed—0.75auth0.69quality
- 09Qwen3.5MoE·17B / 397BMoE17B / 397B0.71auth0.67quality
- 10Llama 3.1 405B baseDense·405BDense405B0.78auth0.64quality
How the dataset was built.
We asked each of 38 language models to produce ten APA-formatted scholarly citations for each of 24 research topics, at temperature zero, with no token cap. Every returned reference was passed through SourceVerify (using the SVRIS comparison standard), which resolves five fields (title, identifier, authors, year, venue) against OpenAlex, Crossref, and the open web. A separate Kimi K2 judge rated topic relevance. The product of authenticity and relevance is the per-cell quality score plotted throughout.
- Requested
- 9,120
- 38 models × 24 topics × 10 refs
- Produced
- 8,913−207
- cells where the model returned fewer than ten parseable APA citations
- Analyzed
- 8,829−84
- normalized-title duplicates collapsed within each model × topic
Four independent reviewers, blinded to the SourceVerify verdict, manually searched 288 unique references against the open literature. SourceVerify's field-level scoring agrees with human judgment at κ = 0.887, with no false positives across the audit set: every citation SourceVerify flagged as real was confirmed by a reviewer.
- Recall
- 88.9%
- Specificity
- 100%
- Cohen's κ
- 0.887
- Ratings
- n = 301
The 17 references SourceVerify marked as unverified that reviewers later judged real all carried at least one citation defect: a wrong subtitle, a misattributed coauthor, a first-edition year paired with a later publisher. SourceVerify isn't missing clean citations; it's flagging defective ones for review.