ARQ Leaderboard

Name: ARQ — Academic Reference Quality
Creator: SourceVerify

ARQ · Academic Reference Quality. We ask language models to cite 10 scholarly references for 24 research topics. Every reference is verified for authenticity by SourceVerify.ai and judged for topic relevance by an independent LLM judge.

≥ 0.700.50–0.690.30–0.49< 0.30

DenseCoTMoESearch

#	Model						Refs
🥇	Claude Opus 4.6 AnthropicMar 2026 Dense	UNK	9.7/10	0.924	0.872	0.806	233
🥈	DeepSeek V4 Pro MoE	1.6T49B active	9.5/10	0.891	0.952	0.847	240
🥉	GPT-5.4 OpenAIMar 2026 CoT	UNK	9.5/10	0.884	0.858	0.761	240
4	DeepSeek V4.1 Flash MoE	552B8B/16B active	9.4/10	0.911	0.977	0.891	240
5	Claude Sonnet 4.6 AnthropicMar 2026 Dense	UNK	9.3/10	0.861	0.804	0.693	240
6	Llama 3.1 405B Hermes 3 Meta / NousResearchFeb 2026 Dense	405B	9.2/10	0.879	0.788	0.703	231
7	GPT-5 OpenAIJan 2025 CoT	UNK	9.2/10	0.876	0.868	0.762	228
8	DeepSeek V3 DeepSeekMar 2026 MoE	671B37B active	9.1/10	0.851	0.885	0.754	230
9	Llama 3.1 405B MetaFeb 2025 Dense	405B	8.4/10	0.779	0.821	0.638	231
10	GPT-5 Mini OpenAIMar 2026 CoT	UNK	8.4/10	0.795	0.792	0.623	237
11	DeepSeek R1 DeepSeekFeb 2025 MoECoT	671B37B active	8.4/10	0.758	0.822	0.626	238
12	Grok 3 CoT	UNK	7.9/10	0.749	0.938	0.690	240
13	Llama 4 Maverick MetaFeb 2025 MoE	400B17B active	7.9/10	0.735	0.820	0.605	236
14	Llama 3.1 70B Dense	70B	7.7/10	0.727	0.855	0.610	212
15	Mistral Medium 3.1 MistralFeb 2025 Dense	250B	7.5/10	0.678	0.818	0.560	232
16	Qwen3.5 AlibabaMar 2026 MoE	397B17B active	7.1/10	0.711	0.932	0.665	221
17	Kimi K2 MoonshotFeb 2025 MoE	1T32B active	7.1/10	0.673	0.839	0.570	239
18	Sonar Reasoning Pro PerplexityMar 2026 SearchCoT Produced 124 of 240 requested refs, 117 of them distinct (15/24 topics). Refused to generate citations when search results lacked complete metadata. Scores recomputed 2026-09-12 after removing 99 rows where that refusal prose had been parsed as references and judged for relevance.	70B	6.8/10	0.688	0.749	0.526	117
19	Mixtral 8x22B MistralMar 2026 MoE	141B39B active	6.4/10	0.571	0.856	0.493	240
20	Llama 3.3 70B MetaFeb 2025 Dense	70B	5.9/10	0.570	0.812	0.446	236
21	GPT-5 Nano OpenAIJan 2025 CoT	UNK	5.7/10	0.547	0.824	0.432	226
22	Sonar PerplexityFeb 2025 Search	70B	5.6/10	0.521	0.783	0.390	222
23	Mistral Large 2 MistralFeb 2025 Dense	123B	4.8/10	0.450	0.860	0.365	235
24	Llama 3.1 8B MetaFeb 2025 Dense	8B	4.5/10	0.418	0.768	0.316	227
25	Mistral Small 3.2 MistralFeb 2025 Dense	24B	4.2/10	0.398	0.820	0.311	231
26	Gemma 4 31B Dense	31B	3.8/10	0.365	0.927	0.326	237
27	Qwen3 32B AlibabaFeb 2026 Dense	32B	3.8/10	0.362	0.819	0.285	239
28	MiniMax M2.5 MiniMaxFeb 2025 MoE	230B10B active	3.6/10	0.354	0.894	0.309	239
29	Mixtral 8x7B MoE	47B13B active	3.4/10	0.297	0.880	0.259	208
30	Qwen3 14B AlibabaFeb 2026 Dense	14B	3.0/10	0.274	0.810	0.227	239
31	Qwen3 32B (think) AlibabaFeb 2026 DenseCoT	32B	2.8/10	0.263	0.862	0.213	238
32	Gemma 3 27B GoogleFeb 2025 Dense	27B	2.7/10	0.273	0.813	0.220	238
33	Llama 4 Scout MetaFeb 2025 MoE	109B17B active	2.7/10	0.267	0.809	0.207	236
34	Qwen3 30B-A3B MoE	30B3B active	2.3/10	0.215	0.929	0.194	240
35	Qwen3 14B (think) AlibabaFeb 2026 DenseCoT	14B	2.1/10	0.219	0.802	0.174	240
36	GPT-OSS 120B GroqFeb 2025 MoE	120B5.1B active	1.8/10	0.162	0.858	0.136	240
37	Gemma 3 12B GoogleFeb 2025 Dense	12B	1.4/10	0.143	0.779	0.113	239
38	Qwen3 8B (think) AlibabaFeb 2026 DenseCoT	8B	1.0/10	0.139	0.833	0.107	238
39	Gemma 3 4B GoogleFeb 2025 Dense	4B	0.9/10	0.100	0.853	0.082	239
40	Llama 3.2 1B MetaMar 2026 Dense	1B	0.8/10	0.095	0.867	0.083	160
41	Qwen3 8B AlibabaFeb 2026 Dense	8B	0.3/10	0.068	0.888	0.055	235

Key Findings

Scaling works: Llama 8B (0.329) → 70B (0.466) → 405B (0.614) quality — more parameters, fewer hallucinations

Topic specificity matters: Broad topics (Health) score high; narrow topics (Insecticide-treated bed nets) score low

Representation drives quality: Quality scales with log(topic representation) in OpenAlex — R² = 0.44

MoE active params matter: Scout (17B active / 109B total) scores 0.223 — active parameters, not total, determine quality

Most models are worse than a coin flip: “Usable” counts how many of every 10 distinct references a model returns are both verifiably real and relevant to the topic. The median model manages 5.7 — 19 of 41 fall below 5, and Qwen3 8B returns 0.3