Qaf· Retrieval analysis · Oct 14 2026Answers cite deep into the 20 passages. Cutting to 5 drops over half of what they use today.
Top-ranked passages are cited most (56% of rank-1 passages vs 14% at rank 20), but there is no clear cut-off. Ranks 11–20 still supply 29% of citations. A limit of 10 keeps 71%; a limit of 5 keeps 46%.
How often a passage gets cited, by its rank
V2 agent, 667 searches over 224 production-question turns. Bars show 95% intervals.
Share of today's citations each limit would keep
Dashed line = what it would look like if every rank were equally useful.
Per limit
| Passages per search | Citations kept | Citations lost | Turns with all citations kept | Cite rate of last 5 ranks |
|---|---|---|---|---|
| 5 | 46% | 54% | 8% | 40% |
| 10 | 71% | 29% | 18% | 24% |
| 15 | 89% | 11% | 38% | 19% |
| 20 | 100% | 0% | 100% | 15% |
Arabic vs English is similar: a limit of 5 keeps 45% / 47% and 10 keeps 71% / 72%.
What this means
- No natural elbow. The cite rate falls fast over ranks 1–3, then flattens at 15–25%. Every extra passage keeps paying off a little.
- 5 is aggressive. Over half of today's citations come from ranks 6–20, and only 8% of turns cite only from the top 5. The model will need more searches to compensate. The running bench (V2, low thinking + 5 passages) measures whether it does and what that costs.
- 10 is the better middle ground if the goal is fewer input tokens: about half the passages, keeping 71% of citations.
Method: 100 production questions from Langfuse (50 Arabic / 50 English, 112 turns), V2 runs from the Oct 10 (prompt v29) and Oct 11 (PR #595 baseline) benchmarks, Gemini 3.7 Flash. Each delivered passage is counted once, at the rank where it first appeared in the turn; expand results count toward the rank of the passage they expanded. "Cited" means its ID appears in the answer's citations. This measures what answers use, not what they need: models also favour items listed first, and with fewer passages the model would search differently.