Four scoring signals feed into EvidenceLens’s ranking pipeline: BM25 from Meilisearch, cosine similarity from Milvus, PageRank from Neo4j’s citation graph, and an exponential time-decay function for recency. The challenge isn’t running all four — it’s combining them. BM25 returns term-frequency scores from 0 to infinity. Vector cosine similarity returns values between 0 and 1. Citation PageRank is power-law distributed with a long tail. Recency decay is bounded between 0 and 1, with most older papers scoring near zero. Naively summing these produces rankings dominated by whichever signal has the largest magnitude — usually BM25 or citation count — regardless of what’s actually relevant.
Reciprocal Rank Fusion sidesteps this by never looking at raw scores. It converts each scorer’s output into an ordered list and combines the rank positions. A document appearing at rank 3 in BM25 and rank 1 in vector search gets a specific combined score regardless of what those underlying numbers actually were. That robustness to scale differences is what makes RRF a useful default when you don’t want to spend weeks calibrating normalization strategies.
What You Need Coming In
- Python familiarity: dataclasses,
defaultdict, sorting - Basic understanding of what BM25 and vector search return — ranked lists of documents with associated scores
- No knowledge of PageRank or decay functions required; RRF treats each sub-scorer as a black box
Why Weighted Sums Break
Before getting into RRF, it’s worth understanding why the naive approach fails in a way that’s hard to fix.
final_score = w1*bm25 + w2*vector + w3*pagerank + w4*recency looks controllable. But the weights interact with raw score distributions in ways that shift as the corpus changes. BM25 scores depend on corpus statistics — average document length, term frequency across the entire index — so they drift when you ingest large batches of new documents. A weight that balanced BM25 against cosine similarity after the initial corpus load might stop working after ingesting another 50,000 papers that shifted the IDF values.
PageRank compounds this. Citation networks follow power-law distributions: a handful of seminal papers have values orders of magnitude higher than typical papers. Any coefficient you choose for PageRank either lets high-citation papers dominate everything else, or discounts it so heavily that it contributes almost nothing. There’s no static weight that handles both extremes.
RRF avoids all of this by discarding raw values entirely. It only cares about order.
How RRF Works
The formula is:
RRF_score(doc) = Σ_scorer [ 1 / (k + rank) ]rank is the document’s position in that scorer’s result list, 1-indexed. k is a smoothing constant, conventionally 60. For a document at rank 1 in BM25 and rank 5 in vector search with k=60:
score = 1/(60+1) + 1/(60+5) = 0.01639 + 0.01538 = 0.03177A document at rank 200 in both:
score = 1/(60+200) + 1/(60+200) = 0.00385 + 0.00385 = 0.00769The k constant controls how steeply weight drops off with rank. With k=60, the weight ratio between rank 1 and rank 2 is 61/62 ≈ 0.984 — the top result is barely more valuable than the second. This means a document consistently in the top 10 across multiple scorers outranks one that’s rank 1 in only a single scorer. With k=1, that ratio becomes 2/3, making top-rank dominance much stronger.
The Implementation: Eleven Lines
scorer/fusion.py is the entire thing:
from collections import defaultdict
from dataclasses import dataclass
@dataclass
class FusedItem:
doc_id: str
rrf_score: float = 0.0
bm25_score: float = 0.0
vector_score: float = 0.0
citation_pagerank: float = 0.0
recency_score: float = 0.0
def rrf(rankings: dict[str, list[str]], k: int = 60) -> list[FusedItem]:
accum: dict[str, FusedItem] = defaultdict(lambda: FusedItem(doc_id=""))
for _scorer, ids in rankings.items():
for rank, doc_id in enumerate(ids):
item = accum[doc_id]
item.doc_id = doc_id
item.rrf_score += 1.0 / (k + rank + 1)
return sorted(accum.values(), key=lambda x: x.rrf_score, reverse=True)rankings maps scorer name to a list of doc IDs in rank order — the caller sorts each sub-scorer’s output before passing it in. The accumulator builds one FusedItem per unique document across all scorers. The +1 in the denominator makes the formula 1-indexed: position 0 in the Python list becomes rank 1 in the formula, giving 1/(60+0+1) = 1/61.
FusedItem carries the raw sub-scorer values as fields even though they don’t affect fusion. The frontend uses them for the detail panel — clicking a result shows which signals contributed to its ranking.
How the Sub-Scorers Feed In
Each sub-scorer returns hits sorted by its own metric. The orchestrator in scorer/main.py extracts ordered doc ID lists:
bm25_hits = await _bm25() # list[BM25Hit], sorted by BM25 score
vec_hits = await _vector() # list[VectorHit], sorted by cosine similarity
rankings = {
"bm25": [h.doc_id for h in bm25_hits],
"vector": [h.doc_id for h in vec_hits],
}
fused = rrf(rankings, k=60)BM25 hits come from Meilisearch with showRankingScore: True. Vector hits come from Milvus with cosine metric. Citation hits are PageRank values fetched from Neo4j for the union of BM25+vector doc IDs. Recency is computed inline from each document’s published_at timestamp using a 730-day half-life:
# scorer/recency.py
def recency_score(published_at_iso: str | None, half_life_days: int = 730) -> float:
age_days = (datetime.now(timezone.utc) - ts).total_seconds() / 86400.0
return 0.5 ** (age_days / float(half_life_days))A paper published today scores ~1.0. A paper published two years ago (one half-life) scores 0.5. A paper from 2010 scores close to zero.
When all four signals are ready, the full fusion runs:
rankings["citation"] = [c.doc_id for c in citation_hits_sorted_by_pagerank]
rankings["recency"] = recency_sorted_ids
fused = rrf(rankings, k=60)The union of documents across all four scorers expands the candidate set. A document appearing in only one scorer’s top 200 still gets an RRF score — just lower than documents appearing across multiple signals.
Streaming Waves Work Because RRF Is Incremental
The three-wave streaming architecture relies on a specific property of RRF: it can be applied with any subset of scorers and produces a valid ranking. Wave 1 at 200ms uses just BM25 and vector (fastest to compute). Wave 2 at 500ms adds citation and recency. Wave 3 at 1000ms applies the LambdaMART reranker on top of the four-signal fusion.
The consistency property that makes this work: documents that rank well under two-signal RRF tend to rank well under four-signal RRF. In biomedical search, papers that are semantically relevant and terminologically relevant are usually the same papers with meaningful citation presence — the signals correlate positively in the range that matters. The first wave gives the user something useful immediately, and subsequent waves refine rather than overturn it.
Tuning k as an A/B Variable
The k=60 default comes from the original Cormack, Clarke, and Buettcher paper. It’s not magic — it’s a starting point. In config/experiments.yaml, k is exposed as an A/B experiment variable so different values can be compared against click data without code changes.
Higher k values flatten the rank weights, making consensus across signals more important than individual-signal dominance. For biomedical search, where a highly-cited landmark paper might be perfectly relevant even if its abstract uses different terminology than the query, higher k tends to favor those papers. Lower k amplifies top-rank dominance in individual signals, which works better when you trust one scorer more than the others.
The LambdaMART reranker on top of RRF partially substitutes for k-tuning by learning relative signal weights from actual click behavior — covered in a separate tutorial on team draft interleaving and learning-to-rank.
