EvidenceLens: A Self-Hosted Biomedical Search Engine with Hybrid Ranking and Conflict-of-Interest Transparency

The Problem with Medical Evidence Is That It’s Everywhere

Finding reliable medical evidence requires searching seven different places: PubMed for peer-reviewed literature, bioRxiv and medRxiv for preprints, ClinicalTrials.gov for trial registrations, openFDA for drug recalls, CMS Open Payments for industry funding disclosures, NIH RePORTER for research grants, OpenAlex for citation networks. None of these databases talk to each other. You can’t rank results across them by relevance. And there’s no way to see at a glance whether the author you’re reading received payments from the company whose product they’re studying.

EvidenceLens is a self-hosted search engine that unifies 20+ authoritative sources into a single hybrid-ranked interface, automatically surfaces conflict-of-interest badges next to author names, and streams answer synthesis to the user’s own inference key so the backend never pays for a single LLM token.

A Five-Stage Pipeline Across Three Languages

The system is event-driven end-to-end, split across five stages that communicate through NATS JetStream:

Ingestion — 15+ Go ingesters run on cron schedules (via ofelia), each watermark-tracked so reruns are safe. They pull from source APIs at configured rate limits, archive raw payloads to S3-compatible storage, and publish RawDocEvent messages to NATS.

Processing — A Python service subscribes to raw events and runs each document through a pipeline: source-specific parsing, author normalization, optional scispaCy entity linking, sliding-window tokenization into 512-token chunks, BGE-M3 embedding via gRPC, and conflict-of-interest fuzzy matching against Open Payments records. It runs 50 concurrent pipelines before publishing enriched IndexableDocEvent messages downstream.

Indexing — A Go indexer fans out to three parallel batch writers: Meilisearch for full-text (BM25), Milvus for semantic vector search across 1024-dimensional embeddings, and Neo4j for the citation graph. Failed documents route to a dead-letter queue for audit. Every writer handles upserts idempotently.

Ranking — When a search query arrives, a Python scorer spawns four concurrent sub-scorers and streams results in three waves: BM25 and vector results by 200ms, citation PageRank and recency signals by 500ms, then Reciprocal Rank Fusion over all four signals followed by optional LambdaMART reranking by 1000ms.

Synthesis — The frontend receives ranked results and hands off to whatever inference the user brings. Three free options: paste a BYOK API key (Anthropic, OpenAI, Groq, Ollama), connect Claude Desktop or Cursor via the bundled MCP server, or use in-browser WebLLM inference that runs entirely client-side. The backend never sees an API key and never pays for tokens.

Why the BYOK Design Was the Right Call

The economic constraint shaped every synthesis-layer decision. If the backend proxied LLM calls on behalf of users, cost grows linearly with usage and the only way to control it is rate limiting — which kills the tool’s usefulness. Shifting synthesis to the user’s own inference key removes the backend from the cost equation entirely. A popular result set that triggers thousands of synthesis requests costs me nothing.

The MCP server integration is the option I’m most interested in. When Claude Desktop or Cursor connects to the local MCP server, the full search API is available as a tool — including faceted filtering by study type, year range, license, and MeSH terms. An AI assistant can issue structured queries, synthesize answers from ranked results, and cite sources with full provenance, without any backend token cost. That’s a more capable workflow than a chat UI that just takes freeform queries.

Stack

Three languages for three different performance profiles. Go for the I/O-bound ingesters and indexer — goroutines handle concurrent source crawling and batch writes cheaply. Python for the ML-heavy processing and scoring layers where PyTorch, scikit-learn, and spaCy live. TypeScript for the gateway and MCP server where the async patterns of NestJS and the Node ecosystem fit naturally.

The data plane (PostgreSQL, NATS, Redis, Meilisearch, Milvus, Neo4j) runs on Docker Compose with profiles for five deployment configurations: local dev with everything colocated, VPS with the app layer pointing at a remote TrueNAS data plane, a TrueNAS-native setup where heavy compute runs on the NAS, and a production overlay for external managed databases.

The Hardest Part: Combining Four Incommensurable Signals

The ranking system is the most architecturally demanding piece. The challenge isn’t running four sub-scorers — it’s that their outputs operate on completely different statistical scales:

• BM25 returns raw term-frequency scores in the range 0 to infinity
• Vector cosine similarity returns values between 0 and 1
• Citation PageRank is power-law distributed with a heavy tail
• Recency is exponential decay capped at 1

Naively summing or averaging these produces rankings dominated by whichever signal has the largest magnitude — usually BM25 or citation count. Papers with thousands of citations crowd out relevant recent work regardless of semantic match.

The solution is Reciprocal Rank Fusion. Rather than combining raw scores, RRF converts each sub-scorer’s results into a ranked list and combines the ranks:

RRF_score(doc) = Σ_scorer [ 1 / (60 + rank_in_that_scorer) ]

The constant k=60 is tunable and controls how much weight rank-1 results get relative to rank-10 results. Because RRF operates on ordinal positions rather than raw values, it’s robust to scale differences — a document ranked first in vector similarity and fifth in BM25 scores well regardless of what those raw numbers actually were. It doesn’t require normalizing any of the sub-scorers.

On top of RRF, an optional XGBoost LambdaMART model reranks the top 50 candidates using a feature vector that includes BM25 score, vector similarity, PageRank, recency decay, COI flag, and study type. LambdaMART is trained on click-based pairwise preference labels collected through team draft interleaving A/B tests — when result set A and result set B are interleaved and a user clicks, the clicked document gets a positive preference signal over non-clicked documents at similar positions. The model learns weights that reflect actual user behavior rather than intuitions about which signal matters most.

The three-wave streaming architecture exists because BM25 and vector search return in under 200ms, but citation PageRank and LTR reranking take longer. Rather than making the user wait for the full pipeline, the frontend renders a useful first result set immediately and progressively refines it as subsequent waves arrive over WebSocket.

Conflict-of-Interest Badges Deserve More Engineering Than They Usually Get

The COI badge system is the feature that required the most careful thought about correctness. CMS Open Payments is a public database of payments from pharmaceutical and medical device companies to physicians — millions of records per year, searchable by name and NPI. Matching paper authors to payment records sounds straightforward until you account for how names actually appear in the data: “Smith JA” in a PubMed author list might match “John A. Smith” or “John Andrew Smith” or “Jonathan Smith” or none of them.

The matching pipeline uses trigram similarity on a normalized representation — last name plus first initial — with a confidence threshold of 0.90. When affiliation state is available (some sources include institutional affiliations), state-aware matching lowers the false-positive risk by restricting to payment records in the same geography. Matches are cached in Postgres for 30 days with a gin_trgm_ops index on the normalized author key for fast lookups.

At 0.90 confidence, the system accepts some false negatives (missed relationships) to minimize false positives (wrongly flagging an author). For a transparency feature that affects how readers interpret research, false positives are the worse failure. A missed badge is a missed disclosure; a wrong badge is an accusation.

What Building This Changed About How I Think About Search

RRF is a deceptively good default. I went into this expecting to spend significant time tuning normalization strategies for the sub-scorer outputs. RRF made most of that unnecessary. The insight is that rank position is more stable than raw score — a document’s relative position in a result list is less sensitive to query characteristics than its absolute score. Using ranks rather than scores as the fusion input sidesteps a class of normalization problems entirely.

Event-driven decoupling is worth the operational overhead. NATS JetStream adds complexity: you’re now reasoning about consumer groups, delivery policies, and dead-letter queues instead of just function calls. But the benefit is that each stage can scale independently and fail independently. When the embedder goes down for maintenance, raw events accumulate in the JetStream without losing data. When it comes back up, it drains the backlog. Without the message bus, a processing failure during ingestion means lost documents.

The inference cost problem is a design problem, not a capacity problem. It’s tempting to approach LLM-powered search as a capacity planning exercise — how many tokens per query, how many queries per day, what tier of API access. The question underneath is why the backend is responsible for synthesis at all. Shifting synthesis to the user’s own key doesn’t degrade the product; if anything it gives users more control over which model they use and removes a major operational risk from the system.

What’s Next

The citation graph is currently populated from reference lists in ingested documents, but the PageRank is computed as a batch process when new edges arrive rather than incrementally. For a rapidly-ingested corpus, that means citation scores can be stale by hours. Incremental PageRank updates on new edge batches would keep scores current without the batch overhead.

The LambdaMART model is trained offline on collected click data, but there’s no automated retraining pipeline — it’s a manual step. Connecting the click event pipeline to a scheduled training job that retrains on a rolling window of recent clicks would let the ranker adapt as the corpus and user behavior evolve.

The MCP server currently exposes search and synthesis tools, but not faceted browsing or document metadata retrieval. Adding tools for structured queries — “find phase III trials on this intervention in the last two years” — would make the AI tool interface more useful for systematic evidence review workflows, which is the use case where structured querying matters most.