Retrieval-Augmented Generation (RAG)

In one line

RAG answers questions from your knowledge base, not the model's memory: fetch the most relevant passages, hand them to the model with the question, and require the answer to cite them.

The problem it solves

A large language model knows only what it absorbed in training: it can't see your hospital's protocols, it goes stale, and when it doesn't know, it tends to invent a fluent, confident answer. In healthcare that's unacceptable. RAG fixes all three at once — it injects current, owned, authoritative text into the prompt at answer time, so the model reasons over real sources instead of memory.

The two phases

Indexing (offline):

1. Split documents into chunks (a paragraph or a section — size matters).
2. Compute an embedding for each chunk.
3. Store them in a vector index.

Retrieval + generation (at query time):

1. Embed the user's question the same way.
2. Retrieve the nearest chunks (often hybrid: vector + keyword), then re-rank.
3. Assemble them into the prompt with the question.
4. Generate — and require the answer to cite which chunk supports which claim.

What makes a RAG system good

• Chunking — too big and retrieval is noisy; too small and context is lost.
• Hybrid retrieval — semantic similarity plus keyword match catches both meaning and exact terms (drug names, codes).
• Re-ranking — a second pass orders candidates by true relevance before they hit the prompt.
• Grounding discipline — instruct the model to answer only from retrieved context and to say "I don't know" when retrieval is empty. "I don't know" beats invention.
• Citations — surfacing sources lets a human verify, which is the whole point in health.

Evaluating it

You can't ship a clinical assistant on vibes. The standard axes: faithfulness (does the answer stay true to the retrieved text?), answer relevance, and context relevance (did retrieval fetch the right passages?). Measuring these is part of LLMOps.

Where it shows up in digital health

• Clinical-policy assistants grounded in a hospital's own protocols.
• Coding/billing helpers grounded in payer rules and guidelines.
• This platform's Vaidya, grounded in Kosha entries with visible citations and match scores — exactly the pattern on the Ask page.

In health, ungrounded generation is a safety issue, not a style choice; RAG, with guardrails, is the difference.

Common pitfalls

• Retrieval misses the model papers over — the fluent answer hides that the right passage was never fetched. Measure context relevance.
• Stale index — the protocol changed; the index didn't. Re-index on a schedule.
• Bad chunk boundaries — splitting mid-table or mid-list destroys meaning.
• No "abstain" path — without an explicit "I don't know," empty retrieval becomes a hallucination.

Key takeaways

• RAG grounds answers in retrieved, owned text — fixing stale knowledge and invention.
• Index (chunk → embed → store), then retrieve (embed → search → re-rank) and generate with citations.
• Quality lives in chunking, hybrid retrieval, re-ranking, and an honest "I don't know."
• In healthcare it's a safety mechanism — pair it with guardrails and human review.