VisaSpark
Sign in
intermediate28 min

Retrieval-Augmented Generation: The Full Pipeline

Connect retrieval and generation into one system that answers from your own documents.

What you'll learn

  • Describe every stage of a RAG pipeline in order
  • Explain why RAG reduces (but does not eliminate) hallucination
  • Assemble retrieved chunks into a grounded prompt

Prerequisites

Explanation

Large language models only "know" what was in their training data, frozen at whatever point they were trained — they can't see your company's internal documents, this morning's database update, or anything private. Retrieval-augmented generation (RAG) solves this by looking up relevant information at the moment of the question and handing it to the model as part of the prompt, rather than expecting the model to already know it.

The full pipeline, end to end:

  1. Ingestion (offline, done ahead of time): documents are chunked (lesson 6) and each chunk is embedded and stored, along with metadata.
  2. Query time — retrieval: when a user asks a question, the question itself is embedded, and the system finds the most relevant stored chunks using semantic/hybrid search (lesson 7).
  3. Augmentation: the retrieved chunks are inserted into the prompt sent to the language model, typically in the system message or as clearly-labeled context, along with instructions to answer only using that content.
  4. Generation: the model produces an answer, ideally referencing the specific retrieved chunks it used.
  5. Response: the answer (and its supporting sources, covered in the next lesson) is returned to the user.

Why does this reduce hallucination (a model confidently stating something false)? Because instead of asking the model to recall a fact purely from its training, you're asking it to summarize and reason over text you just handed it — a task language models are considerably more reliable at than pure memory recall, especially for niche, private, or recently-changed information. It does not eliminate hallucination entirely: a model can still misread or misinterpret the retrieved text, or "helpfully" blend in outside knowledge when the retrieved content doesn't fully answer the question — which is why the following lessons cover explicit hallucination mitigation, evaluation, and safety measures.

A critical instruction in the augmentation step is telling the model to say "I don't know" or "the provided content doesn't answer this" when the retrieved chunks genuinely don't contain the answer, rather than falling back on unrelated training knowledge — this single instruction is one of the most important levers for building a trustworthy RAG system.

The RAG pipeline

Ingestion (chunk → embed → store) happens ahead of time. At query time: embed the question → retrieve top matching chunks → insert them into the prompt as context → the model generates an answer grounded in that context → return the answer with sources.

Example

A hand-written, offline simulation of the retrieval + augmentation steps of a RAG pipeline (no real embedding or generation API is called).

function cosineSimilarity(a, b) {
  const dot = a.reduce((s, v, i) => s + v * b[i], 0);
  const magA = Math.sqrt(a.reduce((s, v) => s + v * v, 0));
  const magB = Math.sqrt(b.reduce((s, v) => s + v * v, 0));
  return dot / (magA * magB);
}

const knowledgeBase = [
  { text: "Refunds are available within 30 days of purchase.", vector: [0.1, 0.9] },
  { text: "Shipping typically takes 3-5 business days.", vector: [0.9, 0.1] },
];

function retrieve(queryVector, topK) {
  return knowledgeBase
    .map((chunk) => ({ ...chunk, score: cosineSimilarity(queryVector, chunk.vector) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, topK);
}

function buildGroundedPrompt(question, retrievedChunks) {
  const context = retrievedChunks.map((c, i) => `[${i + 1}] ${c.text}`).join("\n");
  return `Answer ONLY using the context below. If it doesn't contain the answer, say so.\n\nContext:\n${context}\n\nQuestion: ${question}`;
}

const queryVector = [0.15, 0.85]; // stands in for embedding "can I get my money back?"
const topChunks = retrieve(queryVector, 1);
console.log(buildGroundedPrompt("Can I get my money back?", topChunks));

Try it yourself

Change topK to 2 and see both chunks appear in the grounded prompt.

Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.

Loading editor…

Guided exercise

Guided exercise

Complete `retrieveTopK(queryVector, chunks, k)` where each chunk is `{ text, vector }`. Return the k chunks (full objects) with the highest cosine similarity to queryVector, highest first.

Checks: Returns exactly k results · plus 1 hidden check

Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.

Loading editor…

Stuck? Get a hint.

Independent exercise

Independent exercise

Write `buildGroundedPrompt(question, retrievedChunks)` (retrievedChunks is an array of `{ text }`). Return a single string that contains the word 'Context' followed by every chunk's text prefixed with its 1-based index in brackets like '[1]', then the word 'Question' followed by the question text.

Checks: Includes a Context label · Numbers each chunk · plus 1 hidden check

Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.

Loading editor…

Stuck? Get a hint.

Common mistakes

  • Assuming RAG makes hallucination impossible — it reduces it, but the model can still misread or overreach beyond the retrieved content.
  • Forgetting to instruct the model to say 'I don't know' when retrieval doesn't actually cover the question.
  • Skipping the retrieval step's ranking/filtering and dumping every chunk into the prompt regardless of relevance.

Knowledge check

Knowledge check

1. What problem does RAG primarily solve?
2. In what order do the RAG pipeline's main steps happen at query time?
3. Why does RAG reduce, but not eliminate, hallucination?

Takeaway

RAG hands a model the specific facts it needs at question time instead of trusting it to already know them.

Summary

RAG chunks and embeds documents ahead of time, then at query time retrieves the most relevant chunks, inserts them into the prompt, and asks the model to answer using only that context — substantially reducing (but not eliminating) hallucination compared to relying on the model's training memory alone.

References

Your notes

Notes save automatically.

Finished this lesson?

Mark it complete to track your progress and schedule a future review.

Next: Reranking and Citations →