VisaSpark
Sign in
intermediate24 min

Semantic, Keyword, and Hybrid Search

Compare meaning-based search, exact-word search, and combining the two.

What you'll learn

  • Contrast keyword search with semantic (embedding-based) search
  • Rank a small set of chunks by similarity to a query
  • Explain when hybrid search outperforms either approach alone

Prerequisites

Explanation

Keyword search (also called lexical search) matches literal words — it finds documents containing "refund" when you search "refund," using techniques like inverted indexes and ranking algorithms such as BM25. It's fast, precise for exact terms, codes, and names, and doesn't require any AI model — but it completely misses a query like "get my money back" against a document that only ever says "refund," since the words don't literally match.

Semantic search uses embeddings and cosine similarity (from the previous lesson) to match meaning instead of exact words — "get my money back" and "refund policy" can be close in embedding space even though they share zero words. This is exactly what makes it powerful for natural-language questions. But it has its own weaknesses: it can struggle with exact identifiers (an order number, a product SKU, an error code) where you need a literal match, not a semantic approximation, and it can occasionally surface a topically-similar-but-wrong result.

Hybrid search runs both approaches and combines their results — often by computing both a keyword relevance score and a semantic similarity score for each candidate, then merging or re-ranking based on both. This tends to outperform either technique alone: keyword search anchors exact terms and identifiers, semantic search catches paraphrases and conceptual matches, and combining them covers more of the ways a real user might phrase a question.

For this beta's exercises, "keyword search" is modeled simply as counting shared words between a query and a chunk, and "semantic search" reuses cosine similarity over small hand-picked vectors from the previous lesson — real production hybrid search uses proper inverted indexes (like PostgreSQL full-text search or Elasticsearch/BM25) and real embedding models, merged with a tuned weighting or reranking step (covered in the next lesson).

Example

A simplified keyword-overlap score next to a semantic (vector) similarity score for the same query/chunk pair.

function keywordScore(query, text) {
  const queryWords = new Set(query.toLowerCase().split(/\s+/));
  const textWords = text.toLowerCase().split(/\s+/);
  const matches = textWords.filter((w) => queryWords.has(w)).length;
  return matches / queryWords.size;
}

function cosineSimilarity(a, b) {
  const dot = a.reduce((s, v, i) => s + v * b[i], 0);
  const magA = Math.sqrt(a.reduce((s, v) => s + v * v, 0));
  const magB = Math.sqrt(b.reduce((s, v) => s + v * v, 0));
  return dot / (magA * magB);
}

const query = "get my money back";
const chunkText = "our refund policy allows returns within 30 days";
const queryVector = [0.1, 0.9]; // stands in for a real embedding
const chunkVector = [0.15, 0.85]; // stands in for a real embedding

console.log("keyword score:", keywordScore(query, chunkText));
console.log("semantic score:", cosineSimilarity(queryVector, chunkVector));

Try it yourself

Try a query that shares no words with the chunk text but has a high semantic score.

Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.

Loading editor…

Guided exercise

Guided exercise

Complete `keywordOverlapCount(query, text)`, returning how many of the query's whitespace-separated words also appear (case-insensitively) somewhere in text.

Checks: Counts matching words correctly · plus 1 hidden check

Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.

Loading editor…

Stuck? Get a hint.

Independent exercise

Independent exercise

Write `hybridRank(queryVector, queryWords, candidates)` where each candidate is `{ text, vector }`. Compute a combined score per candidate as `0.5 * cosineSimilarity(queryVector, candidate.vector) + 0.5 * (keyword overlap count / queryWords.length)`, and return the candidates sorted by combined score, highest first (array of the original candidate objects).

Checks: Ranks the best combined match first · plus 1 hidden check

Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.

Loading editor…

Stuck? Get a hint.

Common mistakes

  • Assuming semantic search is strictly 'better' than keyword search — it's worse for exact identifiers and codes.
  • Forgetting that identical wording is not required for high semantic similarity, which is the whole point of using embeddings.
  • Weighting keyword and semantic scores arbitrarily without ever measuring which weighting actually improves results on real queries.

Knowledge check

Knowledge check

1. Which search approach would best find a document containing an exact order number like 'ORD-88213'?
2. Why can semantic search find a relevant document even with zero shared words?
3. What is the main benefit of hybrid search over using only one approach?

Takeaway

Keyword search nails exact terms, semantic search understands paraphrasing, and hybrid search plays both to their strengths.

Summary

Keyword search matches literal words and excels at exact identifiers; semantic search matches meaning via embeddings and excels at paraphrased natural-language queries. Hybrid search combines both signals, typically outperforming either alone.

References

Your notes

Notes save automatically.

Finished this lesson?

Mark it complete to track your progress and schedule a future review.

Next: Retrieval-Augmented Generation: The Full Pipeline →