Chunking and Document Ingestion
Split documents into retrieval-sized pieces before they can be searched or embedded.
What you'll learn
- Explain why documents are split into chunks before embedding
- Implement fixed-size chunking with overlap
- Explain the tradeoff between chunk size and retrieval precision
Prerequisites
Explanation
You can't usefully embed an entire book as one vector — a single embedding has to represent one coherent chunk of meaning, and cramming an entire document into it would blur together dozens of unrelated ideas into a mushy average. So before embedding anything, real retrieval systems chunk documents into smaller pieces first, each becoming its own embedding.
There's a real tradeoff in choosing chunk size. Chunks too large dilute relevance — a chunk covering five different subtopics might rank as "somewhat relevant" to a query about any one of them, without being clearly the best match for any. Chunks too small lose context — a sentence fragment pulled out of its paragraph might be technically on-topic but useless without its surrounding explanation, and you multiply the number of vectors you have to search and pay to store.
A common practical approach is fixed-size chunking with overlap: split text into chunks of roughly N characters (or tokens), but let consecutive chunks overlap by some amount (e.g. 10-20%) so an idea that happens to fall right at a chunk boundary still appears fully within at least one chunk. More sophisticated approaches split along natural boundaries (paragraphs, headings, sentences) instead of a blind character count, which usually produces more coherent chunks at the cost of more implementation complexity.
Every chunk should also keep metadata: which source document it came from, its position, maybe a heading it fell under. This is what lets a RAG system later cite "paragraph 3 of the Refund Policy page" instead of just returning a floating, unattributed blob of text — citations are only possible if you tracked provenance all the way from ingestion.
Finally, ingestion should be idempotent: re-running it on an unchanged document shouldn't create duplicate chunks. Production systems typically compute a stable ID (often a hash of the chunk's content plus its source) so re-ingesting the same content is a no-op, and only genuinely changed chunks get re-embedded and replaced.
Example
Fixed-size chunking with overlap over a short piece of text.
function chunkText(text, size, overlap) {
const chunks = [];
let start = 0;
while (start < text.length) {
const end = Math.min(start + size, text.length);
chunks.push(text.slice(start, end));
if (end === text.length) break;
start += size - overlap;
}
return chunks;
}
const doc = "Retrieval systems split documents into chunks before embedding them for search.";
console.log(chunkText(doc, 30, 10));Try it yourself
Change the chunk size and overlap and see how the chunks change.
Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.
Guided exercise
Guided exercise
Complete `chunkBySize(text, size)` (no overlap yet) that splits text into an array of chunks each at most `size` characters long.
Checks: Splits a 10-character string into 4-character chunks correctly · plus 1 hidden check
Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.
Stuck? Get a hint.
Independent exercise
Independent exercise
Write `chunkWithOverlap(text, size, overlap)` that splits text into chunks of at most `size` characters, where each next chunk starts `size - overlap` characters after the previous chunk's start (matching the example lesson's algorithm), and stops once the final chunk reaches the end of the text.
Checks: Produces multiple overlapping chunks for longer text · First chunk starts at the beginning · plus 1 hidden check
Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.
Stuck? Get a hint.
Common mistakes
- Choosing a chunk size without testing retrieval quality — too large blurs relevance, too small loses context.
- Forgetting to store the source document/position metadata alongside each chunk, making citations impossible later.
- Re-ingesting an entire document from scratch on every run instead of only updating chunks that actually changed.
Knowledge check
Takeaway
Chunk size is a real tuning decision, overlap protects boundary-straddling ideas, and metadata is what makes citations possible later.
Summary
Documents are split into chunks before embedding, since one vector can't usefully represent an entire long document. Overlapping fixed-size chunking is a simple, common strategy, and every chunk should retain source metadata for citations and support idempotent re-ingestion.
References
Your notes
Notes save automatically.
Finished this lesson?
Mark it complete to track your progress and schedule a future review.
AI tutor
The optional AI tutor isn't enabled in this deployment. All lessons, exercises, quizzes, and search work fully without it.