Large Language Models
Transformer-based models trained on huge text corpora to generate and understand language.
CurrentintermediateFull course available
Overview
Large language models (LLMs) are transformer-based neural networks trained on vast text corpora, predicting the next token given prior context. That simple training objective, at sufficient scale, produces models capable of translation, summarization, coding help, and conversation.
- What it is
- Transformer-based neural networks trained to predict the next token in text, at large scale.
- Why it's used
- They generalize to an enormous range of language tasks from one training objective, without task-specific engineering for each use case.
- Where it fits
- The technology underneath most current generative AI products, including this platform's own optional AI tutor feature.
Core concepts
- Tokens and tokenization
- The transformer architecture (attention, intuition-level)
- Context windows
- Next-token prediction
Example
Tokenization (splitting text into sub-word units) is why LLMs sometimes struggle with exact letter-counting or spelling tasks -- they operate on tokens, not individual characters.
// An LLM doesn't see words -- it sees tokens.
// "unbelievable" might tokenize as ["un", "believ", "able"]
// The model predicts the next token, one at a time,
// each prediction conditioned on everything before it.Common use cases
- Conversational assistants
- Text summarization and drafting
- Code generation and explanation
Project ideas
- Manually tokenize a few sentences by guessing sub-word splits, then compare against how an LLM interface visualizes tokens