How LLMs Actually Work

What turns a next-token predictor into a useful assistant: pretraining and perplexity, scaling laws, sampling, fine-tuning and LoRA, RLHF and alignment, prompting and in-context learning, agents and tool use, and evaluation. Concepts made concrete with worked numbers and hands-on Lab widgets.

1. 01Pretraining and PerplexityThe whole of pretraining is one stubborn game: guess the next token, measure the surprise, nudge, repeat — a trillion times.7 sections
2. 02Scaling LawsBigger is not a hope, it is a curve. Scaling laws let you predict a model's loss before you spend a dollar training it.7 sections
3. 03SamplingThe model hands you a probability for every next token. Sampling is the art of choosing one — and temperature, top-k, and top-p are the three dials that decide.8 sections
4. 04Fine-Tuning and LoRAPretraining teaches a model language; fine-tuning teaches it a job — and LoRA does it without touching most of the weights.7 sections
5. 05RLHF and AlignmentFine-tuning teaches the model to imitate good answers; alignment teaches it which answer people actually prefer.7 sections
6. 06Prompting and In-Context LearningThe strangest trick in modern AI: a frozen model can learn a new task from a few examples typed into the prompt — no training at all.7 sections
7. 07Agents and Tool UseFrom a model that only predicts text to a model that can act: call tools, read the result, and decide the next step.7 sections
8. 08EvaluationHow do we know a model is good? Perplexity measures the language, benchmarks measure the skills, and every number hides a trap.7 sections