Series 01 · Building Foundation Models
Foundation Models,
from code to frontier.
Four agent-created field guides for understanding, training, and extending the models underneath modern AI.
Series 01 · August 4, 2026
Read it as a path.
The sequence moves from first principles to a small implementation you can hold in your head, through the practice of training, and finally to the architecture of a frontier model.
- 01 Build the intuition
Inside a Transformer, slowly
An illustrated, math-and-code-friendly guide from raw web pages to token batches, attention, residual streams, training runs, inference, and Kimi K3.
- 02 Start with the code
Inside nanoGPT, line by line
A source-pinned, tensor-shape-driven walkthrough of Andrej Karpathy's nanoGPT, from LayerNorm and causal attention through training, checkpoints, and generation.
- 03 Train what matters
Practical Model Training for ML Practitioners
A rigorous field guide to choosing, adapting, evaluating, and shipping language and multimodal foundation models.
- 04 Follow the architecture
From GPT-2 to Kimi K3, from first principles
A first-principles tour from GPT-2 attention to Kimi K3's hybrid memory, sparse experts, depth retrieval, native vision, training, and serving systems.
How these editions are made
Built by agents.
Kept accountable.
Each book begins as a learning question, then moves through source research, technical review, revision, illustration, and a final publication proof.
The Markdown is the editable source. The EPUB is the validated reading edition. Verification dates and primary references travel with both.
Start with book one →