Cover of From GPT-2 to Kimi K3, from first principles
The library

Foundation Models · Book 04 of 04

From GPT-2 to Kimi K3, from first principles

A first-principles tour from GPT-2 attention to Kimi K3's hybrid memory, sparse experts, depth retrieval, native vision, training, and serving systems.

For readers who know the standard Transformer and want a path into Kimi K3's architecture and systems design.

Reading time
118 minutes
Contents
21 chapters + self-test
Verified
August 4, 2026
Read online Download EPUB 1.2 MB Markdown source ↘

A guided route through the book

Inside the book.

  1. 01 Begin with GPT-2 attention
  2. 02 Build editable sequence memory
  3. 03 Keep global attention
  4. 04 Scale width and depth
  5. 05 Assemble the K3 backbone
  6. 06 Train and post-train the model
  7. 07 Serve the complete system

The complete learning path

In this series.

Book 03 · PreviousPractical Model Training You’ve reached the frontier.