Sub-project 1 — Framework + Tic-Tac-Toe
Generic AlphaZero framework (Game protocol, MCTS, training loop) with Tic-Tac-Toe as the first target. Reached 0 losses vs a perfect minimax solver by iteration 10.
Learning project · 2026
AlphaZero from Scratch
Building AlphaZero ground-up in Python — MCTS, neural network policy/value heads, self-play pipelines, and progressively harder games. Every design decision is documented; every sub-project ships working code.
→ Start with the visual primerLearn AlphaZero
A visual primer, a foundational text reference, and a self-test quiz.
Primer · visual
Five concepts at the heart of it
Animated visual notes on MCTS, ResNet skip connections, PUCT, replay buffer, arena. Best for the first read or a quick refresher.
Read primer →Reference · text
Foundational concepts
Detailed text reference for every term: ply, logits, BatchNorm, ResNet forward/backward, MCTS phases, Dirichlet noise, temperature, π and z, symmetries, glossary. Includes a code-tracing path.
Read reference →Quiz · interactive
Self-assessment quiz
38 multiple-choice questions across basics, MCTS, self-play, training, architecture, engineering, and pre-training. Click an answer for the explanation; topic breakdown + retry at the end.
Take quiz →Build log
Sub-projects in chronological order. Each ships working code with a spec and an implementation plan.
Sub-project 2 — Connect 4
Scaled the framework to a 6×7 board with gravity, column-based moves, and horizontal symmetry. Used α-β minimax-depth-8 as the eval baseline. The framework code (MCTS, network, trainer) was reused unchanged — only the Game subclass changed.
Sub-project 3 — Chess
Full 8×8 chess: AlphaZero's 8×8×73 = 4672 move encoding (queen-like + knight + underpromotion), 20-plane input, StockfishOpponent via UCI as the eval baseline, and parallel self-play with a central NN-server batching inference across worker processes.
Sub-project 3b — Supervised Pre-training + AZ Refinement
Cold-start AlphaZero on chess is wall-clock infeasible on consumer hardware (~13 days on M4 Pro). Bootstrap the network from 50k Stockfish-vs-Stockfish games via supervised pre-training, then refine with a short AlphaZero loop. ~15 hours total. New code: corpus generator, supervised trainer, pretrain CLI subcommand — framework primitives reused unchanged.
Sub-project 4 — Productionize
UCI engine wrapper for lichess-bot and Arena, scaling to 2000+ ELO, distributed self-play across cloud workers, and experiment tracking.
Sub-project 5 — Hybrid Search
Combining learned heuristics with classical search (alpha-beta, endgame tablebases). Distillation from larger teacher networks.