Foundations Isle: fastbook ch.13–20 (CNNs, ResNets, optimizers, from-scratch Learner)

AGENTS.md

@@ -42,9 +42,12 @@ node scripts/gen-favicon.mjs  # regenerate PNG favicons from the design in the s
    Backward functions must match the *clamped/actual* forward (see `log`).
 7. **Concept-first naming, no chapter numbers in-world.** Building names,
    district names, panel titles, and blurbs describe concepts ("Loss
-   District", "First Steps Quarter", "Sequence Summit"), never book
-   chapters. The book is how the *curriculum* is organized, not how the
-   city is organized: the fastai chapter mapping lives only in README.md.
+   District", "First Steps Quarter", "Sequence Summit", "Residual Ridge"),
+   never book chapters. The book is how the *curriculum* is organized, not
+   how the city is organized: the fastai chapter mapping lives only in
+   README.md. The world now covers chapters 4–20: old town (ch.4–6), the
+   Frontier across the river (ch.7–12), and the Foundations Isle across the
+   sea, reached by ferry (ch.13–20).
 8. **The book is the source material — read it, don't guess.** A full copy
    of fast.ai's *Deep Learning for Coders* notebooks is vendored at
    `reference/fastbook-master.zip`. Before building or changing any feature
@@ -57,6 +60,14 @@ node scripts/gen-favicon.mjs  # regenerate PNG favicons from the design in the s
    `DotProductBias` with `sigmoid_range` and weight decay, label smoothing
    uses the book's (1−ε+ε/N) targets. Simplifications are fine, but they
    must be acknowledged in the panel copy, never silent.
+9. **Conv feature maps are `[batch, channels*height*width]`.** The engine's
+   tensors are 1D/2D, so the Foundations Isle keeps convolutional activations
+   in a 2D tensor with the channel/height/width geometry passed explicitly to
+   the ops in `src/engine/conv.ts` (`conv2d`, `avgPoolAll`, `batchNorm`).
+   Every new engine op still needs a finite-difference test (principle 6);
+   the conv/pool/bn/optimizer ops are covered in `tests/engine.test.ts`.
+   Batchnorm behaves differently in training vs eval (batch stats vs running
+   averages) — pass the right `training` flag, exactly as the book describes.
 
 ## UI conventions
 
@@ -85,11 +96,14 @@ node scripts/gen-favicon.mjs  # regenerate PNG favicons from the design in the s
 
 - Tiles are 32px; the map is drawn procedurally in `src/world/city.ts` —
   no image assets anywhere.
-- Buildings carry a `tour` number (1–21) shown as a gold badge on their
-  sign; the route is also painted as road chevrons. If you add a building,
-  give it a tour stop and keep the route geographically sensible:
+- Buildings carry a `tour` number (1–29) shown as a gold badge on their
+  sign; the Horizon Beacon is the final stop (㉚, a monument, not a
+  building). The route is also painted as road chevrons. If you add a
+  building, give it a tour stop and keep the route geographically sensible:
   data → forward → loss → backward → step → metrics, then extensions,
-  then south across the river into the Frontier (non-image data).
+  then south across the river into the Frontier (non-image data), then by
+  ferry across the sea to the Foundations Isle (convolutions → residual →
+  optimizers/foundations → the Learner → the beacon).
 - District names are lawn-sign placards (`DISTRICT_SIGNS` in
   `src/world/city.ts`): bright text on a dark board, planted on the grass
   strip *above* their building group. `generate()` keeps trees/flowers out
@@ -99,7 +113,13 @@ node scripts/gen-favicon.mjs  # regenerate PNG favicons from the design in the s
   central sign).
 - Tour chevrons must form a continuous, bounded route: they keep going
   past the last stop of a row to the junction that leads onward, and they
-  STOP at the final stop (Echo Tower) — no arrows pointing at nothing.
+  STOP at the final stop (the Horizon Beacon on the Foundations Isle) — no
+  arrows pointing at nothing.
+- The sea (south of the Frontier) has no bridge: the only crossing is the
+  ferry. `City.ferryAt(tx,ty)` reports which pier the avatar is on;
+  `Game.startFerry(dest)` runs the scripted boat glide (reusing the
+  `travel` mechanism with `mode:"boat"`, which draws the hull). The island
+  is a full second landmass below `y≈74`; `MAP_H` was extended to fit it.
 - The plaza holds the network monument (animated while `main` trains) and
   the tour kiosk: standing next to the kiosk and pressing E rides the
   "express" (a scripted glide, `Game.startTourExpress`) to stop ①. The

README.md

@@ -8,10 +8,14 @@ buildings, and inspect what is actually happening during the forward pass,
 the backward pass, and inference: down to the individual multiply-adds of a
 single matmul cell. South of the river, **the Frontier** leaves images
 behind: training tricks, movie-taste embeddings, decision-tree forests,
-text pipelines and a tiny language model — all running live too.
+text pipelines and a tiny language model — all running live too. And across
+the sea, reached by ferry, the **Foundations Isle** goes under the hood:
+a convolutional net learning its own kernels, residual networks racing a
+skipless twin, an optimizer derby, class-activation lighthouse beams, and
+the training loop itself rebuilt from scratch — every piece training live.
 
 Built around the concepts of **fast.ai's _Deep Learning for Coders_
-([fastbook](https://github.com/fastai/fastbook)), chapters 4–12**. A full
+([fastbook](https://github.com/fastai/fastbook)), chapters 4–20**. A full
 copy of the book's notebooks is vendored at `reference/fastbook-master.zip`
 so features can always be checked against the source material (see
 AGENTS.md). In-world, everything is named after *concepts*, never after
@@ -31,8 +35,10 @@ committed; regenerate it with `npm run fetch-mnist`.
 **Controls:** WASD / arrows to walk · **E** to enter buildings and inspect
 machines · **Esc** to go back · Shift to run. On touch devices an on-screen
 pad appears (with a run/walk toggle). Press **▶ train** in the top bar to
-set the whole city in motion, and follow the numbered signs ① → ㉑ for the
-guided tour through the training loop and on across the river.
+set the whole city in motion, and follow the numbered signs ① → ㉚ for the
+guided tour through the training loop, across the river into the Frontier,
+and by ferry to the Foundations Isle. Standing on a ferry pier (center of
+the map, at the water's edge) and pressing **E** takes the boat across.
 
 ## The city → curriculum map
 
@@ -51,12 +57,22 @@ guided tour through the training loop and on across the river.
 | **Table Grove** (ch. 9) | Decision Arboretum | tabular modeling without gradients: CART regression trees grown by variance reduction (the candidate-question audition shown with real numbers), a walkable grown tree, bagging into a random forest, out-of-bag error and split-gain feature importance |
 | **Language Lane** (ch. 10–11) | Tokenizer Mill, Sentiment Studio | the text pipeline as mid-level plumbing: tokenization with special tokens (xxbos/xxunk), train-only vocab with a min-freq cutoff, numericalization into tensor rows — then a live bag-of-words classifier with one inspectable weight per word and weight decay |
 | **Sequence Summit** (ch. 12) | Echo Tower | a language model from scratch on human numbers: the book's `LMModel2` (shared hidden state, one reused weight matrix — an unrolled RNN), quizzed on the held-out tail of the count, compared against the most-common-token baseline, and generating text greedily or by temperature sampling |
+| **Kernel Shore** (ch. 13) | Kernel Workshop, Convolution Works | what a convolution *is* — a hand-made 3×3 kernel sliding over a real digit, the output-size arithmetic of stride & padding, and the conv-as-shared-weight-matmul view; then a live CNN (the book's `simple_cnn`) learning its own kernels, with a feature-map floor, receptive-field tracing, and the 'colorful dimension' stability plot with a batchnorm switch |
+| **Residual Ridge** + **Saliency Point** (ch. 14, 18) | Residual Works, Saliency Lighthouse | a residual network (blocks that start as exact identities via zero-init, computing `x + f(x)`) racing a skipless twin on identical batches, adaptive pooling that accepts any input size, and the compute-vs-parameters breakdown; plus class activation maps (CAM) and Grad-CAM shining a light back through the live model to show which pixels decided the prediction |
+| **Assembly Row** + **Momentum Row** (ch. 15, 16) | Architecture Atelier, Optimizer Institute | cutting a trained body and sewing on a new head to build a siamese "same digit?" matcher (with freeze / per-group learning rates), plus the upsampling mechanisms behind segmentation; and a four-way optimizer race — SGD vs momentum vs RMSProp vs Adam on identical nets — with a momentum moving-average desk and an Adam microscope |
+| **Bedrock Quarter** + **Loop Landing** (ch. 17, 19) | Foundations Quarry, Learner Hall | a neural net from the foundations: matmul three ways (and why plain Python is hopeless), broadcasting rules, weight init that decides whether training even starts, and backprop derived by hand and checked against autograd; then the training loop rebuilt as a machine of named events and callbacks, including a `TerminateOnNaN` guard you can break and watch fire |
+| **the Horizon Beacon** (ch. 20) | — | the tour's final stop: a look back over every district and the concept it taught, and a look out at the open water where the map ends and your own projects begin |
 
 The Frontier's non-image datasets (ratings, reviews, rents, the counting
 corpus) are generated deterministically in code with *planted* structure
 plus noise — the models never see the generators and must rediscover the
 structure live, which is what lets panels compare "planted truth" with
-"learned factors" honestly.
+"learned factors" honestly. The Foundations Isle goes back to the MNIST
+digits, but deeper: its convolutional and residual nets, the optimizer
+race, the siamese matcher, and the from-scratch Learner all train live on
+the same digits, and the siamese encoder can be *cut* straight from the
+Convolution Works' trained body — real transfer learning between two
+buildings of the same city.
 
 ## How it works
 
@@ -65,16 +81,22 @@ structure live, which is what lets panels compare "planted truth" with
   softmax/CE/BCE/MSE built **from primitives** so the recorded graph
   contains every station you walk through). Verified against numeric
   differentiation in `tests/engine.test.ts`.
-- **`src/sim/`** — eight live training scenarios (10-class MLP, linear 3v7,
-  multi-label, regression, the refinery twins, collaborative filtering,
-  bag-of-words sentiment, the RNN language model), trainers, the LR finder,
-  the pixel baseline, plus gradient-free CART trees/forests and the
+- **`src/sim/`** — thirteen live training scenarios (10-class MLP, linear
+  3v7, multi-label, regression, the refinery twins, collaborative filtering,
+  bag-of-words sentiment, the RNN language model, a convolutional net, a
+  residual-vs-plain twin race, a four-optimizer derby, a siamese pair-matcher,
+  and a from-scratch Learner with a callback system), trainers, the LR
+  finder, the pixel baseline, plus gradient-free CART trees/forests and the
   Frontier's generated datasets. After each SGD step a fresh recorded
   forward/backward pass keeps every panel's values and gradients mutually
   consistent.
+- **`src/engine/conv.ts`** — convolution, adaptive average pooling and batch
+  normalization ops (with backward passes verified by finite differences),
+  plus `src/engine/optim.ts`'s Momentum / RMSProp / Adam / grouped-SGD, all
+  built to the book's formulas.
 - **`src/world/`** — the procedurally-drawn city, avatar, interiors
   (Canvas 2D, no game engine, no assets).
-- **`src/ui/`** — the HUD and 55 inspection panels (DOM overlays with live
+- **`src/ui/`** — the HUD and ~80 inspection panels (DOM overlays with live
   heatmaps, charts and element-level expansions).
 
 No backend, no GPU, no dependencies beyond Vite/TypeScript/Vitest.