Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .claude/board/EPIPHANIES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,25 @@
## 2026-07-11 — E-PALETTE-NNUE-SCALAR-FENCED-1 — MEASURED: scalar-256 palette of the NNUE FT weights is FENCED; the wide-corpus ρ trap (0.99 vs 0.78 near-equal) nearly greenlit a wrong "palette tenant" conclusion
**Status:** FINDING (MEASURED, `AdaWorldAPI/stockfish-rs` PR #5, 2026-07-11). Spatial-axis probe of the perturbation-cascade synthesis (`.claude/knowledge/stockfish-nnue-as-perturbation-cascade.md`, D-SF-V3-2 / D-PALETTE-NNUE — flagged there as "the single highest-value probe: is the frozen 90 MB net a palette tenant?"). Reads with the probe-first / truth-architect discipline — this is a worked example of a naive metric caught by an adversarial cut.

**What was measured:** a Lloyd/k-means **scalar-256** codebook fit over `ft.weights` (23 M i16, feature-major), reconstructed in place; Spearman ρ of palette-eval vs exact-eval over a 677-position corpus on `nn-1b6a82263149`.
- ρ_all (677 pos, wide spread) = **0.9934** — misleadingly high.
- ρ_quiet (146 pos, |eval| ≤ 200 cp — the fine-discrimination bar) = **0.7812**; eval MAE **100 cp**, max |Δ| 449 cp.

**The finding has two halves:**
1. **Scalar-256 palette is FENCED.** On near-equal positions (where an engine's discrimination lives) a scalar 256-level codebook does NOT preserve eval ranking. The FT carries fine magnitude a scalar palette destroys → it needs a **raw-magnitude lane**, not a scalar palette tenant. This is the doc's predicted NO branch, now measured — the scalar-palette rhyme is fenced.
2. **The wide-corpus ρ trap (methodological).** ρ_all = 0.99 is an ARTIFACT of material imbalance: gross material dominates ranking on a random-playout corpus, keeping ρ high cheaply. A single naive ρ would have greenlit "the net IS a palette tenant." The near-equal cut (|eval| ≤ 200 cp) is the honest bar and it fails. **Lesson for any palette/quantization/ρ-preservation probe: gate on the near-equal / quiet subset, never the full wide-spread corpus.**

**Consequence:** D-SF-V3-2 stays [H] (NOT promoted); the scalar branch is fenced. **D-PALETTE-NNUE-VEC queued:** vector palette256² / CAM-PQ (product-quantized 1024-dim subspaces, each a 256-centroid codebook) is the faithful test of the L4 tenant shape (`6×palette256:palette256` = pairs, not scalars) and exploits inter-dimension correlation a scalar codebook cannot — a strictly stronger representation, separate probe. Probe: `stockfish-rs examples/palette_nnue.rs` (net-gated, CI-safe, exits 0 — a measurement, not a gate). The synthesis doc remains SYNTHESIS-not-canon; this probe FENCES one row rather than promoting it.

## 2026-07-11 — E-SF-EPISODIC-1-GREEN-1 — MEASURED: a chess game is a byte-exact, replayable temporal version-stream (D-SF-EPISODIC-1 GREEN); position-at-ply-v is a zero-copy projection [H]→[G]
**Status:** FINDING (MEASURED, `AdaWorldAPI/stockfish-rs` PR #5, 2026-07-11). First measured probe of the temporal axis of the perturbation-cascade synthesis (`.claude/knowledge/stockfish-nnue-as-perturbation-cascade.md`). Reads with E-CHESS-TRANSCODE-COMPLETE-1 (the spatial L4 oracle this probe reuses) and E-MARKOV-TEMPORAL-STREAM-1 (the temporal-stream ruling it instantiates).

**The claim, now measured, not asserted:** the NNUE incremental update (`apply_move`) is BOTH the spatial L4 board delta AND one temporal Markov step — so a game is the accumulator's trajectory through version-space, and "position after ply *v*" is a **zero-copy projection** off that stream. The probe reused the L4 byte-exact oracle (`HalfKaAccumulator` incremental == full refresh) as a temporal-replay oracle over the Opera Game (33 plies; captures + queenside castling), on the pinned net `nn-1b6a82263149`:
- **GATE A (projection):** fresh `refresh(pos@v)` == the incrementally-maintained version-*v* accumulator, byte-for-byte — **34/34** plies. Reading version *v* recomputes from the ply-*v* FEN identity alone, no replay of the stream.
- **GATE B (order-independent replay):** an out-of-order / random-access read to ply *v* (independent incremental walk from the start) == the forward-walk version-*v* — **11/11** queries. The stream is replayable.

**Consequence:** the doc's "position-at-version is a zero-copy projection" row is promoted **[H]→[G]**. Honest scope: proven for `HalfKaAccumulator`, the incrementally-updated carrier (the exact analog of the spatial L4 delta); the full eval's threats term is refresh-only in current code and does not bear on the version-stream claim. Probe: `stockfish-rs examples/temporal_replay.rs` (net-gated, CI-safe; CI clippy leg extended to `--examples`). The synthesis doc remains SYNTHESIS-not-canon overall — this promotes ONE row; the basin (D-SF-BASIN-1), rung (D-SF-RUNG-1), and AriGraph (D-SF-ARIGRAPH-1) rows stay [H] pending their own probes.

## 2026-07-11 — E-ACK-SLA-GATE-ACTIONHANDLER-QUEUE-1 — naming refinement: the ack tier is the "SLA gate"; the ack mechanism is NOT retired — it is repurposed OGAR-consumer-side as the "actionhandler queue"
**Status:** RULING (operator, 2026-07-11) — naming refinement of E-ACK-HARD-GATE-VS-KANBANSTEP-STREAM-1 (below), which stands in substance; only the sanctioned names change. Append-only: the prior entry is not edited.

Expand Down
58 changes: 41 additions & 17 deletions .claude/knowledge/stockfish-nnue-as-perturbation-cascade.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,14 +101,29 @@ Stated as deliverables so it can be executed and falsified, never hand-waved:
stockfish-rs re-expresses its L3/L4 as the chess instantiation. **Gate:** the
existing L3/L4 byte-exact oracles STILL pass through the generic kernel (no
regression — the reference stays green).
- **D-SF-V3-2 — the NNUE accumulator AS a V3 SoA tenant.** Map `[[i16;1024];2]`
onto a V3 tenant lane (le-contract): is it an L4 palette pair carrier, or a new
raw-i16 lane? **Probe D-PALETTE-NNUE:** does palette256²-quantizing the FT
weight columns preserve the eval within the ρ anchor (Pflug-10 / Jirak floor)?
If YES → NNUE weights are a genuine L4 tenant (huge: 90 MB → palette-compressed
+ one-table-read similarity). If NO → NNUE needs a raw-magnitude lane and the
palette-tenant rhyme is fenced. *This is the single highest-value probe — it
tells us whether the frozen 90 MB net is a palette tenant or not.*
- **D-SF-V3-2 — the NNUE accumulator AS a V3 SoA tenant. ⚠ SCALAR-256 FENCED
(2026-07-11, MEASURED).** Probe D-PALETTE-NNUE asked: does palette-quantizing
the FT weights preserve the eval within the ρ anchor? Measured a Lloyd/k-means
**scalar-256** codebook over `ft.weights` (23 M i16, feature-major),
reconstructed in place, Spearman ρ of palette-eval vs exact-eval over a
677-position corpus on `nn-1b6a82263149`:
- ρ_all (wide-spread corpus) = **0.9934** — but this is an ARTIFACT of material
imbalance (gross material dominates ranking, so ρ stays high cheaply).
- ρ_quiet (146 near-equal positions, |eval| ≤ 200 cp — the fine-discrimination
bar where an engine actually lives) = **0.7812**; eval MAE **100 cp**.
**Verdict: the scalar-256 palette is FENCED** — on near-equal positions a scalar
256-level codebook does NOT preserve eval ranking; the FT carries fine magnitude
a scalar palette destroys, so it needs a **raw-magnitude lane**, not a scalar
palette tenant. This is the doc's predicted NO branch, now measured. **The
wide-corpus ρ trap is the finding's teeth:** a single naive ρ would have
greenlit "palette tenant" — the near-equal cut caught it. *Escalation queued —*
**D-PALETTE-NNUE-VEC:** vector palette256² / CAM-PQ (product-quantized 1024-dim
subspaces, each a 256-centroid codebook) exploits inter-dimension correlation a
scalar codebook cannot and is a strictly stronger representation; it is a
SEPARATE probe (the L4 tenant is `6×palette256:palette256` — pairs, not
scalars — so the vector probe is the faithful test of the tenant shape). Probe:
`stockfish-rs examples/palette_nnue.rs`. Honest scope: measured for the FT
weight matrix; the threat-weight lane (i8) is a further probe.
- **D-SF-V3-3 — make_index → HHTL/Morton route.** Probe D-MORTON-KA: re-project
HalfKA's king-bucket × piece-square addressing onto a Morton 2bit×2bit tile and
measure whether nearest-in-Morton ⇒ nearest-in-feature (the quorum τ). Confirms
Expand Down Expand Up @@ -159,7 +174,7 @@ two axes.
| Chess object | Temporal substrate | Grade |
|---|---|---|
| A game (ply sequence) | a `temporal.rs` sorted version-stream; ply *v* = one Lance version | **[G]** — moves already carry a total order; `apply_move` IS the step |
| "position after ply *v*" | `QueryReference::at(v, rung)` + deinterlace — a zero-copy projection, no replay | **[H]** — the read is [G]; that it needs no recomputation is the D-SF-EPISODIC-1 gate |
| "position after ply *v*" | `QueryReference::at(v, rung)` + deinterlace — a zero-copy projection, no replay | **[G]** (2026-07-11) — D-SF-EPISODIC-1 ran GREEN: the accumulator projected at ply *v* is byte-identical to the fresh-from-FEN computation (34/34), and to an out-of-order replay (11/11); see the deliverable below |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep projection [H] until QueryReference is exercised

Here the promotion is tied to gates that compare refresh(pos@v) with an incremental accumulator and an independent replay, but those checks do not exercise the documented Lance path (QueryReference::at(v, rung) + deinterlace) in this repo; crates/lance-graph-planner/src/temporal.rs only classifies stored rows by version and has no HalfKA/FEN projection path. Marking the row GREEN therefore records the zero-copy Lance projection as verified even though the measured path is a Stockfish recomputation/replay oracle, which can mislead follow-up work into skipping the actual projection test.

Useful? React with 👍 / 👎.

| Analysis horizon (present vs hindsight) | the **rung**: low rung = reason strictly at ply *v*; high rung = spoiler-read the game's outcome to label the move (training-target labeling) | **[H]** — rung semantics are shipped; the chess *use* (hindsight = eval-vs-result delta) is the D-SF-RUNG-1 probe |
| Opening variants (e4/d4/…) | **episodic basins** = le-contract **L1 `part_of:is_a`** rails (a variant is-a line is-a opening) | **[H]** — rails are [G]; that opening-trees fit the L1 rail shape is the D-SF-BASIN-1 probe |
| Transpositions (same position, different move-order) | **L2 `memberof:members`** — one position node, member-of many variant basins | **[H]** — the many-basins-one-node shape is exactly L2; the probe confirms it round-trips |
Expand All @@ -178,14 +193,23 @@ Games flow: `apply_move` → temporal-stream version → le-contract basin →

### Probe-gated deliverables (temporal)

- **D-SF-EPISODIC-1 — a game as a temporal version-stream.** Model an N-ply game
as N `temporal.rs` entries; read position-at-ply-*v* via `QueryReference::at`.
**Gate:** the projected accumulator at ply *v* == the accumulator freshly
computed from the ply-*v* FEN, byte-for-byte — *reusing the L4 chained oracle
already green in `incremental.rs`*. This is the **strongest temporal probe**:
it turns the existing incremental oracle into a temporal-replay oracle at zero
new ground-truth cost. If it passes, "position-at-version is a zero-copy
projection" is [H]→[G].
- **D-SF-EPISODIC-1 — a game as a temporal version-stream. ✅ GREEN (2026-07-11,
MEASURED).** Modelled the Opera Game (33 plies, captures + queenside castling)
as a `temporal.rs`-shaped version-stream over `HalfKaAccumulator` (the
incrementally-updated carrier — the temporal analog of the spatial L4 delta),
and measured two gates on the pinned net `nn-1b6a82263149`:
- **GATE A (projection):** fresh `refresh(pos@v)` == the incrementally-maintained
version-*v* accumulator, byte-for-byte — **34/34** plies. `QueryReference::at(v)`
reproduces from the version's own FEN identity, no recomputation of the stream.
- **GATE B (order-independent replay):** an out-of-order / random-access read to
ply *v* (independent incremental walk from the start) == the forward-walk
version-*v* — **11/11** queries. The stream is replayable.
Together these promote "position-at-version is a zero-copy projection" **[H]→[G]**.
Reused the L4 byte-exact oracle as a temporal-replay oracle at zero new
ground-truth cost, exactly as designed. Honest scope: proven for the HalfKA
incremental carrier; the full eval's threats term is refresh-only in current
code (does not affect the version-stream claim). Probe:
`stockfish-rs examples/temporal_replay.rs` (PR #5); net-gated, CI-safe.
- **D-SF-BASIN-1 — opening variants as L1 basins.** Encode a small opening tree
(e.g. 3 openings × 2 variants) as L1 `part_of:is_a` rails; assert a position
resolves to its basin set. **Gate:** transposition (same FEN via two move-orders)
Expand Down