Build MultiSwitch: model-triggered, mid-sequence adapter invoke/return (revisit ADR-018 Pattern-1)

[AlonMalach]

Build MultiSwitch: model-triggered, mid-sequence adapter invoke/return (revisit ADR-018 Pattern-1)

Motivation

We want the model itself to trigger adapter invocation during generation — not just
respond to an external caller's adapter selection. ADR-022 records the decision to make
this work by fine-tuning control-token embeddings so the model actually emits the right
control tokens. But the interesting form of the capability — an adapter (or the base)
invoking another adapter mid-generation and control returning to the caller (call/return,
REQ-003) — needs more than one adapter per request. The shipped switch cannot do that
today. This issue parks and tracks that future work and the spec tension it creates.

Gaps

• ADR-018 (Accepted) restricts the shipped SingleSwitch to Pattern 1 — at most one
  adapter per request, no return-to-base mid-sequence, and explicitly defers arbitrary
  switching to "a future MultiSwitch." That MultiSwitch (CMP-009, status: planned)
  does not exist yet.
• REQ-003 (Function-call-style dynamic switching) is Not satisfied precisely because
  there is no call/return, multi-adapter switch.
• The emission side of ADR-022 (fine-tuned control-token embeddings) is un-validated:
  per INSIGHT-2, an untuned new control token is essentially unreachable during generation,
  so "the model decides to invoke an adapter" is currently unproven end-to-end.

Task overview

Build the MultiSwitch detection structure (mid-sequence, multi-adapter, return-to-caller)
and validate the fine-tuned-embedding emission path, so the model can trigger adapter
invocation with call/return semantics — then revisit ADR-018's Pattern-1 scope accordingly.
Backlog / future work; ADR-022 stays Proposed and CMP-009 planned until this lands.

What we solve

• Advances REQ-002 from a mechanism-only claim toward a real behavior guarantee (the
  model deciding to emit the right token).
• Moves REQ-003 toward Satisfied (call/return, function-stack invocation).
• Resolves the standing ADR-018 tension by defining when and how Pattern-1 is generalized.

Related spec objects

• REQ-002 — Model-driven adapter switching
• REQ-003 — Function-call-style dynamic switching
• ADR-018 — SingleSwitch is one attention op in one KV-cache slot (Pattern 1)
• ADR-022 — Model-triggered adapter invocation via fine-tuned control-token embeddings
• CMP-009 — MultiSwitch

Proposed approach (high level)

• Generalize the deterministic detector (CMP-003 SingleSwitch) to MultiSwitch:
  more than one adapter per request, and return-to-base / return-to-caller mid-sequence,
  keeping global KV-cache indexing coherent across both backends.
• Preserve ADR-016's zero-parameter deterministic detection — the fine-tuned-embeddings
  path (ADR-022) changes only what the model emits, not how the switch detects. A
  learned selector head is explicitly out of scope here.
• Validate the emission side: confirm fine-tuned control-token embeddings let the model
  reliably and correctly emit tokens (right token, avoid false positives, know when to stay
  on base).
• Revisit ADR-018's Pattern-1 scope once MultiSwitch exists — amend or supersede its
  scope rather than leaving the contradiction implicit.

Open questions & alternatives

• Open question: how does return-to-caller map onto the single-KV-cache-slot design of
  ADR-018 — extra cache slots, a stack encoded in control tokens, or something else?
• Open question: what test demonstrates REQ-003's nested call/return (the satisfied_when)?
• Alternative: a learned/embedding-based selector head (CLIP-style projection) instead
  of fine-tuned embeddings — rejected as the opposite bet from ADR-016's zero-parameter
  deterministic stance; would need its own ADR.
• Alternative: amend ADR-018 immediately rather than tracking separately — deferred; we
  chose to keep ADR-018 as-is and revisit it as part of this work.