Skip to content

REQ-002 "Satisfied" overstates model-autonomous invocation — criterion verifies only the plumbing #105

Description

@AlonMalach

Motivation

The spec should not read as if a capability works when it doesn't. REQ-002 (a MUST / MVP
requirement) is marked Satisfied, implying the model can autonomously decide to switch
adapters. In reality only the mechanism is verified — a model-emitted control token takes
effect. Whether the model has learned when to invoke an adapter is unsolved. ADR-022 (Proposed)
now formally targets that gap, so REQ-002's status needs reconciling to keep the spec honest.

Gaps

  • REQ-002's satisfied_when verifies only that a model-emitted control token changes the active
    adapter (plumbing) — not that the model decides appropriately when to emit it.
  • No requirement or shipped mechanism covers a learned invocation policy bounded by precision /
    low false-invocation (not over-invoking the base).
  • INSIGHT-0001 records the trap directly: "freely generatable ≠ learned-to-emit." A reader taking
    REQ-002 at face value would conclude autonomous invocation is delivered.

Task overview

Decide how to reconcile REQ-002's status/criterion with the fact that model-autonomous invocation
(the learned when-to-invoke policy) is not yet delivered, now that ADR-022 records that goal.

What we solve

The spec accurately reflects reality: the plumbing is done; the learned invocation policy is open
work. Downstream planning (and INSIGHT-0001's warning) stop being contradicted by a Satisfied MUST.

Related requirements / ADRs

  • REQ-002 — Model-driven adapter switching (Satisfied; status in question)
  • REQ-003 — Function-call-style dynamic switching (Not satisfied; adjacent call/return goal)
  • ADR-022 — Learned when-to-invoke policy for model-autonomous adapter invocation (Proposed)
  • INSIGHT-0001 — Distinguish externally-driven adapter selection from model-autonomous invocation
  • BG-01 — Capabilities compose like software (goal REQ-002 supports)

Proposed approach (high level)

Reconcile REQ-002 via one of these (to be decided, not pre-judged):

  • Tighten REQ-002's satisfied_when to add a learned-policy criterion (precision / false-
    invocation bound, shown by an eval), flipping it Satisfied → Not satisfied until ADR-022's
    follow-up delivers a mechanism.
  • Split concerns: keep REQ-002 as the (Satisfied) plumbing requirement and add a new
    requirement for the learned invocation policy that ADR-022's mechanism would serve.
  • Leave as-is and rely on ADR-022's consequences note to record the gap (lowest effort; keeps
    the overstatement risk).

Open questions & alternatives

  • Open question: Is "model-driven switching" meant to mean the plumbing (current criterion) or the
    learned policy? The answer picks between tightening vs splitting.
  • Open question: What precision / false-invocation bound and eval would define "satisfied" for the
    policy?
  • Alternative trade-off: tightening reopens a MUST/MVP requirement (cleaner spec, but a MUST goes
    red); splitting preserves REQ-002's history at the cost of a new REQ id; leaving as-is is cheapest
    but perpetuates the overstatement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions