feat(extension): add orchestrator extension for intelligent agent rou…#2236
feat(extension): add orchestrator extension for intelligent agent rou…#2236pragya247 wants to merge 1 commit intogithub:mainfrom
Conversation
…ting Add a new Spec Kit extension that provides cross-catalog agent discovery and intelligent prompt-to-command routing. Commands: - speckit.orchestrator.route — Match user prompts to best-fit commands - speckit.orchestrator.index — Build unified capability index across all catalogs - speckit.orchestrator.discover — Scan linked repos for agent definitions Includes: - extension.yml manifest with after_init hook for auto-indexing - config-template.yml with matching strategy and cross-repo scan settings - README with usage examples and routing algorithm description Ref: github#2142
There was a problem hiding this comment.
Pull request overview
Adds a new built-in Spec Kit extension (“Intelligent Agent Orchestrator”) that scaffolds a discover → index → match → route workflow via extension manifest, config template, and command markdown instructions.
Changes:
- Introduces the
orchestratorextension manifest with commands + hook + default config. - Adds a config template for routing/discovery behavior.
- Adds three new command markdown files (
index,route,discover) plus extension README documentation.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| extensions/orchestrator/extension.yml | Defines the extension (commands, config template, hooks, tags, defaults). |
| extensions/orchestrator/config-template.yml | Provides user-editable routing + discovery configuration template. |
| extensions/orchestrator/commands/speckit.orchestrator.route.md | Documents prompt-to-command routing behavior and expected output. |
| extensions/orchestrator/commands/speckit.orchestrator.index.md | Documents building a unified capability index across sources. |
| extensions/orchestrator/commands/speckit.orchestrator.discover.md | Documents cross-repo scanning for agent/capability sources. |
| extensions/orchestrator/README.md | User-facing overview, install steps, configuration, and roadmap. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| hooks: | ||
| after_init: | ||
| command: speckit.orchestrator.index | ||
| optional: true | ||
| prompt: "Index available commands and agents for intelligent routing?" | ||
| description: "Auto-index capabilities after project initialization" | ||
|
|
There was a problem hiding this comment.
hooks.after_init is not a core-defined hook event, and there are no references in the repo to an after_init hook being executed. As a result, this hook will never run and the advertised auto-indexing after initialization won’t happen. Consider switching to an existing core hook event (see EXTENSION-API-REFERENCE.md “Hook Events”) or adding core support for an init lifecycle hook before relying on it here.
| hooks: | |
| after_init: | |
| command: speckit.orchestrator.index | |
| optional: true | |
| prompt: "Index available commands and agents for intelligent routing?" | |
| description: "Auto-index capabilities after project initialization" |
| - "SKILL.md" | ||
| - "AGENTS.md" | ||
| - "extension.yml" | ||
| - "workflow.yml" | ||
| - "preset.yml" |
There was a problem hiding this comment.
The default scan_patterns here don’t match the patterns documented in config-template.yml / speckit.orchestrator.discover.md (e.g., .claude/skills/*.md, .github/copilot-instructions.md, .specify/extensions/*/extension.yml, etc.). Since these defaults are used when the user hasn’t created a config file, this mismatch will lead to surprising discovery behavior. Align the defaults with the template/docs, or update the template/docs to match these defaults.
| - "SKILL.md" | |
| - "AGENTS.md" | |
| - "extension.yml" | |
| - "workflow.yml" | |
| - "preset.yml" | |
| - "AGENTS.md" | |
| - ".claude/skills/*.md" | |
| - ".github/copilot-instructions.md" | |
| - ".specify/extensions/*/extension.yml" | |
| - ".specify/workflows/*/workflow.yml" | |
| - ".specify/presets/*/preset.yml" |
| For each capability in the index, compute a relevance score: | ||
|
|
||
| 1. **Keyword match** (weight: 0.4) — Count how many of the capability's keywords appear in the user prompt | ||
| 2. **Description match** (weight: 0.3) — Check if words from the user prompt appear in the capability description | ||
| 3. **Name match** (weight: 0.2) — Check if the capability name is mentioned or closely related | ||
| 4. **Type bonus** (weight: 0.1) — Prefer workflows > extension commands > core commands for complex prompts; prefer core commands for simple ones | ||
|
|
||
| Normalize scores to a 0.0–1.0 range. | ||
|
|
||
| ## Step 3: Rank and Filter Results | ||
|
|
||
| 1. Sort capabilities by score (descending) |
There was a problem hiding this comment.
This command describes a multi-factor weighted scoring algorithm, but the extension config/template introduces matching_strategy with a keyword vs weighted choice. As written, the routing behavior is ambiguous because it doesn’t specify how to change scoring based on matching_strategy (and the default is keyword). Update the instructions to explicitly branch on the configured strategy (or remove the config option if it isn’t intended to change behavior).
| For each capability in the index, compute a relevance score: | |
| 1. **Keyword match** (weight: 0.4) — Count how many of the capability's keywords appear in the user prompt | |
| 2. **Description match** (weight: 0.3) — Check if words from the user prompt appear in the capability description | |
| 3. **Name match** (weight: 0.2) — Check if the capability name is mentioned or closely related | |
| 4. **Type bonus** (weight: 0.1) — Prefer workflows > extension commands > core commands for complex prompts; prefer core commands for simple ones | |
| Normalize scores to a 0.0–1.0 range. | |
| ## Step 3: Rank and Filter Results | |
| 1. Sort capabilities by score (descending) | |
| Read `matching_strategy` from `.specify/extensions/orchestrator/orchestrator-config.yml`. | |
| - If `matching_strategy` is missing, treat it as `keyword`. | |
| - If `matching_strategy` is `keyword`, use keyword-only matching. | |
| - If `matching_strategy` is `weighted`, use the multi-factor weighted scoring algorithm below. | |
| For each capability in the index, compute a relevance score according to the configured strategy: | |
| ### Strategy: `keyword` | |
| 1. Count how many of the capability's `keywords` appear in the user prompt | |
| 2. Compute the score using only keyword overlap | |
| 3. Normalize the keyword-only score to a 0.0–1.0 range | |
| Do **not** apply description, name, or type weighting when `matching_strategy` is `keyword`. | |
| ### Strategy: `weighted` | |
| 1. **Keyword match** (weight: 0.4) — Count how many of the capability's keywords appear in the user prompt | |
| 2. **Description match** (weight: 0.3) — Check if words from the user prompt appear in the capability description | |
| 3. **Name match** (weight: 0.2) — Check if the capability name is mentioned or closely related | |
| 4. **Type bonus** (weight: 0.1) — Prefer workflows > extension commands > core commands for complex prompts; prefer core commands for simple ones | |
| Normalize the final weighted score to a 0.0–1.0 range. | |
| ## Step 3: Rank and Filter Results | |
| 1. Sort capabilities by the score produced by the selected `matching_strategy` (descending) |
| | Command | Description | Keywords | | ||
| |---------|-------------|----------| | ||
| | `speckit.constitution` | Set up project constitution and rules | constitution, rules, setup, project | | ||
| | `speckit.specify` | Generate a specification from a feature description | spec, specification, feature, describe, requirements | | ||
| | `speckit.clarify` | Clarify and refine a specification | clarify, refine, questions, ambiguity | | ||
| | `speckit.plan` | Generate an implementation plan from a specification | plan, implementation, design, architecture | | ||
| | `speckit.tasks` | Break down a plan into actionable tasks | tasks, breakdown, issues, work items | | ||
| | `speckit.implement` | Implement code from tasks | implement, code, build, develop | | ||
| | `speckit.checklist` | Generate a review checklist | checklist, review, verify, quality | | ||
| | `speckit.analyze` | Analyze existing code or specifications | analyze, analysis, inspect, audit | | ||
|
|
There was a problem hiding this comment.
The core command list here omits speckit.taskstoissues, which is a bundled core command (templates/commands/taskstoissues.md exists and the extension system’s fallback core command set includes taskstoissues). Since this index is meant to be comprehensive, add it (and its keywords) so routing/index stats don’t undercount core capabilities.
There was a problem hiding this comment.
Pull request overview
Adds a new orchestrator Spec Kit extension scaffold intended to provide cross-catalog capability indexing, prompt-to-command routing, and cross-repo agent discovery via new extension commands.
Changes:
- Adds
extensions/orchestrator/extension.ymlmanifest with three newspeckit.orchestrator.*commands and a proposed init-time hook. - Adds orchestrator configuration template (
config-template.yml) and user-facing docs (README.md). - Adds three command instruction markdown files implementing the discover → index → route workflow as agent-executable specs.
Show a summary per file
| File | Description |
|---|---|
| extensions/orchestrator/extension.yml | Declares the new extension, its commands, config file, hook, tags, and default settings. |
| extensions/orchestrator/config-template.yml | Provides a user-editable config template for routing strategy and cross-repo scanning settings. |
| extensions/orchestrator/commands/speckit.orchestrator.route.md | Defines the agent instructions for scoring and routing user prompts to commands/workflows. |
| extensions/orchestrator/commands/speckit.orchestrator.index.md | Defines the agent instructions for building a unified JSON capability index. |
| extensions/orchestrator/commands/speckit.orchestrator.discover.md | Defines the agent instructions for scanning linked repos/submodules for agent/capability files. |
| extensions/orchestrator/README.md | Documents the extension’s purpose, usage, commands, and configuration. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (2)
extensions/orchestrator/extension.yml:61
- The manifest schema/documentation uses a top-level
defaults:key for default configuration values, notconfig: { defaults: ... }. As written, these defaults are likely to be ignored/misread by tooling and are inconsistent withextensions/template/extension.ymlandextensions/EXTENSION-API-REFERENCE.md. Move these values under a top-leveldefaults:key.
config:
defaults:
matching_strategy: "keyword"
confidence_threshold: 0.5
cross_repo_scan: true
scan_patterns:
- ".agent.md"
- "SKILL.md"
- "AGENTS.md"
- "extension.yml"
- "workflow.yml"
- "preset.yml"
extensions/orchestrator/commands/speckit.orchestrator.route.md:71
- This command describes a fixed weighted scoring algorithm, but the extension config exposes
matching_strategy: "keyword" | "weighted". As written, the config setting has no effect and the docs are internally inconsistent. Update the routing steps sokeyworduses only keyword matching (or document that weights are always applied) and ensure the implementation readsmatching_strategy.
## Step 2: Score Each Capability Against the Prompt
For each capability in the index, compute a relevance score:
1. **Keyword match** (weight: 0.4) — Count how many of the capability's keywords appear in the user prompt
2. **Description match** (weight: 0.3) — Check if words from the user prompt appear in the capability description
3. **Name match** (weight: 0.2) — Check if the capability name is mentioned or closely related
4. **Type bonus** (weight: 0.1) — Prefer workflows > extension commands > core commands for complex prompts; prefer core commands for simple ones
- Files reviewed: 6/6 changed files
- Comments generated: 3
| ```bash | ||
| # Install the extension | ||
| specify extension add orchestrator | ||
|
|
There was a problem hiding this comment.
The README suggests specify extension add orchestrator, but this extension is not present in the repo’s bundled catalog (extensions/catalog.json) or community catalog, so users won’t be able to install it via that command by default. Either add it to the appropriate catalog(s) or adjust the installation instructions (e.g., adding a catalog URL or installing from a local path/repo).
| # Linked repositories to include in cross-repo discovery | ||
| # Add paths (relative or absolute) to repos you want to scan | ||
| linked_repos: [] | ||
| # - path: "../frontend-app" | ||
| # name: "Frontend" | ||
| # - path: "../backend-api" | ||
| # name: "Backend API" |
There was a problem hiding this comment.
linked_repos: [] combined with the indented example list entries means that uncommenting the examples will produce invalid YAML (you can’t have both the flow sequence [] and a block sequence under the same key). Prefer a block list (e.g., linked_repos: followed by commented - path: entries) or move examples out of the key indentation.
| ```bash | ||
| index_file=".specify/extensions/orchestrator/index.json" | ||
|
|
||
| if [ ! -f "$index_file" ]; then | ||
| echo "⚠️ Index not found. Building index first..." | ||
| # Trigger index rebuild | ||
| fi | ||
|
|
||
| echo "📋 Loaded capability index" |
There was a problem hiding this comment.
When the index file is missing, this step only prints a message and doesn’t actually invoke /speckit.orchestrator.index (or otherwise build the index). Since the prerequisites require an index, add an explicit instruction to run the index command and wait for it to complete before continuing routing.
This issue also appears on line 63 of the same file.
|
Please create it as a community extension as per https://github.com/github/spec-kit/tree/main/extensions |
Description
Adds a new Spec Kit extension — Intelligent Agent Orchestrator — that provides cross-catalog agent discovery
and intelligent prompt-to-command routing.
Problem
Spec Kit has robust discovery (extension/preset/workflow catalogs) and dispatch (workflow engine with 10 step
types), but no intelligent matching layer that connects a user's natural-language intent to the right command.
Users need to already know what exists and where it lives.
Solution
Three new extension commands that implement a discover → index → match → route pipeline:
speckit.orchestrator.indexspeckit.orchestrator.routespeckit.orchestrator.discover.agent.md,SKILL.md,extension.yml, etc.) and merges into the indexDesign Decisions
after_inithook — auto-indexes capabilities on project initializationlinked_reposinorchestrator-config.ymlrouteworkflow step type (per discussion in Workflow Engine with Catalog System #2142)Files
extensions/orchestrator/ ├── extension.yml # Manifest ├── README.md # Usage docs ├── config-template.yml # Matching
strategy & scan settings └── commands/ ├── speckit.orchestrator.route.md # Prompt → command matching ├──
speckit.orchestrator.index.md # Capability indexer └── speckit.orchestrator.discover.md # Cross-repo agent scanner
Ref: #2142
Testing
extensions/template/extension.ymlandextensions/git/extension.ymlfor structural correctness
speckit.{extension-id}.{command-name}patternextension.ymlschema fields (schema_version, requires, provides, hooks, tags)uv run specify --helpuv sync && uv run pytestAI Disclosure
GitHub Copilot CLI (Claude) was used to:
All design decisions (routing algorithm, phased approach, integration points) were authored by me based on my
existing Agent Orchestrator implementations (Agency plugin + VS Code extension). I reviewed and understand all
generated content.