Skip to content

fix: improve issue-triage prompt to reliably trigger MCP tool calls#10506

Draft
Copilot wants to merge 2 commits intomainfrom
copilot/debug-agentic-triage-failure
Draft

fix: improve issue-triage prompt to reliably trigger MCP tool calls#10506
Copilot wants to merge 2 commits intomainfrom
copilot/debug-agentic-triage-failure

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 27, 2026

Third failure of the Agentic Triage workflow — MCP gateway logs show the agent listed tools but made zero tools/call requests across 6 minutes and 31 LLM API calls. The agent was reasoning without acting.

Changes to .github/workflows/issue-triage.md

  • Remove web-fetch from tools config — declared in frontmatter but never available through the MCP gateway (only github and safeoutputs servers register), creating noise in the prompt context
  • Front-load safe output requirement — moved from line ~148 (end of prompt, never reached) to a prominent blockquote immediately after the task description
  • Add explicit "call the tool right away" directive — breaks the pure-reasoning loop by instructing immediate issue_read invocation with specific parameters (owner, repo, issue_number)
  • Move add_labels to step 3 — produces a safe output early, before optional complex steps (duplicate detection, playground links)
  • Mark context gathering and duplicate detection as optional — agent won't block on complex analysis before producing output
  • Reduce prompt from ~150 to ~93 lines — removed verbose sub-instructions for reproduction extraction, code block parsing, and comment formatting that contributed to the agent looping without acting

Copilot AI linked an issue Apr 27, 2026 that may be closed by this pull request
The agent was listing MCP tools but never calling any of them during
its 6-minute execution (zero tools/call requests in MCP gateway logs).

Three root causes addressed:

- Remove `web-fetch` from tools config: declared but never available
  through the MCP gateway, creating noise in the prompt context
- Front-load safe output requirement: moved from end of prompt to
  immediately after task description as a prominent blockquote, so the
  agent sees it before any instructions
- Restructure prompt for immediate tool invocation: added explicit
  "call the tool right away" directive, included specific parameter
  names for each tool call, moved label application (safe output) to
  step 3 before optional complex steps, and made context gathering
  and duplicate detection explicitly optional
- Reduce prompt complexity: from ~150 lines to ~93 lines, removing
  verbose sub-instructions that caused the agent to reason without
  acting

Agent-Logs-Url: https://github.com/microsoft/typespec/sessions/1ba3cf16-5565-498e-8c9b-5d3df159087d

Co-authored-by: timotheeguerin <1031227+timotheeguerin@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix Agentic Triage workflow failure to generate safe outputs fix: improve issue-triage prompt to reliably trigger MCP tool calls Apr 27, 2026
Copilot AI requested a review from timotheeguerin April 27, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Agentic Triage failed

2 participants