From b1cdb3b5b4428f01d055aa0c28c0547abec67c03 Mon Sep 17 00:00:00 2001 From: Tomasz Janiszewski Date: Thu, 7 May 2026 11:40:51 +0200 Subject: [PATCH 1/2] Add stackrox/skills repo and comment on all issues with --comment flag MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two key improvements to the triage workflow: 1. Clone stackrox/skills repository during setup phase - Added to Phase 1a setup alongside stackrox/stackrox - Makes reusable skills available (rhacs-patch-eval, etc.) - Updated reference/constants.md with skills repo path 2. Comment on ALL issues when --comment flag is used - Previously: only commented on issues with confidence ≥70-80% - Now: comments on all triaged issues regardless of confidence - Low confidence (<80%) issues get ⚠️ warning for manual review - Ensures all triaged issues receive analysis feedback Updated files: - .claude/commands/triage.md: Phase 1a and Phase 7 updated - .claude/commands/comment-issues.md: Removed confidence filtering - CLAUDE.md: Updated command descriptions - README.md: Updated workflow phases - reference/constants.md: Added skills repository path This ensures teams see triage analysis on all issues, not just high-confidence ones, improving visibility of the automated process. Co-Authored-By: Claude Sonnet 4.5 --- .../.claude/commands/comment-issues.md | 7 ++-- .../acs-triage/.claude/commands/triage.md | 38 ++++++++++++++----- workflows/acs-triage/CLAUDE.md | 4 +- workflows/acs-triage/README.md | 10 ++--- workflows/acs-triage/reference/constants.md | 10 +++-- 5 files changed, 44 insertions(+), 25 deletions(-) diff --git a/workflows/acs-triage/.claude/commands/comment-issues.md b/workflows/acs-triage/.claude/commands/comment-issues.md index a8bdc3e..ce8553c 100644 --- a/workflows/acs-triage/.claude/commands/comment-issues.md +++ b/workflows/acs-triage/.claude/commands/comment-issues.md @@ -56,13 +56,12 @@ Add triage analysis comments to JIRA issues based on team assignment results. Po - Total issues processed - Successfully commented - Failures (with reasons) - - Skipped (low confidence <70%) ## Safety Guardrails -**Confidence Filter:** -- Only comment on issues with confidence ≥70% -- Flag low-confidence issues for manual review +**No Confidence Filtering:** +- Comments ALL triaged issues regardless of confidence level +- Low-confidence issues (<80%) include ⚠️ warning: "Low confidence - requires manual review" **Dry Run Mode:** - Default: dry_run = true diff --git a/workflows/acs-triage/.claude/commands/triage.md b/workflows/acs-triage/.claude/commands/triage.md index 02ac708..2297711 100644 --- a/workflows/acs-triage/.claude/commands/triage.md +++ b/workflows/acs-triage/.claude/commands/triage.md @@ -6,9 +6,11 @@ Complete end-to-end triage workflow for StackRox/ACS JIRA issues. Fetches untria **Options:** - `/triage` - Full triage pipeline using JQL search (READ-ONLY, no JIRA writes) -- `/triage --comment` - Full triage + post comments to JIRA + add auto-triaged label -- `/triage ROX-12345` - Triage a specific issue by key -- `/triage ROX-12345 --comment` - Triage specific issue, post comment, and add label +- `/triage --comment` - Full triage + post comments to ALL issues (regardless of confidence) + add auto-triaged label +- `/triage ROX-12345` - Triage a specific issue by key (READ-ONLY) +- `/triage ROX-12345 --comment` - Triage specific issue + post comment + add label + +**Behavior with --comment:** Posts triage analysis comments to ALL triaged issues regardless of confidence level. Low confidence (<80%) issues include a warning flag for manual review. ## Prerequisites @@ -26,7 +28,7 @@ This command executes the following phases: **PERFORMANCE OPTIMIZATION:** These phases have no interdependencies and SHOULD run concurrently to save 10-20 seconds. #### Phase 1a: Setup (if needed) - Async -Clone StackRox repository for CODEOWNERS and reference data if not already present. +Clone required repositories for CODEOWNERS, reference data, and skills if not already present. **Actions:** - Check if `/tmp/triage/stackrox/.github/CODEOWNERS` exists @@ -35,6 +37,11 @@ Clone StackRox repository for CODEOWNERS and reference data if not already prese - Check if `/tmp/triage/stackrox/.claude/agents/stackrox-ci-failure-investigator.md` exists - If present: deep CI failure analysis will use this agent's methodology - If missing: log warning "CI failure investigator agent not found - deep analysis will use description-only mode" +- Check if `/tmp/triage/skills/.claude/skills/` exists +- If missing, clone `https://github.com/stackrox/skills` to `/tmp/triage/skills` +- Skills available for use: + - `/tmp/triage/skills/.claude/skills/rhacs-patch-eval/` - Patch evaluation and VEX analysis + - Any other skills in the repository can be loaded on-demand **Output:** Setup metadata in `artifacts/acs-triage/setup-info.json` @@ -249,10 +256,10 @@ Create output reports in two formats. ``` ### Phase 7: Comment to JIRA and Add Label (Optional) -Only if `--comment` flag is provided. +Only executed if `--comment` flag is provided. When enabled, processes ALL triaged issues regardless of confidence level. **Actions:** -- For each issue with confidence ≥80%: +- For each triaged issue (NO CONFIDENCE FILTERING): - **Convert team mention:** Convert GitHub team handle to JIRA team mention - Use mapping from `reference/jira-team-mappings.md` - Format: `[Team Display Name](https://redhat.atlassian.net/jira/people/team/{team-id}?ref=jira$&src=issue)` @@ -261,13 +268,14 @@ Only if `--comment` flag is provided. - **Post comment:** Post structured comment with team recommendation, confidence, reasoning - Use comment format from `templates/jira-comment.md` - Use `mcp__mcp-atlassian__jira_add_comment` + - Include confidence level in comment (always show percentage) + - For low confidence (<80%), add warning: "⚠️ Low confidence - requires manual review" - **Add label:** After successful comment post, add `auto-triaged` label to the issue - Use `mcp__mcp-atlassian__jira_update_issue` with `labels` parameter - Append to existing labels (don't replace) - Example: `{"labels": ["auto-triaged"]}` - **Log:** Record issue key, team, confidence, comment status, and label status -- Skip issues with low confidence (<80%) -- Log all posted comments, labels added, and skipped issues (with reason) +- Log all posted comments and labels added (no issues are skipped based on confidence) **Comment Template:** See `templates/jira-comment.md` for format and variable substitution. @@ -290,13 +298,23 @@ Triage all untriaged issues (read-only): ``` /triage ``` +This will: +- Fetch all untriaged issues (without `auto-triaged` label) +- Classify, analyze, and assign teams +- Generate reports (no JIRA writes) Triage all untriaged issues and post comments: ``` /triage --comment ``` - -Triage a specific issue: +This will: +- Fetch all untriaged issues +- Classify, analyze, and assign teams +- Post triage comments to ALL issues (regardless of confidence level) +- Low confidence (<80%) issues get ⚠️ warning in comment +- Add `auto-triaged` label to prevent duplicate processing + +Triage a specific issue (read-only): ``` /triage ROX-12345 ``` diff --git a/workflows/acs-triage/CLAUDE.md b/workflows/acs-triage/CLAUDE.md index 76e881f..bbbf338 100644 --- a/workflows/acs-triage/CLAUDE.md +++ b/workflows/acs-triage/CLAUDE.md @@ -19,9 +19,11 @@ This is a **single-purpose workflow** for automated triage of StackRox/ACS JIRA The workflow provides 2 main commands: - `/triage` - Complete end-to-end triage pipeline: setup → fetch → classify → analyze → assign → report (READ-ONLY) -- `/triage --comment` - Full triage pipeline + post analysis comments to JIRA + add auto-triaged label (⚠️ WRITES to JIRA) +- `/triage --comment` - Full triage pipeline + post analysis comments to ALL issues (regardless of confidence) + add auto-triaged label (⚠️ WRITES to JIRA) - `/comment-issues` - Standalone command to add triage comments to JIRA (requires prior /triage run) +**Note:** When using `--comment`, ALL triaged issues receive comments regardless of confidence level. Low confidence (<80%) issues include a ⚠️ warning flag in the comment for manual review. + **Simplified Design:** All triage steps are consolidated into a single `/triage` command for ease of use. **Idempotent Execution:** The workflow uses JQL search with `labels NOT IN (auto-triaged)` exclusion. After triaging an issue and posting a comment, the `auto-triaged` label is added. This makes the workflow safe to run repeatedly - only new untriaged issues will be processed. diff --git a/workflows/acs-triage/README.md b/workflows/acs-triage/README.md index 84c7b3b..e65d32b 100644 --- a/workflows/acs-triage/README.md +++ b/workflows/acs-triage/README.md @@ -14,15 +14,13 @@ This workflow provides systematic triage of untriaged StackRox issues using: ### Workflow Phases -1. **Setup** - Clone StackRox repository for CODEOWNERS and reference data -2. **Fetch Issues** - Retrieve untriaged issues from JIRA filters (103399, 95004) +1. **Setup** - Clone StackRox repository for CODEOWNERS and stackrox/skills for reusable analysis skills +2. **Fetch Issues** - Retrieve untriaged issues from JIRA using JQL search 3. **Classify** - Categorize as CI_FAILURE, VULNERABILITY, FLAKY_TEST, or UNKNOWN 4. **Analyze** - Apply specialized analysis based on type 5. **Assign Team** - Multi-strategy assignment with confidence scoring -6. **Generate Reports** - Create markdown, HTML, and Slack outputs -7. **Review** - Human review of recommendations -8. **Comment** (Optional) - Add triage comments to JIRA -9. **Execute** - Manual JIRA updates based on report +6. **Generate Reports** - Create markdown and JSON outputs +7. **Comment** (Optional with --comment) - Add triage comments to ALL issues (regardless of confidence) + auto-triaged label ## Getting Started diff --git a/workflows/acs-triage/reference/constants.md b/workflows/acs-triage/reference/constants.md index fc60210..1fa02a9 100644 --- a/workflows/acs-triage/reference/constants.md +++ b/workflows/acs-triage/reference/constants.md @@ -113,7 +113,9 @@ Central location for all hardcoded values used throughout the ACS triage workflo ## Repository Paths -| Repository | Clone Path | Files Needed | -|-----------|-----------|--------------| -| stackrox/stackrox | /tmp/triage/stackrox | .github/CODEOWNERS, VERSION | -| stackrox/skills | /tmp/triage/skills | plugins/rhacs-patch-eval/* | +| Repository | Clone Path | Resources Needed | +|-----------|-----------|-----------------| +| stackrox/stackrox | /tmp/triage/stackrox | .github/CODEOWNERS, VERSION, .claude/agents/* | +| stackrox/skills | /tmp/triage/skills | .claude/skills/* (rhacs-patch-eval, etc.) | + +**Skills Repository:** Contains reusable skills for ACS-specific analysis tasks. Skills can be loaded on-demand during triage workflow execution. From aa8923e6d7bd302b78d9d2c4535416033a1d1e0e Mon Sep 17 00:00:00 2001 From: Tomasz Janiszewski Date: Thu, 7 May 2026 11:43:49 +0200 Subject: [PATCH 2/2] Clean up triage workflow documentation Code quality improvements from /simplify review: 1. Add low confidence warning threshold to constants - Defined 80% threshold in reference/constants.md - Removed magic number from multiple locations 2. Remove redundant behavior description - Consolidated --comment flag behavior into options list - Eliminated duplicate explanation 3. Remove deprecated field documentation - Removed vuln_analysis.component (marked deprecated) - Use language + package_name instead Co-Authored-By: Claude Sonnet 4.5 --- workflows/acs-triage/.claude/commands/triage.md | 4 +--- workflows/acs-triage/FIELD_REFERENCE.md | 9 +-------- workflows/acs-triage/reference/constants.md | 5 ++++- 3 files changed, 6 insertions(+), 12 deletions(-) diff --git a/workflows/acs-triage/.claude/commands/triage.md b/workflows/acs-triage/.claude/commands/triage.md index 2297711..85877b2 100644 --- a/workflows/acs-triage/.claude/commands/triage.md +++ b/workflows/acs-triage/.claude/commands/triage.md @@ -6,12 +6,10 @@ Complete end-to-end triage workflow for StackRox/ACS JIRA issues. Fetches untria **Options:** - `/triage` - Full triage pipeline using JQL search (READ-ONLY, no JIRA writes) -- `/triage --comment` - Full triage + post comments to ALL issues (regardless of confidence) + add auto-triaged label +- `/triage --comment` - Full triage + post comments to ALL issues (regardless of confidence, <80% get ⚠️ warning) + add auto-triaged label - `/triage ROX-12345` - Triage a specific issue by key (READ-ONLY) - `/triage ROX-12345 --comment` - Triage specific issue + post comment + add label -**Behavior with --comment:** Posts triage analysis comments to ALL triaged issues regardless of confidence level. Low confidence (<80%) issues include a warning flag for manual review. - ## Prerequisites - JIRA MCP connection configured diff --git a/workflows/acs-triage/FIELD_REFERENCE.md b/workflows/acs-triage/FIELD_REFERENCE.md index 1b2e6d7..797e2f4 100644 --- a/workflows/acs-triage/FIELD_REFERENCE.md +++ b/workflows/acs-triage/FIELD_REFERENCE.md @@ -299,14 +299,7 @@ These fields are added by the `/analyze-vuln` command for VULNERABILITY issues: - **Example:** "github.com/stackrox/rox/scanner/pkg", "react-dom", "golang.org/x/net" - **Purpose:** Specific package/library affected with full import path - **Extracted From:** Issue description or CVE details -- **Note:** Used for Go dependency team assignment - -#### vuln_analysis.component -- **Type:** string -- **Example:** "scanner", "central", "ui" -- **Purpose:** Affected component/module (informational, secondary to language/package) -- **Extracted From:** Issue description or labels -- **Deprecated:** Use language + package_name for team assignment instead +- **Note:** Primary field for Go dependency team assignment (with language) #### vuln_analysis.decision_tree - **Type:** object diff --git a/workflows/acs-triage/reference/constants.md b/workflows/acs-triage/reference/constants.md index 1fa02a9..554e65e 100644 --- a/workflows/acs-triage/reference/constants.md +++ b/workflows/acs-triage/reference/constants.md @@ -25,10 +25,13 @@ Central location for all hardcoded values used throughout the ACS triage workflo | Range | Classification | Recommendation | |-------|---------------|----------------| | ≥90% | High | Ready for automatic assignment | -| 70-89% | Medium | Review before assignment | +| 80-89% | Medium-High | Safe for assignment with comment | +| 70-79% | Medium | Review before assignment | | <70% | Low | Manual review required | | 0% | None | Needs manual assignment | +**Low Confidence Warning Threshold**: 80% - Issues below this threshold receive "⚠️ Low confidence - requires manual review" warning in JIRA comments. + ## Severity Thresholds (Vulnerabilities) | Severity | CVSS Range | Triage Decision |