feat(labeler): code/metadata AI adapter (W8.2)#1994
Conversation
Add analyzeCode: a pure module that analyzes a plugin's source and metadata with Kimi via the injected env.AI binding and returns validated NormalizedFinding[] (source "model"), with strict JSON-schema output constrained to the policy's allowed categories. Plugin-controlled text (file content, paths, metadata) is fenced as untrusted with a per-request boundary and sentinel-escaped so it can't forge a boundary or inject instructions. Model/parse failures and model-produced invalid findings are ModelTransientError (retryable); model-supplied source/evidenceRefs/sourceMetadata are overridden. Input is budgeted against assembled prompt size and degrades coverage to partial rather than truncating silently. Not yet wired to the orchestrator's codeAi stage; that waits on the W7.3 bundle-input producer.
|
Scope checkThis PR changes 652 lines across 4 files. Large PRs are harder to review and more likely to be closed without review. If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs. See CONTRIBUTING.md for contribution guidelines. |
github-actions
Bot
added
size/XL
review/needs-review
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-demo-do | ed11953 | Jul 12 2026, 10:09 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-demo-cache | ed11953 | Jul 12 2026, 10:10 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-playground | ed11953 | Jul 12 2026, 10:11 PM |
![emdashbot[bot]](https://avatars.githubusercontent.com/in/3255304?s=60&v=4){kind=small}
There was a problem hiding this comment.
This PR adds a self-contained W8.2 code/metadata AI adapter for the labeler, and the approach is sound: a pure module with injected AI binding, deterministic output normalization through validateFindings, explicit provenance overrides, fence escaping, and retry/failure semantics mapped to ModelTransientError.
I checked the prompt-fence escaping, path injection hardening, field-override behavior, policy-derived schema enum, failure mapping, Worker config updates, and test coverage. They all look correct and well-tested.
The one real issue is in capFilesToBudget: the per-file cost and overhead constants undercount the actual assembled user content by N + 1 newline characters for N files. That contradicts the PR's "honest coverage" guarantee—near the 200 k-char limit the adapter can still report coverage: "complete" while sending a slightly oversized context to the model. This is small but exactly the kind of silent truncation the change is meant to prevent.
Fixing the arithmetic and adding a close-to-limit regression test would make this clean to approve.
| const renderedCost = (file: CodeAnalysisFile) => buildFence(file, boundary).length + 1; | ||
| const total = files.reduce((sum, file) => sum + renderedCost(file), fixedOverheadChars); |
There was a problem hiding this comment.
[needs fixing] buildUserContent assembles the user message by joining [metadataFence, "", file1Fence, "", file2Fence, "", ...], which produces 2N + 1 newline characters for N files. However, the budget math in capFilesToBudget only adds + 1 newline per file and starts from fixedOverheadChars without accounting for the leading separator between metadataFence and the first empty part. The assembled prompt is therefore undercounted by N + 1 characters.
For a bundle near the 200 k-char limit this allows the adapter to mark coverage: "complete" while still exceeding the model's context window, silently truncating instead of reporting partial coverage.
| const renderedCost = (file: CodeAnalysisFile) => buildFence(file, boundary).length + 1; | |
| const total = files.reduce((sum, file) => sum + renderedCost(file), fixedOverheadChars); | |
| const renderedCost = (file: CodeAnalysisFile) => buildFence(file, boundary).length + 2; | |
| const total = files.reduce((sum, file) => sum + renderedCost(file), fixedOverheadChars + 1); |
The current oversized-bundle test uses very large files, so please also add a regression test sized just under/over the real 200 k limit so this arithmetic doesn't drift again.
github-actions
Bot
added
review/awaiting-author
1 participant
What does this PR do?
Adds the labeler's code/metadata AI adapter — plan item W8.2.
analyzeCode(input, deps)analyzes a plugin's source files and metadata with a code model (Kimi@cf/moonshotai/kimi-k2.7-code) via the Workers AI binding (env.AI, injected — not AI Gateway) and returns validatedNormalizedFinding[](source"model"), feeding the W8.5 policy resolver.AIbinding is injected viadeps; the module does no DB writes and imports nothing fromcloudflare:workers, so it's fully testable against a fake binding and decoupled from the orchestrator's still-abstractStageContext. Wiring it into thecodeAistage — threading the acquired bundle and recording evidence objects — is deferred to the W7.3-consumer work once a real bundle-input producer exists.response_format: json_schemaconstrains each finding'scategoryto exactly the policy's allowed set (automated-block ∪ warning) andseverity/confidenceto their ranges. The enum is derived from the versioned policy, not hardcoded.coverage: "partial"with the dropped paths — it never truncates silently.ModelTransientError(retryable) — a flaky model is not the non-retryable stage-adapter bug aFindingValidationErrorsignals. Model-suppliedsource/evidenceRefs/sourceMetadataare overridden so a model can't spoof provenance. A blankpromptVersionis a caller bug and throws up front (not retried).No migration, no lexicon change, no public-API change. Adds the
AIbinding toapps/labeler/wrangler.jsonc(regeneratedworker-configuration.d.ts).Reviewed with an opus adversarial pass that found and drove fixes for a fence-escape via unescaped paths, budget undercounting, a misclassified-retryable config bug, and two missing tests — all addressed here before opening.
Targets
feat/plugin-registry-labelling-service. Related to #1909 and RFC #694.Type of change
Checklist
pnpm typecheckpassespnpm lintpassespnpm testpasses (or targeted tests for my change)pnpm formathas been runmessages.pochanges except in translation PRs — a workflow extracts catalogs on merge tomain.Admin i18n and changeset are n/a — this touches only
apps/labeler, a private, unpublished Worker app (no admin UI, not on npm).AI-generated code disclosure
Screenshots / test output
pnpm --filter @emdash-cms/labeler test: 228 pass (16 files) — 17 new for the adapter (happy path; content-fence + malicious-path injection resistance; model-field override; schema enum = policy; transport/parse/out-of-enum →ModelTransientError; blank promptVersion →TypeError; oversized-bundle coverage/drops; no-cache option; promptHash stability + policy-sensitivity)pnpm lint:jsondiagnostics unchanged at 19 (0 inapps/labeler)AiBindingonly — never the live model (Workers AI bills even in local dev)Try this PR
Open a fresh playground →
A full working EmDash site, deployed from this branch. Each visit gets its own session-scoped sandbox: no login needed and no shared state. Try the admin, edit content, hit the public site.
Tracks
feat/labeler-code-ai-adapter. Updated automatically when the playground redeploys.