fix: scope incremental structure/roles neighbor expansion to the changed file#1950
Conversation
…ged file The "1-file rebuild" benchmark regressed ~85-155% vs the 3.15.0 baseline on both engines, blocking the pre-publish and perf-canary CI gates on every open PR. Two independent, correct correctness fixes each added unbounded-ish cost that only shows up when the changed file is itself a hub (imported/re-exported by many other files) — exactly what the benchmark's fixed probe file, src/domain/queries.ts, is: - #1840's directory-metrics neighbor discovery was scoped to the changed file's whole containing directory instead of the file itself, pulling in unrelated sibling files' edges. For queries.ts (leaf dir src/domain, itself a hub) this found 251 neighbour files / 29 affected directories, including the repo-root src/ (a near-full-table scan). Scoping to the exact file cuts this to 46 / 13. - #1849 added symbol-level reexports edges so re-exported symbols get fan-in credit. That correctly widened do_classify_incremental's neighbour-file set for barrel files like queries.ts (0 -> 15 files), which then hit two pre-existing inefficiencies never exercised at this scale on a 1-file change: an OR-based self-join SQLite can't index, and a repo-wide recursive CTE (the same one do_classify_full legitimately needs) answering "is this barrel production-reachable" for a question only 1-3 barrels actually need answered. Fixes, applied to both the native Rust classifier and its TypeScript/WASM mirror: - Scope neighbor discovery to the exact touched file (indexed point lookup) instead of a directory-range scan. - Rewrite the OR self-join as a UNION of two directional equi-joins (identical result set, ~2.4x faster). - Replace the global reachability closure with a backward-scoped check: find the small set of barrels reexporting into the affected files, then answer reachability per-barrel by walking reexports edges backward (bounded by that barrel's own chain depth) instead of materializing the whole graph's forward closure. Verified against the dedicated correctness suites for both #1840 (issue-1738-structure-metrics-incremental.test.ts) and #1849 (issue-1742-reexport-symbol-scope.test.ts) on both engines, plus the full test suite and engine-parity tests. docs check acknowledged: internal performance bug fix to the incremental build pipeline, no new feature/language/CLI surface/architecture change -- README.md, CLAUDE.md, and ROADMAP.md are unaffected. Fixes #1855 Impact: 4 functions changed, 5 affected
Codegraph Impact Analysis4 functions changed → 5 callers affected across 3 files
|
Greptile SummaryThis PR fixes a performance regression (~85–155% on the "1-file rebuild" benchmark) introduced by two prior correctness fixes (#1840, #1849) that each added cost to the incremental structure/role classification path when the changed file is a widely-imported hub like
Confidence Score: 5/5Safe to merge — all three optimizations preserve the result sets of the queries they replace, correctness is backed by the dedicated regression suites for #1840 and #1849, and the Rust/TypeScript implementations mirror each other faithfully. The neighbor-scoping change is a straightforward predicate swap (range to equality) that can only reduce the working set. The UNION rewrite is semantically equivalent to the original OR-join — both directions are covered, UNION deduplicates the same way DISTINCT did, and parameter binding is correctly repeated four times. The backward-scoped is_barrel_prod_reachable correctly answers the same yes/no question as the global CTE, and UNION (not UNION ALL) in the recursive CTE guarantees termination on cycles. The cachedStmt additions address the prior review comment cleanly. No files require special attention. Important Files Changed
Reviews (3): Last reviewed commit: "fix: cache the per-barrel reachability s..." | Re-trigger Greptile |
…hable Addresses Greptile's review note on #1950: both queries in is_barrel_prod_reachable / isBarrelProdReachable are re-prepared on every call even though their SQL text never changes (the interpolated test-file filter is a fixed set of LIKE patterns keyed only by column name). Harmless at the expected 1-3 barrels per incremental build, but free to fix: Rust now uses prepare_cached (already the established idiom elsewhere in this file and connection.rs) and TypeScript mirrors it with the existing cachedStmt/StmtCache WeakMap pattern used in graph-read.ts and exports.ts. No behavior change — verified against the same test suites as #1950 (issue-1738, issue-1742, full suite, lint). docs check acknowledged: internal micro-optimization, no new feature/ language/CLI surface/architecture change -- README.md, CLAUDE.md, and ROADMAP.md are unaffected. Impact: 1 functions changed, 2 affected
Summary
Fixes the "1-file rebuild" benchmark regression (~85-155% vs the 3.15.0 baseline on both engines) that's currently blocking the pre-publish and perf-canary CI gates on every open PR, including #1885, #1886, and #1889.
Root cause: two independent, correct correctness fixes (#1840, #1849) each add cost to the incremental role/structure-metrics recompute that only shows up when the changed file is itself a "hub" (imported/re-exported by many other files) — exactly what the benchmark's fixed probe file,
src/domain/queries.ts, is.refresh_affected_directory_metrics/refreshAffectedDirectoryMetrics) was scoped to the changed file's whole containing directory, not the file itself, pulling in unrelated sibling files' edges. Forqueries.ts(leaf dirsrc/domain, itself a hub), this found 251 neighbour files / 29 affected directories, including the repo-rootsrc/(a near-full-table scan). Scoping to the exact file cuts this to 46 / 13.reexportsedges so re-exported symbols get fan-in credit and aren't misclassified dead. That correctly wideneddo_classify_incremental's neighbour-file set for barrel files likequeries.ts(0 → 15 files), which then hit two pre-existing inefficiencies never exercised at this scale on a 1-file change: an OR-based self-join SQLite can't index, and a repo-wide recursive CTE (the same one the full-classify path legitimately needs) answering "is this barrel production-reachable" for a question only 1-3 barrels actually need answered.Fix (both native Rust and TypeScript/WASM)
UNIONof two directional equi-joins (identical result set, ~2.4x faster).reexportsedges backward (bounded by that barrel's own chain depth) instead of materializing the whole graph's forward closure.Every correctness guarantee from #1840/#1849 is preserved — verified against their dedicated test suites (
issue-1738-structure-metrics-incremental.test.ts,issue-1742-reexport-symbol-scope.test.ts) on both engines.Before / after (local,
src/domain/queries.tsprobe, native engine)Note: WASM's
structureMs/rolesMsare fixed to the same degree as native (e.g.structureMs44.7ms → 24-32ms,rolesMs46.9ms → 16-23ms). Its total still shows some residual elevation on my local (shared, noisy) dev machine, traced todetectMs(change detection, unrelated to this fix — filed separately as #1948) rather than anything this PR touches.Test plan
npm run lintcleancargo build --release/cargo check --workspaceclean (only a pre-existing, unrelated warning)cargo test --release: 447 passedtests/engines/,build-parity.test.ts,dropped-language-gap.test.ts) passingstructureMs/rolesMsphase-level fix is consistent, not a lucky sample — see perf: 1-file incremental rebuild regresses +79-81% (117ms -> ~209ms), likely from PR #1840 structure-metrics recompute #1855 for full numbers and reconciliation with CI: '1-file rebuild' benchmark baseline (117ms) appears stale against current runner performance #1864Fixes #1855