fix: eliminate non-deterministic ordering in community detection#1835
Merged
carlos-alm merged 13 commits intoJul 6, 2026
Conversation
BATCH_COMMANDS entries with sig: 'dbOnly' (currently only complexity) always wrote their target into opts.target. complexityData treats opts.target as a symbol-name filter and opts.file as the file-path filter, so batch complexity <file> never matched anything and the function list silently fell back to empty with a whole-repo summary regardless of which file was requested. Add an optional targetKey on BatchCommandEntry so each dbOnly command can declare which opts key its targets map to, and set it to 'file' for complexity. batchData and multiBatchData now route the target through the declared key instead of assuming opts.target. Fixes #1721 Impact: 4 functions changed, 6 affected
…ssification codegraph roles --role dead flagged every function parameter as dead-leaf and every interface/type member as dead-unresolved, regardless of actual usage. Root cause: the role classifier's fan-in-based "no callers = dead" heuristic is meaningless for these two symbol categories, since neither can ever have inbound call edges by construction: - Parameters: `kind = 'parameter'` nodes were unconditionally force-assigned dead-leaf via a fast-path bypass in classifyNodeRolesFull/Incremental (TS) and do_classify_full/do_classify_incremental (native), before any fan-in was even computed. A parameter's liveness is a local dataflow question (is it referenced within its own function body), not a call-graph reachability question, so this produced ~90% noise in --role dead output. - Interface/type members: every language extractor qualifies interface/type members as `Owner.member` top-level definitions (mirroring class method qualification), and they never receive inbound call edges (they're consumed via type annotations, not calls). Lacking any recognition of this, they fell through the normal fan-in/fan-out path and landed in dead-unresolved. Fix: - Parameters are now fully excluded from role classification (role stays NULL), the same treatment already given to file/directory nodes. - Interface/type members are now recognized in the classifier by resolving the Owner.member prefix against same-file TYPE_DEF_KINDS declarations (interface/type/struct/enum/trait/record) and classified `leaf` unconditionally. Class methods use the identical Owner.member naming convention but are unaffected since `class` is not in TYPE_DEF_KINDS, so real dead methods/functions remain detected. Applied to both the TS/WASM classifier (graph/classifiers/roles.ts, features/structure.ts) and the native Rust classifier (graph/classifiers/roles.rs) per the dual-engine parity requirement; verified both engines produce identical results end-to-end against this repo's own graph. Fixes #1723 Impact: 6 functions changed, 5 affected
codegraph exports <file> --json (and the --unused dead-export filter it
feeds) reported zero consumers for interfaces/types that are demonstrably
imported and used elsewhere via `import type { X }`, even though
`codegraph deps <file>` correctly showed the importing file in
`importedBy`. Example: ChaContext in src/domain/graph/builder/cha.ts is
imported (type-only) by build-edges.ts and native-orchestrator.ts, but
`codegraph exports` showed consumerCount: 0.
Root cause: exportsFileImpl's per-symbol consumers query
(domain/analysis/exports.ts) only looked at kind = 'calls' edges. The
builder already emits a symbol-level `imports-type` edge for `import
type { X }` statements (source = importing file node, target = the
specific imported symbol -- see emitTypeOnlySymbolEdges in
build-edges.ts/incremental.ts), which `codegraph deps` reads from, but
the exports consumer query never looked at this edge kind. Role
classification (features/structure.ts) already includes 'imports-type'
in its fan-in formula, so `codegraph roles --role dead` was unaffected --
this was purely a gap in exports's independent consumer list.
Fix: widen the consumers query to `kind IN ('calls', 'imports-type')`,
matching the edge-kind set already used everywhere else in the codebase
for cross-file usage credit (structure.ts, graph-enrichment.ts,
boundaries.ts, dependencies.ts). No native Rust changes needed --
domain/analysis/exports.ts is pure query-layer code that reads the
already-built edges table and has no engine-specific mirror.
Deliberately does NOT add `extends`/`implements`: investigation found
those edges are resolved by symbol name only, with no file/import
scoping (buildClassHierarchyEdges in both build-edges.ts/incremental.ts
and the native emit_hierarchy_edges), so they link same-named
declarations across unrelated files (verified: this repo's own graph has
false `implements`/`extends` edges between unrelated fixture classes
across languages and a `Repository` interface in src/types.ts). Filed
as #1812. Also filed #1813 for a related but distinct gap: `import {
type X }` inline per-specifier modifiers aren't tracked as type-only in
either engine's extractor, so such X still gets no credit even after
this fix.
Added tests/integration/exports.test.ts coverage: an interface consumed
only via a symbol-level `imports-type` edge gets consumerCount >= 1 and
is excluded from --unused, while a genuinely unreferenced interface
still shows 0 consumers.
Verified against this repo's own graph: `codegraph exports
src/domain/graph/builder/cha.ts -T --json` now credits ChaContext with
2 consumers (build-edges.ts, native-orchestrator.ts). Full test suite
(201 files, 3359 tests) and lint pass clean.
docs check acknowledged: internal bug fix to exports consumer
computation, no new feature/language/CLI surface/architecture change --
README.md, CLAUDE.md, and ROADMAP.md are unaffected.
Fixes #1724
Impact: 1 functions changed, 3 affected
The CLI's `-f/--file` option is a repeatable Commander accumulator (collectFile) that always produces a string[], even on first use. The native composite bindings backing fn-impact (findNodesWithFanIn) and query (fnDeps) forwarded that array straight into napi-rs functions whose Rust signatures only accepted a single String, so any use of -f/--file crashed with "Failed to convert JavaScript value ... into rust type `String`" regardless of engine defaults. context worked only because its code path never touches the native repository for symbol lookup. Widen the native Rust signatures (find_nodes_with_fan_in, fn_deps) to accept Vec<String> and build an OR-of-LIKE clause for multiple files, mirroring buildFileConditionSQL/NodeQuery.fileFilter on the JS side. Normalize the file option before calling into the native binding on the TS side, and thread the widened string | string[] type through QueryOpts, fnDeps's opts, and the call chain down to findMatchingNodes. This also makes repeated -f/--file genuinely multi-file end to end for fn-impact/query, matching context's existing behavior, instead of crashing on any use. findNodesByScope shares QueryOpts but has no CLI caller and its native binding still only accepts one file; it now takes the first value defensively rather than crash or fail to type-check. Fixes #1726 docs check acknowledged: pure bug fix, no new CLI options/languages/ architecture — the -f/--file "repeatable" docs are now accurate rather than needing correction. Impact: 14 functions changed, 34 affected
… exports Two compounding root causes made `where --file`'s `exported` array (and `codegraph exports`) unreliable for entire classes of exports: 1. The JS/TS export-statement handler (WASM query path, WASM walk path, and the mirrored native Rust extractor) only recognized `export function`, `export class`, `export interface`, and `export type` declarations. It never matched `lexical_declaration`/`variable_declaration`, so `export const/let/var …` was never added to the extractor's export list — regardless of the initializer's shape — leaving the `exported=1` DB column permanently unset for every exported constant in the codebase (including ones that happened to look "correct", like a `new Set(...)`-initialized constant referenced as a call argument elsewhere). 2. `where --file`'s `exported` list ignored the `exported` DB column entirely and computed membership from `findCrossFileCallTargets` (symbols targeted by a cross-file `calls` edge) — a heuristic that only "worked" by coincidence for symbols that happened to be passed as call arguments elsewhere. `codegraph exports` already had the right idea (prefer the `exported` column, fall back to the heuristic only for pre-migration DBs) but duplicated that logic locally instead of sharing it. Fixes: - src/extractors/javascript.ts: extract the function/class/interface/type kind map plus a new lexical/variable-declaration branch into a shared `collectExportedDeclarations`, used by both `handleExportCapture` (query path) and `handleExportStmt` (walk path) so they can't drift apart again. Declarator values are classified with the same predicate already used to build the matching Definition (function-valued -> kind 'function', literal/array/object/new-expression-valued `const` -> kind 'constant'). - crates/codegraph-core/src/extractors/javascript.rs: mirror the same fix in `handle_export_declaration` via a new `collect_exported_var_declarations`. - src/db/repository/nodes.ts: add `findExportedNodesByFile`, the single shared implementation of "prefer exported=1, fall back to cross-file calls for pre-migration DBs" — re-exported through db/repository/index.ts and db/index.ts. - src/domain/analysis/exports.ts: use the shared helper instead of a locally duplicated hasExportedCol/exportedNodesStmt implementation. - src/domain/analysis/symbol-lookup.ts: `whereFileImpl` now uses the shared helper instead of `findCrossFileCallTargets` directly, so `where --file` and `exports` agree on what "exported" means. Tests: - tests/parsers/javascript.test.ts: unit coverage for bare-literal, new Set(...), object-literal-with-methods, and arrow-function exported consts, plus a non-exported-const negative case and a function/class regression guard for the shared-helper refactor. - tests/engines/query-walk-parity.test.ts, tests/engines/parity.test.ts: cross-path/cross-engine regression cases for the same shapes. - tests/integration/queries.test.ts: new fixture node with exported=1 but zero incoming edges of any kind, proving `where --file`'s exported list no longer depends on cross-file call presence. No user-facing command/language/architecture changes, so README/CLAUDE.md/ ROADMAP.md are unaffected — docs check acknowledged. Fixes #1728 Impact: 8 functions changed, 17 affected
tree-sitter-typescript's predefined_type production (string, number, boolean, ... primitive type keywords) lexes its keyword as an anonymous grammar token whose type string is identical to the named `string` node type used for real string literals. Both engines matched by node type alone, so `codegraph ast --kind string` spuriously matched bare `string` type annotations (interface fields, parameter types, return types) as if they were string literals. Add an isNamed/is_named() guard scoped to the JS/TS/TSX family on both engines: WASM via astRequiresNamedNode() in ast-analysis/rules/index.ts feeding the shared resolveAstKind() in ast-store-visitor.ts, native via a match guard in javascript.rs's walk_ast_nodes_depth. Genuine string, template, and string-literal-type nodes are always named and unaffected. No user-facing command/language/architecture changes, so README/CLAUDE.md/ ROADMAP.md are unaffected — docs check acknowledged. Fixes #1729 Impact: 8 functions changed, 13 affected
codegraph exports <file> reported zero consumers for an exported
function/symbol that is genuinely imported and used elsewhere, whenever
the importing file renamed the binding at the import site
(import { X as Y } from '...'). Example: collectFiles in
src/domain/graph/builder/helpers.ts is imported (as collectFilesUtil)
and called by collect-files.ts and native-orchestrator.ts, but
`codegraph exports helpers.ts -T --json` showed consumerCount: 0.
Root cause (extractor, both engines): extractImportNames (javascript.ts)
and its mirrored scan_import_names_depth (javascript.rs) handled
import_specifier nodes with `node.childForFieldName('name') ||
childForFieldName('alias')`. Per the tree-sitter grammar, `name` is
*always* present on import_specifier (the name as declared by the source
module); `alias` only exists for `X as Y` and holds the *local* binding
actually referenced by call sites. Preferring `name` unconditionally
meant the local alias was silently dropped for every renamed import —
`imp.names` recorded "collectFiles" instead of "collectFilesUtil", so
`importedNames` (keyed by call-site text) never had a matching entry and
the call fell through unresolved.
Fixing only the extractor was not sufficient: once `imp.names` correctly
holds the local alias, `resolveCallTargets` (call-resolver.ts) /
resolve_call_targets (build_edges.rs) still searched the *target file*
for a symbol literally named "collectFilesUtil" — which doesn't exist
there (only "collectFiles" does). This required threading a second
piece of information end-to-end: for each renamed specifier, the local
alias's *original* exported name, so target-file/barrel lookups search
by the right name while importedNames stays keyed by the call-site text.
Changes (both engines, full-build + incremental/native-orchestrator +
native-hybrid paths):
- types.ts / types.rs: new `Import.renamedImports` /
`Import.renamed_imports` field — { local, imported } pairs, populated
only for specifiers that actually rename a binding.
- extractors/javascript.ts, .rs: extractImportNames /
extract_import_names_with_renames now prefer `alias` (local binding)
for import_specifier and record the rename pair. export_specifier is
deliberately left unchanged (see scope notes below). Fixed in both the
walk path (handleImportStmt) and the query/WASM-worker path
(handleImportCapture) on the TS side.
- build-edges.ts, build_edges.rs, import_edges.rs, pipeline.rs,
call-resolver.ts, incremental.ts: buildImportedNamesMap /
collect_imported_names_for_file now also produce a local-alias ->
original-name map; resolveCallTargets / resolve_call_targets consult
it to search the target file by the original name. Applied
consistently to the primary call-resolution path, the
Object.defineProperty post-pass, points-to alias resolution, barrel
edge / import-type edge emission, and cross-file return-type
propagation — every place that previously assumed a call site's local
name equals the target file's declared name.
Native Rust: touched and verified. Rebuilt locally via
`npx napi build --platform --release` (crates/codegraph-core), codesigned
(`codesign --sign - --force`), and installed over the loaded
node_modules/@optave/codegraph-darwin-arm64/codegraph-core.node so the
fix was exercised by the actual native engine, not the prebuilt npm
binary. `cargo test --lib`: 427 passed.
Tests added:
- tests/parsers/javascript.test.ts: extractor-level coverage for the
local-name/rename-pair extraction, a mixed renamed+non-renamed
specifier list, and confirmation export_specifier is unaffected.
- crates/codegraph-core/src/extractors/javascript.rs: matching Rust unit
tests.
- tests/integration/issue-1730-renamed-import-consumer.test.ts: new
end-to-end test building real files through buildGraph() on both wasm
and native engines, asserting the calls edge exists, no edge is
created against a nonexistent "collectFilesUtil" symbol, and
`codegraph exports` credits the consumer — on both engines.
tests/integration/exports.test.ts was not extended: that file
hand-inserts DB rows and only exercises the query layer, which is
unaffected by this bug (the missing piece was edge creation, not the
consumer query added for #1724).
Verified against this repo's own graph (native engine, rebuilt):
`codegraph exports src/domain/graph/builder/helpers.ts -T --json` now
credits collectFiles with 2 consumers (collect-files.ts,
native-orchestrator.ts); WASM engine agrees exactly. Without -T, also
picks up tests/unit/builder.test.ts via the (unrenamed) builder.ts
barrel re-export, which already worked pre-fix and was only hidden by
-T in the original repro.
Scope notes — filed as separate issues rather than expanding this fix:
- #1823: export { X as Y } from '...' (barrel re-export with rename) is
not tracked — resolveBarrelExport/resolve_barrel_export key off the
original name, and export_specifier extraction was deliberately left
unchanged here since barrel/reexport tracing is a distinct mechanism.
- #1824: dynamic import destructuring rename
(const { a: b } = await import(...)) has the same class of bug in
extractDynamicImportNames, a different extraction function.
- #1825: a renamed import used as a call receiver
(import { X as Y }; Y.method()) still fails to resolve — confirmed
empirically (no calls edge created) — a different resolution strategy
(resolveByReceiver/resolveViaDirectQualifiedMethod in
resolver/strategy.ts) than the one this fix addresses.
Full test suite (202 files, 3398 tests) and lint pass clean.
docs check acknowledged: internal resolver/extractor bug fix, no new
feature/language/CLI surface/architecture change -- README.md,
CLAUDE.md, and ROADMAP.md are unaffected.
Fixes #1730
Impact: 29 functions changed, 47 affected
… builds insertNodes committed file_hashes for changed files in the same transaction as node insertion, before resolveImports/buildEdges rebuilt those files' edges (a separate, later stage/transaction) — in both the JS/WASM pipeline and the native Rust orchestrator. Any exception, crash, or interruption between the two left a hash that claimed a file was "up to date" while its edges still reflected an older revision. Since change-detection trusts file_hashes exclusively, that divergence was never self-healed by later builds. Defer the file_hashes commit so it only runs once edges have been rebuilt to match: - insert-nodes.ts: insertNodes no longer writes file_hashes for changed files; new commitFileHashes() does it, called from pipeline.ts after buildEdges. - insert_nodes.rs / pipeline.rs / connection.rs: do_insert_nodes no longer upserts file_hashes (only removed-file cleanup, which has no coupling risk); new commit_file_hashes() runs after Stage 7 (edges). Watch-mode's rebuildFile never wrote file_hashes at all, leaving it permanently stale after every edit — also fixed by writing/deleting the hash once a file's edges have been rebuilt or the file is deleted. Adds tests/integration/issue-1731-hash-edge-coupling.test.ts: a fault-injection test that throws inside buildEdges mid-incremental-build and asserts the hash does not advance until edges genuinely match (and that a retry self-heals), plus coverage for rebuildFile keeping file_hashes in sync with its edge rebuilds. Fixes #1731 Impact: 8 functions changed, 13 affected
checkNoSignatureChanges compared def.line (post-change, new-file coordinates, from the rebuilt graph) against diff.oldRanges (pre-change, old-file coordinates). Any diff that shifts line counts before a given point in a file — virtually all deletions/insertions — could make an untouched symbol's new line number coincidentally fall inside the old hunk's range, producing a false "signature change" violation. checkMaxBlastRadius already used diff.changedRanges (new-file positions) against the same db; checkNoSignatureChanges now does the same. The parameter is renamed from oldRanges to changedRanges to make the expected coordinate space explicit. diff.oldRanges remains computed by parseDiffOutput and is still exercised directly by its own tests. No README/CLAUDE.md/ROADMAP updates needed — internal predicate logic fix, no new commands, languages, or architecture changes (docs check acknowledged). Fixes #1732 Fixes #1737 Impact: 2 functions changed, 5 affected
…g WASM grammars Every git worktree gets its own untracked node_modules/ and grammars/, so a worktree set up before a host Node upgrade (or where npm install was interrupted) can end up with a better-sqlite3 binary compiled for the wrong Node ABI, or an incomplete grammars/ directory — both fail deep inside a build or test run with confusing, unrelated-looking errors rather than a clear diagnosis. Adds src/infrastructure/doctor.ts with two checks whose decision logic is pure and unit-tested in isolation from the I/O: - native ABI compatibility, via a real require() attempt - grammar completeness against the full LANGUAGE_REGISTRY, split by the registry's own required flag: a missing required (JS/TS/TSX) grammar fails the check, but a missing optional grammar only warns. Non-required parsers are designed to fail gracefully at runtime (per this repo's own CLAUDE.md), so a worktree missing one rarely-used language's grammar must stay able to run npm test, not get hard-blocked before a single test starts. scripts/doctor.ts is the CLI entry point (npm run doctor, optionally with --fix for a scoped, worktree-local repair covering both blocking and non-blocking findings) and is also wired as pretest so npm test fails fast on a genuinely blocking problem instead of a wall of unrelated failures. Fixes #1733 docs check acknowledged: CLAUDE.md already covers the new npm run doctor command and infrastructure/doctor.ts (added in this same change); README.md does not document npm-run dev scripts (only the shipped codegraph CLI), and ROADMAP.md has no phase this bug-fix-sized change affects. Impact: 6 functions changed, 3 affected
`codegraph communities --drift` produced different modularity and community assignments across separate full rebuilds of byte-identical source. Two independent, compounding causes in the native Rust engine: 1. The build pipeline collected parsed file symbols into a `std::collections::HashMap` (pipeline.rs), whose iteration order is randomized per-process. That order drove node/edge insertion order into SQLite, so the same file could get a different autoincrement `id` (and therefore a different position in the in-memory graph) on every rebuild. Fixed by switching `file_symbols` to `BTreeMap` throughout the pipeline, import-edge, and structure-metrics stages, so insertion order is always sorted by file path. 2. The native Louvain local-move phase (louvain.rs) accumulated per-candidate-community weights in a `HashMap`. A genuine tie in modularity gain between candidate communities was broken by hashmap bucket order instead of a reproducible rule -- non-deterministic even with a fixed random seed, since the seed only controls visitation order, not this tie-break. Fixed by switching `cur_edges`/`comm_w` to `BTreeMap`. Also added `ORDER BY` to the node/edge read queries used to build the community-detection graph (both native `graph_read.rs` and the JS/WASM `graph-read.ts` mirror), as defense in depth: SQLite's row order for a bare `WHERE` scan is otherwise unspecified. Verified via 10+ full rebuilds of this repo's own ~900-file graph producing byte-identical `communities --drift --json` output, versus differing modularity/community counts on every rebuild beforehand. This is an internal determinism fix with no new features, commands, or language support changes, so no README/CLAUDE.md/ROADMAP.md updates are needed (docs check acknowledged). Fixes #1734 Impact: 4 functions changed, 17 affected
Contributor
Greptile SummaryThis PR fixes two compounding non-determinism bugs in the native Rust engine that caused
Confidence Score: 5/5Safe to merge. The fix is narrowly scoped to map types and SQL ordering clauses; no logic changes to graph construction, community assignment, or any data-mutation path. Every changed site is either a HashMap to BTreeMap type swap or an ORDER BY addition to a read-only SQL query. Root causes are precisely matched to the responsible data structures, and regression coverage is thorough across unit, algorithm, and integration layers. No files require special attention. Important Files Changed
Reviews (4): Last reviewed commit: "Merge origin/main into fix/issue-1734-co..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Contributor
Codegraph Impact Analysis4 functions changed → 17 callers affected across 9 files
|
…ation allocation (#1835)
Contributor
Author
Resolves conflicts from stacked-branch divergence: main now has #1828's connection.rs atomicity fix, #1822's scoped requireNamedNode fix, #1826's importNamePairs dedup, and #1834's doctor.ts Greptile fixes — all merged after this branch forked. doctor.ts had an add/add conflict (squash-merge history mangling); since this branch never touches that file, resolved by taking origin/main's version entirely. Also removed two leftover local importNamePairs definitions left by git's auto-merge (same pattern as #1829/#1834), staged explicitly this time. docs check acknowledged — conflict resolution only, no new functionality. Impact: 9 functions changed, 40 affected
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two independent, compounding non-determinism bugs in the native Rust engine, both traced to
HashMap's randomized per-construction iteration order:parse_and_index_filescollected symbols into aHashMap<String, FileSymbols>, whose iteration order drove node/edge INSERT order into SQLite — so the same file could get a different autoincrement id on every full rebuild. Confirmed viaSELECT id, file FROM nodes ... ORDER BY id LIMIT 10differing across 3 pre-fix rebuilds.local_move_phaseaccumulated per-candidate modularity-gain weights inHashMaps, so genuine ties in modularity gain (common in real dependency graphs) were broken by hashmap bucket order, not the seeded PRNG. Confirmed with a purpose-built symmetric 3-way-tie graph: 30 repeated calls gave 3 different outcomes pre-fix, 1 post-fix.Fixed by switching the affected maps to
BTreeMap(deterministic sorted iteration) plusORDER BYon the underlying SQL reads in both engines as defense in depth. WASM/JS Leiden path audited and confirmed already safe (insertion-orderedMaps, single seededmulberry32PRNG) — no changes needed there.Closes #1734
Test plan
npm test— full suite green (3431 passed, 0 failed)npm run lint— cleancargo test --lib— 428 passedcommunities --drift --jsonoutput (modularity 0.5658 every time), vs. 3 different values pre-fix (0.5552/0.5670/0.5487)Stacked on #1834 (base branch
fix/issue-1733-worktree-env-doctor) — only the algorithms/builder/repository/test diff is this issue's change.