Skip to content

fix: exclude primitive type keywords from ast --kind string matching#1822

Merged
carlos-alm merged 9 commits into
mainfrom
fix/issue-1729-ast-string-type-annotation
Jul 6, 2026
Merged

fix: exclude primitive type keywords from ast --kind string matching#1822
carlos-alm merged 9 commits into
mainfrom
fix/issue-1729-ast-string-type-annotation

Conversation

@carlos-alm

Copy link
Copy Markdown
Contributor

Summary

  • codegraph ast --kind string matched AST nodes by node.type/node.kind() string alone. Tree-sitter-typescript's grammar reuses the literal string "string" for two different node-types entries: a named node (real string literals) and an anonymous token (the string primitive type keyword in predefined_type). Without checking named-vs-anonymous, name: string type annotations were misclassified as string literals.
  • Fixed in both engines: native Rust walker now guards on is_named() for the "string"/"template_string" arm; TS/WASM side adds astRequiresNamedNode(langId) (scoped to javascript/typescript/tsx) threaded through the ast-store visitor and its three production call sites.

Closes #1729

Test plan

  • npm test — full suite green (3388 passed, 0 failed)
  • npm run lint — clean
  • Native addon rebuilt, codesigned, verified directly
  • Repro confirmed: visitor-utils.ts string-kind hits dropped 49→14 (line 10 name: string; no longer present, genuine literals like '../types.js' still present); cfg-shared.ts 18→0 false positives
  • New tests verified non-tautological (temporarily reverted fix, confirmed the new assertions fail, restored)

Filed #1821 for an out-of-scope finding: PHP has the identical bug (primitive_type > string unnamed token colliding with encapsed_string/string literals via PHP_AST_TYPES).

Stacked on #1820 (base branch fix/issue-1728-where-file-exported-omissions) — only the ast-analysis/extractor/test diff is this issue's change.

BATCH_COMMANDS entries with sig: 'dbOnly' (currently only complexity)
always wrote their target into opts.target. complexityData treats
opts.target as a symbol-name filter and opts.file as the file-path
filter, so batch complexity <file> never matched anything and the
function list silently fell back to empty with a whole-repo summary
regardless of which file was requested.

Add an optional targetKey on BatchCommandEntry so each dbOnly command
can declare which opts key its targets map to, and set it to 'file'
for complexity. batchData and multiBatchData now route the target
through the declared key instead of assuming opts.target.

Fixes #1721

Impact: 4 functions changed, 6 affected
…ssification

codegraph roles --role dead flagged every function parameter as dead-leaf and
every interface/type member as dead-unresolved, regardless of actual usage.

Root cause: the role classifier's fan-in-based "no callers = dead" heuristic
is meaningless for these two symbol categories, since neither can ever have
inbound call edges by construction:

- Parameters: `kind = 'parameter'` nodes were unconditionally force-assigned
  dead-leaf via a fast-path bypass in classifyNodeRolesFull/Incremental (TS)
  and do_classify_full/do_classify_incremental (native), before any fan-in
  was even computed. A parameter's liveness is a local dataflow question (is
  it referenced within its own function body), not a call-graph reachability
  question, so this produced ~90% noise in --role dead output.

- Interface/type members: every language extractor qualifies interface/type
  members as `Owner.member` top-level definitions (mirroring class method
  qualification), and they never receive inbound call edges (they're
  consumed via type annotations, not calls). Lacking any recognition of
  this, they fell through the normal fan-in/fan-out path and landed in
  dead-unresolved.

Fix:
- Parameters are now fully excluded from role classification (role stays
  NULL), the same treatment already given to file/directory nodes.
- Interface/type members are now recognized in the classifier by resolving
  the Owner.member prefix against same-file TYPE_DEF_KINDS declarations
  (interface/type/struct/enum/trait/record) and classified `leaf`
  unconditionally. Class methods use the identical Owner.member naming
  convention but are unaffected since `class` is not in TYPE_DEF_KINDS, so
  real dead methods/functions remain detected.

Applied to both the TS/WASM classifier (graph/classifiers/roles.ts,
features/structure.ts) and the native Rust classifier
(graph/classifiers/roles.rs) per the dual-engine parity requirement; verified
both engines produce identical results end-to-end against this repo's own
graph.

Fixes #1723

Impact: 6 functions changed, 5 affected
codegraph exports <file> --json (and the --unused dead-export filter it
feeds) reported zero consumers for interfaces/types that are demonstrably
imported and used elsewhere via `import type { X }`, even though
`codegraph deps <file>` correctly showed the importing file in
`importedBy`. Example: ChaContext in src/domain/graph/builder/cha.ts is
imported (type-only) by build-edges.ts and native-orchestrator.ts, but
`codegraph exports` showed consumerCount: 0.

Root cause: exportsFileImpl's per-symbol consumers query
(domain/analysis/exports.ts) only looked at kind = 'calls' edges. The
builder already emits a symbol-level `imports-type` edge for `import
type { X }` statements (source = importing file node, target = the
specific imported symbol -- see emitTypeOnlySymbolEdges in
build-edges.ts/incremental.ts), which `codegraph deps` reads from, but
the exports consumer query never looked at this edge kind. Role
classification (features/structure.ts) already includes 'imports-type'
in its fan-in formula, so `codegraph roles --role dead` was unaffected --
this was purely a gap in exports's independent consumer list.

Fix: widen the consumers query to `kind IN ('calls', 'imports-type')`,
matching the edge-kind set already used everywhere else in the codebase
for cross-file usage credit (structure.ts, graph-enrichment.ts,
boundaries.ts, dependencies.ts). No native Rust changes needed --
domain/analysis/exports.ts is pure query-layer code that reads the
already-built edges table and has no engine-specific mirror.

Deliberately does NOT add `extends`/`implements`: investigation found
those edges are resolved by symbol name only, with no file/import
scoping (buildClassHierarchyEdges in both build-edges.ts/incremental.ts
and the native emit_hierarchy_edges), so they link same-named
declarations across unrelated files (verified: this repo's own graph has
false `implements`/`extends` edges between unrelated fixture classes
across languages and a `Repository` interface in src/types.ts). Filed
as #1812. Also filed #1813 for a related but distinct gap: `import {
type X }` inline per-specifier modifiers aren't tracked as type-only in
either engine's extractor, so such X still gets no credit even after
this fix.

Added tests/integration/exports.test.ts coverage: an interface consumed
only via a symbol-level `imports-type` edge gets consumerCount >= 1 and
is excluded from --unused, while a genuinely unreferenced interface
still shows 0 consumers.

Verified against this repo's own graph: `codegraph exports
src/domain/graph/builder/cha.ts -T --json` now credits ChaContext with
2 consumers (build-edges.ts, native-orchestrator.ts). Full test suite
(201 files, 3359 tests) and lint pass clean.

docs check acknowledged: internal bug fix to exports consumer
computation, no new feature/language/CLI surface/architecture change --
README.md, CLAUDE.md, and ROADMAP.md are unaffected.

Fixes #1724

Impact: 1 functions changed, 3 affected
The CLI's `-f/--file` option is a repeatable Commander accumulator
(collectFile) that always produces a string[], even on first use. The
native composite bindings backing fn-impact (findNodesWithFanIn) and
query (fnDeps) forwarded that array straight into napi-rs functions
whose Rust signatures only accepted a single String, so any use of
-f/--file crashed with "Failed to convert JavaScript value ... into
rust type `String`" regardless of engine defaults. context worked
only because its code path never touches the native repository for
symbol lookup.

Widen the native Rust signatures (find_nodes_with_fan_in, fn_deps) to
accept Vec<String> and build an OR-of-LIKE clause for multiple files,
mirroring buildFileConditionSQL/NodeQuery.fileFilter on the JS side.
Normalize the file option before calling into the native binding on
the TS side, and thread the widened string | string[] type through
QueryOpts, fnDeps's opts, and the call chain down to
findMatchingNodes. This also makes repeated -f/--file genuinely
multi-file end to end for fn-impact/query, matching context's
existing behavior, instead of crashing on any use.

findNodesByScope shares QueryOpts but has no CLI caller and its
native binding still only accepts one file; it now takes the first
value defensively rather than crash or fail to type-check.

Fixes #1726

docs check acknowledged: pure bug fix, no new CLI options/languages/
architecture — the -f/--file "repeatable" docs are now accurate
rather than needing correction.

Impact: 14 functions changed, 34 affected
… exports

Two compounding root causes made `where --file`'s `exported` array (and
`codegraph exports`) unreliable for entire classes of exports:

1. The JS/TS export-statement handler (WASM query path, WASM walk path, and
   the mirrored native Rust extractor) only recognized `export function`,
   `export class`, `export interface`, and `export type` declarations. It
   never matched `lexical_declaration`/`variable_declaration`, so
   `export const/let/var …` was never added to the extractor's export list —
   regardless of the initializer's shape — leaving the `exported=1` DB column
   permanently unset for every exported constant in the codebase (including
   ones that happened to look "correct", like a `new Set(...)`-initialized
   constant referenced as a call argument elsewhere).

2. `where --file`'s `exported` list ignored the `exported` DB column entirely
   and computed membership from `findCrossFileCallTargets` (symbols targeted
   by a cross-file `calls` edge) — a heuristic that only "worked" by
   coincidence for symbols that happened to be passed as call arguments
   elsewhere. `codegraph exports` already had the right idea (prefer the
   `exported` column, fall back to the heuristic only for pre-migration DBs)
   but duplicated that logic locally instead of sharing it.

Fixes:
- src/extractors/javascript.ts: extract the function/class/interface/type
  kind map plus a new lexical/variable-declaration branch into a shared
  `collectExportedDeclarations`, used by both `handleExportCapture` (query
  path) and `handleExportStmt` (walk path) so they can't drift apart again.
  Declarator values are classified with the same predicate already used to
  build the matching Definition (function-valued -> kind 'function',
  literal/array/object/new-expression-valued `const` -> kind 'constant').
- crates/codegraph-core/src/extractors/javascript.rs: mirror the same fix in
  `handle_export_declaration` via a new `collect_exported_var_declarations`.
- src/db/repository/nodes.ts: add `findExportedNodesByFile`, the single
  shared implementation of "prefer exported=1, fall back to cross-file calls
  for pre-migration DBs" — re-exported through db/repository/index.ts and
  db/index.ts.
- src/domain/analysis/exports.ts: use the shared helper instead of a locally
  duplicated hasExportedCol/exportedNodesStmt implementation.
- src/domain/analysis/symbol-lookup.ts: `whereFileImpl` now uses the shared
  helper instead of `findCrossFileCallTargets` directly, so `where --file`
  and `exports` agree on what "exported" means.

Tests:
- tests/parsers/javascript.test.ts: unit coverage for bare-literal, new
  Set(...), object-literal-with-methods, and arrow-function exported consts,
  plus a non-exported-const negative case and a function/class regression
  guard for the shared-helper refactor.
- tests/engines/query-walk-parity.test.ts, tests/engines/parity.test.ts:
  cross-path/cross-engine regression cases for the same shapes.
- tests/integration/queries.test.ts: new fixture node with exported=1 but
  zero incoming edges of any kind, proving `where --file`'s exported list
  no longer depends on cross-file call presence.

No user-facing command/language/architecture changes, so README/CLAUDE.md/
ROADMAP.md are unaffected — docs check acknowledged.

Fixes #1728

Impact: 8 functions changed, 17 affected
tree-sitter-typescript's predefined_type production (string, number,
boolean, ... primitive type keywords) lexes its keyword as an anonymous
grammar token whose type string is identical to the named `string` node
type used for real string literals. Both engines matched by node type
alone, so `codegraph ast --kind string` spuriously matched bare `string`
type annotations (interface fields, parameter types, return types) as if
they were string literals.

Add an isNamed/is_named() guard scoped to the JS/TS/TSX family on both
engines: WASM via astRequiresNamedNode() in ast-analysis/rules/index.ts
feeding the shared resolveAstKind() in ast-store-visitor.ts, native via a
match guard in javascript.rs's walk_ast_nodes_depth. Genuine string,
template, and string-literal-type nodes are always named and unaffected.

No user-facing command/language/architecture changes, so README/CLAUDE.md/
ROADMAP.md are unaffected — docs check acknowledged.

Fixes #1729

Impact: 8 functions changed, 13 affected
@greptile-apps

greptile-apps Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Fixes a false-positive in codegraph ast --kind string caused by tree-sitter-typescript reusing the string "string" for both a named literal node and an anonymous keyword token inside predefined_type. Without a named-vs-anonymous guard, type annotations like name: string were captured as string literals.

  • Rust side: adds if node.is_named() to the "string" | "template_string" match arm in walk_ast_nodes_depth, leaving all other arms unchanged.
  • WASM/TS side: introduces astRequiresNamedNode(langId) (returns true for js/ts/tsx) threaded through all three createAstStoreVisitor call sites plus the parity test; resolveAstKind rejects anonymous string/template_string tokens when the flag is set, exactly mirroring the Rust arm scope.
  • Tests: adds TS, TSX, and native-engine fixture suites asserting that type-keyword tokens are excluded while genuine literals, template literals, and string-literal types are still captured.

Confidence Score: 5/5

Safe to merge — the fix is narrowly scoped, mirrored identically across both engines, and backed by new fixture tests that confirm both false-positive elimination and true-positive preservation.

Both engines receive the same targeted guard (named-vs-anonymous on exactly string/template_string), all three TS call sites and the parity test are updated, and the new test suite directly reproduces the original bug. No existing functionality is widened or removed.

tests/parsers/ast-nodes.test.ts — the two native-engine false-positive tests assert the same predicate; a suggestion to align them with the line-number checks used in the WASM suite was left inline.

Important Files Changed

Filename Overview
crates/codegraph-core/src/extractors/javascript.rs Adds if node.is_named() guard to the `"string"
src/ast-analysis/rules/index.ts Introduces astRequiresNamedNode(langId) scoped to the JS/TS/TSX grammar family, with clear documentation of why PHP is excluded (separate issue #1821).
src/ast-analysis/visitors/ast-store-visitor.ts Adds requireNamedNode param to createAstStoreVisitor and guards only string/template_string types in resolveAstKind, precisely mirroring the Rust arm scope.
src/types.ts Adds isNamed: boolean to TreeSitterNode interface matching the web-tree-sitter property getter; includes a doc-comment explaining the collision scenario.
src/ast-analysis/engine.ts Threads astRequiresNamedNode(langId) through to createAstStoreVisitor — a one-line change consistent with the other two call sites.
src/domain/wasm-worker-entry.ts Same threading of astRequiresNamedNode as engine.ts — correctly keeps the WASM worker path aligned.
src/features/ast.ts Third call site for astRequiresNamedNode, completing full coverage of all createAstStoreVisitor invocations.
tests/engines/ast-parity.test.ts Updates the parity test to pass astRequiresNamedNode(langId) so cross-engine comparison now uses the same guard on both sides.
tests/parsers/ast-nodes.test.ts Adds comprehensive TS/TSX WASM and native fixture suites; native engine's first two false-positive tests assert the same predicate on the same result set, making them indistinguishable on failure.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["astRequiresNamedNode(langId)"] --> B{langId in JS_REQUIRES_NAMED_NODE?}
    B -->|javascript / typescript / tsx| C[requireNamedNode = true]
    B -->|other| D[requireNamedNode = false]

    C --> E[createAstStoreVisitor]
    D --> E

    E --> F["resolveAstKind(node, astTypeMap, requireNamedNode)"]
    F --> G{node.type in astTypeMap?}
    G -->|No| H[return null — skip]
    G -->|Yes| I{requireNamedNode AND node type is string or template_string?}
    I -->|Yes| J{node.isNamed?}
    I -->|No| K[return mapped kind]
    J -->|false — anonymous keyword token| H
    J -->|true — real literal node| K

    subgraph Rust [Native Rust — javascript.rs]
    R1["match node.kind()"] --> R2{"string | template_string\nif node.is_named()"}
    R2 -->|named| R3[process literal]
    R2 -->|anonymous keyword| R4[skip arm — no match]
    end
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A["astRequiresNamedNode(langId)"] --> B{langId in JS_REQUIRES_NAMED_NODE?}
    B -->|javascript / typescript / tsx| C[requireNamedNode = true]
    B -->|other| D[requireNamedNode = false]

    C --> E[createAstStoreVisitor]
    D --> E

    E --> F["resolveAstKind(node, astTypeMap, requireNamedNode)"]
    F --> G{node.type in astTypeMap?}
    G -->|No| H[return null — skip]
    G -->|Yes| I{requireNamedNode AND node type is string or template_string?}
    I -->|Yes| J{node.isNamed?}
    I -->|No| K[return mapped kind]
    J -->|false — anonymous keyword token| H
    J -->|true — real literal node| K

    subgraph Rust [Native Rust — javascript.rs]
    R1["match node.kind()"] --> R2{"string | template_string\nif node.is_named()"}
    R2 -->|named| R3[process literal]
    R2 -->|anonymous keyword| R4[skip arm — no match]
    end
Loading

Reviews (4): Last reviewed commit: "Merge origin/main into fix/issue-1729-as..." | Re-trigger Greptile

Comment thread tests/parsers/ast-nodes.test.ts
Comment thread src/ast-analysis/visitors/ast-store-visitor.ts
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Codegraph Impact Analysis

8 functions changed13 callers affected across 4 files

  • setupAstVisitor in src/ast-analysis/engine.ts:458 (3 transitive callers)
  • astRequiresNamedNode in src/ast-analysis/rules/index.ts:398 (9 transitive callers)
  • resolveAstKind in src/ast-analysis/visitors/ast-store-visitor.ts:291 (1 transitive callers)
  • createAstStoreVisitor in src/ast-analysis/visitors/ast-store-visitor.ts:301 (9 transitive callers)
  • enterNode in src/ast-analysis/visitors/ast-store-visitor.ts:328 (0 transitive callers)
  • buildAstVisitor in src/domain/wasm-worker-entry.ts:593 (3 transitive callers)
  • walkAst in src/features/ast.ts:227 (3 transitive callers)
  • TreeSitterNode.isNamed in src/types.ts:789 (0 transitive callers)

@carlos-alm

Copy link
Copy Markdown
Contributor Author

@greptileai

Base automatically changed from fix/issue-1728-where-file-exported-omissions to main July 6, 2026 02:03
Resolves conflicts from stacked-branch divergence: main now has the
findNodesByScope warn()/TODO(#1815) and fn-impact --file test from
#1816, and the findExportedNodesByFile(db, file, knownSymbols) signature
plus its two call-site updates from #1820 — both merged after this
branch forked.

docs check acknowledged — conflict resolution only, no new functionality.

Impact: 5 functions changed, 15 affected
@carlos-alm carlos-alm merged commit acba6cf into main Jul 6, 2026
29 checks passed
@carlos-alm carlos-alm deleted the fix/issue-1729-ast-string-type-annotation branch July 6, 2026 02:29
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 6, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

codegraph ast --kind string misclassifies TypeScript primitive-type annotations as string literals

1 participant