Skip to content

⚡ Bolt: [performance improvement] Defer PathBuf allocations during Tarjan's SCC DFS#265

Open
bashandbone wants to merge 1 commit into
mainfrom
bolt/optimize-tarjan-dfs-8749202397797120216
Open

⚡ Bolt: [performance improvement] Defer PathBuf allocations during Tarjan's SCC DFS#265
bashandbone wants to merge 1 commit into
mainfrom
bolt/optimize-tarjan-dfs-8749202397797120216

Conversation

@bashandbone
Copy link
Copy Markdown
Contributor

@bashandbone bashandbone commented May 26, 2026

💡 What: Defer PathBuf heap allocations during Tarjan's strongly connected components algorithm (tarjan_dfs).
🎯 Why: v.to_path_buf() was called multiple times per node and during cycle checking for dependencies, causing O(E) unnecessary heap allocations during graph traversal.
📊 Impact: Significantly reduces memory allocations and churn in the invalidation path, making dependency graph processing much faster for large projects.
🔬 Measurement: Observe reduction in memory allocation during tests like cargo test -p thread-flow --test invalidation_tests.


PR created automatically by Jules for task 8749202397797120216 started by @bashandbone

Summary by Sourcery

Enhancements:

  • Reduce PathBuf cloning and lookups in Tarjan's SCC DFS to lower heap allocations and improve dependency graph invalidation performance.

By passing borrowed `&Path` values to `.get()`, `.get_mut()`,
and `.contains()` queries on `RapidMap` and `RapidSet`, we avoid
triggering `v.to_path_buf()` multiple times for each node dependency
during Tarjan's strongly connected components algorithm.

This saves O(E) heap allocations during invalidation graph traversals.

Co-authored-by: bashandbone <89049923+bashandbone@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 26, 2026 17:52
@google-labs-jules
Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented May 26, 2026

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Defers repeated PathBuf allocations in Tarjan's SCC DFS by reusing a single PathBuf per node and switching map lookups from owned PathBuf keys to borrowed Path keys, reducing heap allocations during graph traversal.

Sequence diagram for tarjan_dfs PathBuf allocation reuse

sequenceDiagram
    participant Caller
    participant InvalidationDetector
    participant TarjanState

    Caller->>InvalidationDetector: tarjan_dfs(v, state, sccs)
    InvalidationDetector->>InvalidationDetector: v_buf = v.to_path_buf()
    InvalidationDetector->>TarjanState: indices.insert(v_buf.clone, index)
    InvalidationDetector->>TarjanState: lowlinks.insert(v_buf.clone, index)
    InvalidationDetector->>TarjanState: stack.push(v_buf.clone)
    InvalidationDetector->>TarjanState: on_stack.insert(v_buf)

    loop for each dependency
        InvalidationDetector->>TarjanState: lowlinks.get(v)
        InvalidationDetector->>TarjanState: indices.get(v)
    end
Loading

File-Level Changes

Change Details Files
Defer PathBuf allocations and reuse a single buffer during Tarjan DFS to avoid repeated heap allocations per node.
  • Introduce a local PathBuf (v_buf) created once from the input Path and reuse it for initial indices, lowlinks, stack, and on_stack inserts by cloning as needed.
  • Change subsequent lowlinks and indices lookups to use the borrowed Path key (v) instead of reconstructing a new PathBuf each time.
  • Ensure the Tarjan root-node check (v_index and v_lowlink) also uses the borrowed Path key, aligning all map accesses with the new allocation strategy.
crates/flow/src/incremental/invalidation.rs

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • You can avoid one of the PathBuf clones during initialization by reusing the owned v_buf when inserting into lowlinks (e.g., indices.insert(v_buf.clone(), index); lowlinks.insert(v_buf, index);) and only cloning again when pushing to stack/on_stack if needed.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- You can avoid one of the `PathBuf` clones during initialization by reusing the owned `v_buf` when inserting into `lowlinks` (e.g., `indices.insert(v_buf.clone(), index); lowlinks.insert(v_buf, index);`) and only cloning again when pushing to `stack`/`on_stack` if needed.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves performance in the incremental invalidation path by reducing repeated PathBuf heap allocations inside Tarjan’s strongly connected components DFS (tarjan_dfs) while traversing the dependency graph.

Changes:

  • Allocate PathBuf for the current node (v) once per DFS call and reuse it for inserts/pushes.
  • Replace repeated &v.to_path_buf() lookups with borrowed &Path lookups into RapidMap<PathBuf, _> (and similar structures), avoiding per-lookup allocations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants