⚡ Bolt: [performance improvement] Optimize D1 SQL generation

[bashandbone]

💡 What: Optimized dynamic SQL generation (build_upsert_stmt and build_delete_stmt) in the Cloudflare D1 export target to use pre-allocated buffers (String::with_capacity and Vec::with_capacity) and the write! macro.
🎯 Why: Constructing dynamic SQL with intermediate vectors (for columns/placeholders) and the format! / join operations causes redundant heap allocations and string copies. This is particularly wasteful in loops generating multiple queries per record.
📊 Impact: Considerably reduces memory allocations and heap churn during query construction, leading to improved query generation latency and greater overall throughput when building large batches of operations.
🔬 Measurement: Verify tests run via cargo test -p thread-flow --test d1_target_tests --test d1_minimal_tests --test d1_cache_integration and view measurable generation latency via cargo bench -p thread-flow --bench d1_profiling.

---

PR created automatically by Jules for task 18029450483487382053 started by @bashandbone

Summary by Sourcery

Optimize dynamic SQL construction for Cloudflare D1 export operations to reduce allocations and improve query generation performance.

Enhancements:

• Refactor D1 upsert SQL generation to build queries directly into pre-allocated strings and parameter vectors instead of using intermediate collections and string joins.
• Refactor D1 delete SQL generation to construct WHERE clauses incrementally in a pre-allocated buffer to minimize heap churn during query building.

Documentation:

• Extend Bolt performance notes with guidance on using pre-allocated strings and the write! macro for efficient dynamic SQL generation.

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[sourcery-ai]

Reviewer's Guide

Optimizes dynamic SQL generation for Cloudflare D1 export upsert and delete statements by eliminating intermediate string/Vec allocations in favor of pre-allocated buffers and direct writes, and records the performance lesson in the Bolt engineering notes.

File-Level Changes

Change	Details	Files
Optimize D1 upsert SQL generation to build the query directly into a pre-allocated String while accumulating parameters in a pre-sized Vec.	Replace intermediate columns/placeholders/update_clauses Vec with a single mutable SQL String buffer using String::with_capacity. Append column list, VALUES placeholder list, and ON CONFLICT DO UPDATE SET clauses incrementally using push_str, push, and write! while tracking comma placement via a first flag. Pre-size the params Vec based on key and value field counts and push JSON-converted key/value parameters in the same order as SQL placeholders.	crates/flow/src/targets/d1.rs
Optimize D1 delete SQL generation to construct the WHERE clause directly into a pre-allocated String alongside parameter collection.	Replace where_clauses Vec with a single SQL String buffer using String::with_capacity. Write the DELETE FROM header and WHERE predicates incrementally with write!, inserting AND separators using a first flag. Pre-size the params Vec from key length and push JSON key values in WHERE-clause order.	crates/flow/src/targets/d1.rs
Document the performance pattern and guidance for dynamic SQL generation in Bolt engineering notes.	Add a dated performance note describing overhead from intermediate Vec allocations and string joins in dynamic SQL construction. Record an action item recommending String::with_capacity and std::fmt::Write-based construction for performance-critical SQL building paths.	.jules/bolt.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've left some high level feedback:

• Consider avoiding unwrap() on the write! calls when writing into a String (e.g., by using let _ = write!(...) or expect with context), to prevent potential panics from bubbling up in these hot paths and to make the error handling intent explicit.
• The fixed String::with_capacity sizes (e.g., 128 + self.table_name.len(), 64 + self.table_name.len()) could be made a bit more data-driven (for example based on key_fields_schema.len() and value_fields_schema.len()) to better match typical statement sizes and avoid under/over-allocation as schemas evolve.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- Consider avoiding `unwrap()` on the `write!` calls when writing into a `String` (e.g., by using `let _ = write!(...)` or `expect` with context), to prevent potential panics from bubbling up in these hot paths and to make the error handling intent explicit.
- The fixed `String::with_capacity` sizes (e.g., `128 + self.table_name.len()`, `64 + self.table_name.len()`) could be made a bit more data-driven (for example based on `key_fields_schema.len()` and `value_fields_schema.len()`) to better match typical statement sizes and avoid under/over-allocation as schemas evolve.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[Copilot]

Pull request overview

This PR optimizes Cloudflare D1 export SQL statement generation by replacing intermediate vectors/string joins with direct writes into pre-allocated buffers.

Changes:

• Refactors D1 upsert SQL construction to use String::with_capacity, Vec::with_capacity, and write!.
• Refactors D1 delete SQL construction similarly to avoid intermediate clause vectors.
• Adds a Bolt performance note documenting the dynamic SQL generation optimization.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
crates/flow/src/targets/d1.rs	Optimizes build_upsert_stmt and build_delete_stmt SQL construction while preserving existing SQL output semantics.
.jules/bolt.md	Adds a performance learning entry for efficient dynamic SQL generation.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.