Skip to content

Strip trailing FORMAT clauses from SQL queries before sending#11

Merged
yosriady merged 2 commits into
mainfrom
claude/trusting-shannon-dirsml
Jun 17, 2026
Merged

Strip trailing FORMAT clauses from SQL queries before sending#11
yosriady merged 2 commits into
mainfrom
claude/trusting-shannon-dirsml

Conversation

@yosriady

@yosriady yosriady commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes a bug where SQL queries with trailing FORMAT clauses would fail when submitted to the query API. The API wraps user queries in a paginating subquery with its own FORMAT JSON clause, but ClickHouse forbids FORMAT clauses inside subqueries, causing 400 errors.

Changes

  • New module src/lib/sql.ts: Implements stripTrailingFormatClause() function that safely removes trailing FORMAT <name> clauses and semicolons from SQL statements

    • Handles edge cases like string literals, quoted identifiers, line/block comments, and nested subqueries
    • Uses a masking approach to avoid false positives (e.g., formatDateTime() function calls, columns named format)
    • Preserves the original query if there's nothing to strip
  • Comprehensive test suite test/lib/sql.test.ts: 157 lines of tests covering:

    • Basic FORMAT clause removal in various formats (CSV, JSON, CSVWithNames, etc.)
    • Trailing semicolon handling (bare, before/after FORMAT, multiple)
    • False positive prevention (formatDateTime, column aliases, nested FORMAT)
    • Quote and comment awareness
    • Integration test verifying wrapped queries have no FORMAT inside parentheses
  • Updated src/commands/query.ts: Applies stripTrailingFormatClause() to user input before posting to the API, with explanatory comment

Implementation Details

The stripTrailingFormatClause() function uses a two-pass approach:

  1. First pass masks string literals, quoted identifiers, and comments to create a safe version for pattern matching
  2. Second pass iteratively strips trailing semicolons and FORMAT clauses from the masked version, using indices to trim the original
  3. Cleans up dangling whitespace while preserving genuine comments

https://claude.ai/code/session_01D7VzXR4q2UWHrYdU2139MS


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.

The Formo query API wraps submitted SQL in a paginating subquery with its
own FORMAT JSON:

    SELECT * FROM (<query>) LIMIT 100 FORMAT JSON

ClickHouse forbids a FORMAT clause inside a subquery, so a user query that
ends in its own FORMAT clause (e.g. `... FORMAT CSV`) produced a nested
FORMAT and a 400 from the API. A trailing FORMAT can never take effect
through this endpoint anyway -- the outer FORMAT JSON always wins, and
output shaping is the CLI's `--format` job.

Add stripTrailingFormatClause in src/lib/sql.ts to remove a trailing
top-level FORMAT clause and trailing semicolons before the query is sent.
The scan is aware of string literals, quoted identifiers, and comments, and
is anchored to the end of the statement, so it never touches a FORMAT nested
in a subquery or an identifier/function such as formatDateTime(...).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D7VzXR4q2UWHrYdU2139MS

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a utility function stripTrailingFormatClause to strip trailing FORMAT clauses and semicolons from SQL queries before sending them to the ClickHouse API, preventing 400 errors caused by nested FORMAT clauses in subqueries. The feedback suggests extending the comment-masking logic to support # as a single-line comment character (supported by ClickHouse for MySQL compatibility) and adding corresponding test coverage.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/lib/sql.ts Outdated
Comment thread test/lib/sql.test.ts
ClickHouse accepts `#` and `#!` single-line comments (MySQL
compatibility). maskLiterals only handled `--` and block comments, so a
`#` comment ending in a FORMAT-like token could be wrongly stripped, and a
real trailing FORMAT followed by a `#` comment was missed. Fold `#` into
the line-comment branch and cover both directions with tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D7VzXR4q2UWHrYdU2139MS
@yosriady yosriady merged commit f0855d9 into main Jun 17, 2026
6 checks passed
@yosriady yosriady deleted the claude/trusting-shannon-dirsml branch June 17, 2026 04:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants