perf: eliminate schema_changes_alerts temp table to reduce concurrent DDL

[elazarlachkar]

Summary

Each schema_changes test was creating a __schema_changes_alerts__tmp persistent table solely as an intermediary — store_schema_test_results read from it immediately and the test assertion selected from it directly. This table served no purpose beyond being a short-lived middleman.

With many monitored sources running in parallel (e.g. 37 sources × 2 tables = 74 simultaneous CREATE OR REPLACE TABLE DDL operations on the shared ELEMENTARY schema), Snowflake's internal metadata catalog becomes overloaded. This triggered a non-actionable Snowflake fault with a 245-second retry on one query, confirmed via fault_handling_time in ACCOUNT_USAGE.QUERY_HISTORY:

Metric	Value
total_elapsed_time	246.86s
execution_time	1.24s
fault_handling_time	245.11s
partitions_scanned > partitions_total	confirms multi-attempt retry
Concurrent DDL count at fault moment	31

Fix

Run the alert query inline instead of materializing it into a temp table — matching the pattern already used by test_schema_changes_from_baseline, which has always worked this way.

• Before: CREATE OR REPLACE TABLE __schema_changes_alerts__tmp AS {alert_query} → SELECT * FROM that_table
• After: pass schema_changes_alert_query directly to store_schema_test_results and return it as the test assertion

store_schema_test_results already calls run_query(sql) internally, so it handles a raw query string just as well as a SELECT * FROM table.

Impact

• DDL operations per run: 74 → 37 (halved)
• No functional change — same query results, same caching behaviour, same test pass/fail logic
• Minimal diff: 13 lines removed, 2 lines changed

Related

• Linear: CORE-1042 (sub-issue of CORE-1041)

Test plan

• Run test_schema_changes integration test — should detect column_added, column_removed, type_changed as before
• Verify elementary_test_results is populated correctly in on_run_end
• Verify no __schema_changes_alerts__tmp tables are created in the Elementary schema

Made with Cursor

Summary by CodeRabbit

• Refactor
  • Optimized schema change detection test for improved efficiency and streamlined result handling.

[linear]

CORE-1042

[github-actions]

👋 @elazarlachkar
Thank you for raising your pull request.
Please make sure to add tests and document all user-facing changes.
You can do this by editing the docs files in the elementary repository.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1f23a0ff-059b-424c-accb-d37aff66c1e9

📥 Commits

Reviewing files that changed from the base of the PR and between 88743b5 and e48c072.

📒 Files selected for processing (1)

• macros/edr/tests/test_schema_changes.sql

---

📝 Walkthrough

Walkthrough

The schema_changes dbt test macro is simplified by removing the intermediate schema_changes_alerts temporary table creation and the schema_changes_sql string wrapping. The macro now stores results directly via schema_changes_alert_query and returns that query as the test result.

Changes

Schema Changes Test Simplification

Layer / File(s)	Summary
Direct alert query return, drop temp table macros/edr/tests/test_schema_changes.sql	Removes intermediate schema_changes_alerts temp table creation and schema_changes_sql string construction; records results directly against schema_changes_alert_query and returns it as the test result query.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

🐇 No more temp tables, no more SQL strings,
The schema check skips a few extra things.
Direct and clean, the query returns,
A rabbit approves what the diff confirms!
Less is more when the logic is right. 🌿

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: eliminating a temporary table to reduce concurrent DDL operations, which directly matches the core objective of the PR.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch core-1042-level-1-eliminate-__schema_changes_alerts__tmp-table-run

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}