Antalya 26.3: DataLakeCatalog: avoid full catalog read for UNKNOWN_TABLE typo hints#1675
Open
zvonand wants to merge 1 commit intoantalya-26.3from
Open
Antalya 26.3: DataLakeCatalog: avoid full catalog read for UNKNOWN_TABLE typo hints#1675zvonand wants to merge 1 commit intoantalya-26.3from
UNKNOWN_TABLE typo hints#1675zvonand wants to merge 1 commit intoantalya-26.3from
Conversation
…n_drop DataLakeCatalog: avoid full catalog read for `UNKNOWN_TABLE` typo hints
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Avoid scanning the whole remote data lake catalog for “Maybe you meant …” table hints when
show_data_lake_catalogs_in_system_tablesis disabled (ClickHouse#100452 by @alsugiliazova).Cherry-picked from ClickHouse#100452.
Documentation entry for user-facing changes
When
show_data_lake_catalogs_in_system_tables = 0, the server must not implicitly scan the whole remote data lake catalog.Previously, building the “Maybe you meant …” hint for a missing table in a
DataLakeCatalogdatabase still calledgetAllTableNames()→DatabaseDataLake::getTablesIterator(), which lists the entire catalog and loads per-table metadata—heavy work and can OOM on large catalogs, only to enrich an error message.This change makes
TableNameHints::getAllRegisteredNames()return an empty name list for data lake catalogs when that setting is off, so hint generation does not trigger a full catalog listing.Query examples
Drop non-existent table:
Previously (undesired): server could answer with
UNKNOWN_TABLEand a “Maybe you meant …” suggestion after scanning catalog names:After the fix:
UNKNOWN_TABLEwithout hint when the setting is0:Optional follow-up
If local hints are skipped for data lake + setting
0,getHintForTablecan still fall back togetExtendedHintForTable, which only scans non–data-lake databases. In edge cases a suggestion could point at another database’s table. If that is undesirable, a follow-up could skip extended hints under the same condition as local enumeration.