-
Notifications
You must be signed in to change notification settings - Fork 1
Cooling-stage clock sweep (batched sweep + HOT/COOL evictor + strategy removal) #34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
0acf4b4
891a04f
1bcc967
f717334
2d13893
54a7350
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| Diagnostics: | ||
| MissingIncludes: None | ||
|
Comment on lines
+1
to
+2
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This file should not be committed to the repository. The project's root Additionally, note the paths are developer-specific: |
||
| InlayHints: | ||
| Enabled: true | ||
| ParameterNames: true | ||
| DeducedTypes: true | ||
| CompileFlags: | ||
| CompilationDatabase: build/ # Search build/ directory for compile_commands.json | ||
| Remove: [ -Werror ] | ||
| Add: | ||
| - -DDEBUG | ||
| - -DLOCAL | ||
| - -DPGDLLIMPORT= | ||
| - -DPIC | ||
| - -O2 | ||
| - -Wall | ||
| - -Wcast-function-type | ||
| - -Wconversion | ||
| - -Wdeclaration-after-statement | ||
| - -Wendif-labels | ||
| - -Werror=vla | ||
| - -Wextra | ||
| - -Wfloat-equal | ||
| - -Wformat-security | ||
| - -Wimplicit-fallthrough=3 | ||
| - -Wmissing-format-attribute | ||
| - -Wmissing-prototypes | ||
| - -Wno-format-truncation | ||
| - -Wno-sign-conversion | ||
| - -Wno-stringop-truncation | ||
| - -Wno-unused-const-variable | ||
| - -Wpointer-arith | ||
| - -Wshadow | ||
| - -Wshadow=compatible-local | ||
| - -fPIC | ||
| - -fexcess-precision=standard | ||
| - -fno-strict-aliasing | ||
| - -fvisibility=hidden | ||
| - -fwrapv | ||
| - -g | ||
| - -std=c11 | ||
| - -I. | ||
| - -I../../../../src/include | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,156 @@ | ||
| # HOT Indexed Updates — GDB breakpoints for code review | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This entire file is a personal debugging aid and must be removed from the patch. Committing a |
||
| # | ||
| # Usage: gdb -x .gdbinit <postgres-binary> | ||
|
Comment on lines
+1
to
+3
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This Worse, the file is orphaned from the feature it targets: the breakpoints reference a "HOT indexed updates" implementation that is NOT present in this tree. Symbols such as |
||
| # Or from gdb: source .gdbinit | ||
| # | ||
| # These breakpoints cover the major code paths introduced or modified by | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ASCII-only is required in PostgreSQL source and diffs. This comment block uses non-ASCII em-dashes (e.g., "HOT Indexed Updates —", "returns Bitmapset" lines and the "—" separators throughout). Another reason this file cannot be committed as-is. |
||
| # the HOT indexed updates patch series. They are organized by subsystem | ||
| # to make it easy to enable/disable groups during debugging. | ||
| # | ||
| # Tip: to skip to a specific subsystem, disable all then enable selectively: | ||
| # disable breakpoints | ||
| # enable 1 2 3 # just the update-decision group | ||
|
|
||
| # ========================================================================= | ||
| # 1. UPDATE DECISION — heap_update() HOT/HOT-indexed/non-HOT choice | ||
| # src/backend/access/heap/heapam.c | ||
| # ========================================================================= | ||
|
|
||
| # Main entry: heap_update | ||
| break heapam.c:3210 | ||
|
Comment on lines
+19
to
+20
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hardcoded file:line breakpoints are inaccurate and brittle. For example, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Breakpoints anchored to hardcoded source line numbers (heapam.c:3210/4019/4024/4033/4101/4147, heapam_indexscan.c:182/250/297, indexam.c:299, execIndexing.c:370, pruneheap.c:1287/1802/1836/1863/2936) are inherently fragile: they silently drift as surrounding code changes and will land on the wrong statements. Since this file should not be committed at all (see above), this is moot, but even as a private aid, function-name breakpoints are far more robust than line-number ones. |
||
|
|
||
| # HOT decision block: pure HOT vs HOT indexed vs non-HOT | ||
| # Line 4019: pure HOT (no indexed columns changed) | ||
| # Line 4024: HOT indexed path (non-catalog, some indexed columns changed) | ||
| # Line 4031: predict augmented tuple size | ||
| # Line 4033: size+space check before creating augmented tuple | ||
| break heapam.c:4019 | ||
| break heapam.c:4024 | ||
| break heapam.c:4033 | ||
|
|
||
| # Set HEAP_INDEXED_UPDATED flag on new tuple before page insertion | ||
| break heapam.c:4101 | ||
|
|
||
| # Restore HEAP_INDEXED_UPDATED on old tuple (only if it previously had it) | ||
| break heapam.c:4147 | ||
|
|
||
| # ========================================================================= | ||
| # 2. TUPLE CREATION — building the augmented tuple with embedded bitmap | ||
| # src/backend/access/heap/heapam.c | ||
| # ========================================================================= | ||
|
|
||
| # Predict augmented tuple size (returns 0 if t_hoff would overflow) | ||
| break heap_hot_indexed_tuple_size | ||
|
|
||
| # Create augmented tuple with embedded modified-column bitmap | ||
| break heap_hot_indexed_create_tuple | ||
|
|
||
| # Serialize Bitmapset into raw bytes in tuple header | ||
| break heap_hot_indexed_serialize_bitmap | ||
|
|
||
| # ========================================================================= | ||
| # 3. BITMAP UTILITIES — raw bitmap operations for chain following | ||
| # src/backend/access/heap/heapam.c | ||
| # ========================================================================= | ||
|
|
||
| # Compute raw bitmap byte size from natts | ||
| break heap_hot_indexed_bitmap_raw_size | ||
|
|
||
| # Check if tuple header has room for bitmap between null bitmap and data | ||
| break heap_hot_indexed_has_bitmap_space | ||
|
|
||
| # Read HOT indexed bitmap from tuple header (returns Bitmapset) | ||
| break heap_hot_indexed_read_bitmap | ||
|
|
||
| # Fast overlap check: does tuple's raw bitmap overlap with indexed_attrs? | ||
| break heap_hot_indexed_bitmap_overlaps_raw | ||
|
|
||
| # OR a tuple's raw bitmap into an accumulator buffer | ||
| break heap_hot_indexed_bitmap_or_raw | ||
|
|
||
| # Check if accumulated raw bitmap overlaps with indexed_attrs | ||
| break heap_hot_indexed_accum_overlaps | ||
|
|
||
| # Merge bitmaps from dead tuples into a target tuple on the page | ||
| break heap_hot_indexed_merge_bitmaps_raw | ||
|
|
||
| # Deserialize raw bytes back to Bitmapset | ||
| break heap_hot_indexed_deserialize_bitmap | ||
|
|
||
| # ========================================================================= | ||
| # 4. INDEX SCAN — HOT chain following with stale-entry detection | ||
| # src/backend/access/heap/heapam_indexscan.c | ||
| # ========================================================================= | ||
|
|
||
| # Main HOT chain search with indexed update awareness | ||
| break heap_hot_search_buffer | ||
|
|
||
| # Redirect-with-data: initialize bitmap accumulator from collapsed redirect | ||
| break heapam_indexscan.c:182 | ||
|
|
||
| # Accumulate bitmap from INDEXED_UPDATED tuple in chain | ||
| break heapam_indexscan.c:250 | ||
|
|
||
| # Stale entry detection: accumulated bitmap overlaps this index's attrs | ||
| break heapam_indexscan.c:297 | ||
|
|
||
| # ========================================================================= | ||
| # 5. INDEX SCAN SETUP — indexed_attrs bitmap computation | ||
| # src/backend/access/index/indexam.c | ||
| # ========================================================================= | ||
|
|
||
| # Compute indexed_attrs for HOT indexed update chain following | ||
| break indexam.c:299 | ||
|
|
||
| # ========================================================================= | ||
| # 6. INDEX INSERTION — skip unchanged indexes for HOT indexed updates | ||
| # src/backend/executor/execIndexing.c | ||
| # ========================================================================= | ||
|
|
||
| # Entry: insert/update index tuples | ||
| break ExecInsertIndexTuples | ||
|
|
||
| # Index skip decision: skip indexes whose attrs don't overlap modified set | ||
| break execIndexing.c:370 | ||
|
|
||
| # ========================================================================= | ||
| # 7. PRUNING — chain collapsing and redirect-with-data | ||
| # src/backend/access/heap/pruneheap.c | ||
| # ========================================================================= | ||
|
|
||
| # Main prune function | ||
| break heap_page_prune_and_freeze | ||
|
|
||
| # Per-chain pruning entry | ||
| break heap_prune_chain | ||
|
|
||
| # Chain collapsing: collect bitmaps from dead INDEXED_UPDATED intermediates | ||
| break pruneheap.c:1802 | ||
|
|
||
| # OR dead tuple bitmaps into combined bitmap | ||
| break pruneheap.c:1836 | ||
|
|
||
| # Record redirect-with-data for execute phase | ||
| break pruneheap.c:1863 | ||
|
|
||
| # Execute phase: apply redirect-with-data entries on the page | ||
| break pruneheap.c:1287 | ||
|
|
||
| # ========================================================================= | ||
| # 8. WAL REPLAY — recovery of HOT indexed updates | ||
| # src/backend/access/heap/heapam_xlog.c | ||
| # ========================================================================= | ||
|
|
||
| # WAL replay for XLOG_HEAP2_INDEXED_UPDATE | ||
| break heap_xlog_indexed_update | ||
|
|
||
| # ========================================================================= | ||
| # 9. WAL LOGGING — writing HOT indexed update records | ||
| # src/backend/access/heap/heapam.c | ||
| # ========================================================================= | ||
|
|
||
| # WAL logging for heap updates (handles indexed_update flag) | ||
| break log_heap_update | ||
|
|
||
| # Serialize redirect-with-data into WAL record (pruneheap.c) | ||
| break pruneheap.c:2936 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Node modules | ||
| scripts/ai-review/node_modules/ | ||
| # Note: package-lock.json should be committed for reproducible CI/CD builds | ||
|
|
||
| # Logs | ||
| scripts/ai-review/cost-log-*.json | ||
| scripts/ai-review/*.log | ||
|
|
||
| # OS files | ||
| .DS_Store | ||
| Thumbs.db | ||
|
|
||
| # Editor files | ||
| *.swp | ||
| *.swo | ||
| *~ | ||
| .vscode/ | ||
| .idea/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This
.clangdfile is personal editor/LSP tooling configuration and should not be part of a patch destined for pgsql-hackers/commitfest. Per PostgreSQL patch hygiene, a patch must be a minimal diff containing only what the stated change requires; committing local IDE/tooling config (this file, plus.gdbinitandpg-aliases.shin this same change) is unrelated churn and a reliable rejection reason. The PostgreSQL tree does not carry.clangd. If you need it locally, put it in.git/info/excludeor your global gitignore rather than committing it. Remove this file from the patch.Also note the hardcoded relative include path
-I../../../../src/include(line 43) is brittle and workstation/subdirectory-specific, which further confirms this is a local artifact, not a tree-wide config.