Skip to content

[branch-4.1] pick some sort pr#62856

Merged
yiguolei merged 2 commits intoapache:branch-4.1from
Mryange:branch-4.1-pick-pr-dev-4.27-part-3
Apr 28, 2026
Merged

[branch-4.1] pick some sort pr#62856
yiguolei merged 2 commits intoapache:branch-4.1from
Mryange:branch-4.1-pick-pr-dev-4.27-part-3

Conversation

@Mryange
Copy link
Copy Markdown
Contributor

@Mryange Mryange commented Apr 27, 2026

What problem does this PR solve?

#59207

#62428

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

Mryange added 2 commits April 27, 2026 12:55
@Mryange
[opt](sort)use HybridSorter to choose between timsort and pdqsort (ap…
80f0ec2 
…ache#59207)

Timsort is faster than pdqsort for partially-ordered data.

Introduce a HybridSorter that samples at sort time to choose between
timsort and pdqsort.

This optimization can be disabled with enable_use_hybrid_sort; it is
enabled by default.

```
Run on (128 X 2250.02 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x64)
  L1 Instruction 32 KiB (x64)
  L2 Unified 1024 KiB (x64)
  L3 Unified 16384 KiB (x8)
Load Average: 36.25, 46.31, 138.22
--------------------------------------------------------------------------------
Benchmark                      Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------
BM_PdqSort/10000000/0        350 ms          350 ms            2 items_per_second=28.5574M/s random
BM_PdqSort/10000000/1        216 ms          216 ms            3 items_per_second=46.3077M/s ascending_saw
BM_PdqSort/10000000/2        217 ms          217 ms            3 items_per_second=46.1045M/s descending_saw
BM_PdqSort/10000000/3        274 ms          274 ms            3 items_per_second=36.4636M/s generic
BM_PdqSort/10000000/4        326 ms          326 ms            2 items_per_second=30.6559M/s random_tail
BM_PdqSort/10000000/5        346 ms          346 ms            2 items_per_second=28.8922M/s random_half
BM_PdqSort/10000000/6        230 ms          230 ms            3 items_per_second=43.5162M/s wave
BM_TimSort/10000000/0       1113 ms         1113 ms            1 items_per_second=8.98417M/s random
BM_TimSort/10000000/1       89.9 ms         89.9 ms            8 items_per_second=111.209M/s ascending_saw
BM_TimSort/10000000/2       91.0 ms         91.0 ms            8 items_per_second=109.926M/s descending_saw
BM_TimSort/10000000/3        533 ms          533 ms            1 items_per_second=18.7505M/s generic
BM_TimSort/10000000/4        228 ms          228 ms            3 items_per_second=43.7805M/s random_tail
BM_TimSort/10000000/5        559 ms          559 ms            1 items_per_second=17.8817M/s random_half
BM_TimSort/10000000/6       87.4 ms         87.4 ms            8 items_per_second=114.384M/s wave
```
@Mryange
[refine](sort) remove VSortExecExprs wrapper class (apache#62428)
8271775 
`VSortExecExprs` was a legacy wrapper class (ported from Impala) that
held
`VExprContextSPtrs _ordering_expr_ctxs` along with a dead
`_materialize_tuple`
branch. Investigation showed that FE's `SortInfo.toThrift()` never sets
`sort_tuple_slot_exprs` in `TSortInfo`, so `_materialize_tuple` is
always
`false` and the materialization code path is unreachable dead code.

This PR removes the `VSortExecExprs` class entirely and replaces all
usages
with direct `VExprContextSPtrs`.

- Delete `vsort_exec_exprs.h` and `vsort_exec_exprs.cpp`
- Replace `VSortExecExprs&` with `const VExprContextSPtrs&` in all
sorter
constructors (`FullSorter`, `TopNSorter`, `HeapSorter`,
`PartitionSorter`)
- Update all operator/sink files: `sort_sink_operator`,
  `local_merge_sort_source_operator`, `exchange_source_operator`,
  `partition_sort_sink_operator`, `viceberg_sort_writer`
- Remove dead materialization block in `Sorter::partial_sort` and
  `Sorter::_prepare_sort_columns`
- Update all affected unit tests
@Mryange Mryange requested a review from yiguolei as a code owner April 27, 2026 06:37
@hello-stephen
Copy link
Copy Markdown
Contributor

hello-stephen commented Apr 27, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange
Copy link
Copy Markdown
Contributor Author

Mryange commented Apr 27, 2026

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

hello-stephen commented Apr 27, 2026

FE UT Coverage Report

Increment line coverage 50.00% (2/4) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

hello-stephen commented Apr 27, 2026

FE Regression Coverage Report

Increment line coverage 100.00% (4/4) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Copy Markdown
Contributor

hello-stephen commented Apr 27, 2026

BE Regression && UT Coverage Report

Increment line coverage 94.63% (229/242) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.37% (26277/36817)
Line Coverage 54.40% (278987/512830)
Region Coverage 51.73% (231826/448117)
Branch Coverage 53.07% (100167/188733)

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Apr 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 27, 2026

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 27, 2026

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit 4803f79 into apache:branch-4.1 Apr 28, 2026
28 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Mryange @hello-stephen @yiguolei