Skip to content

fix: Incorrect behavior for FILTER on NULLs#22068

Open
neilconway wants to merge 2 commits intoapache:mainfrom
neilconway:neilc/fix-agg-filter-null-handling
Open

fix: Incorrect behavior for FILTER on NULLs#22068
neilconway wants to merge 2 commits intoapache:mainfrom
neilconway:neilc/fix-agg-filter-null-handling

Conversation

@neilconway
Copy link
Copy Markdown
Contributor

@neilconway neilconway commented May 7, 2026

Which issue does this PR close?

Rationale for this change

In the grouping code, accumulate_multiple and accumulate_indices take a BooleanArray parameter, which has the result of the aggregate's FILTER clause (if any). Both functions only consider the value bits of the array, not the NULL bitmap, which means they consider NULL filter results to be effectively true, not false.

What changes are included in this PR?

  • Fix NULL handling in accumulate_multiple and accumulate_indices
  • Refactor accumulate_multiple to be more readable and make use of NullBuffer::union_many
  • Introduce a new helper, filter_to_validity
  • Optimize filter_to_nulls to use filter_to_validity and avoid constructing an unnecessary intermediate NullBuffer
  • Add unit tests for NULL handling in accumulate_multiple and accumulate_indices
  • Add SLT tests with SQL repros for both code paths

Are these changes tested?

Yes, with new tests added.

Are there any user-facing changes?

This changes query behavior for the affected (buggy) queries.

This PR also changes the signature of filter_to_nulls, which is technically a public API.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels May 7, 2026
@neilconway
Copy link
Copy Markdown
Contributor Author

If we want to backport this, let me know -- I can send a separate PR that is more minimal and doesn't use union_many.

@mbutrovich mbutrovich self-requested a review May 7, 2026 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FILTER does not handle NULLs correctly

1 participant