Preserve bool dtype in SingleAmountTaxScale.calc()#508
Merged
Conversation
Boolean single_amount brackets (amount_unit: bool, amount: true/false) previously returned an int64 0/1 array from .calc() because the sentinel padding [0] + self.amounts + [0] coerced the whole array to int64. Applying ~ (logical NOT) to that int array performed a bitwise negation (~1 == -2, ~0 == -1, both truthy) instead of logical NOT, silently producing wrong results in consuming formulas. Build the amounts array from the bracket amounts first so numpy infers their natural dtype, then pad the out-of-range sentinel cells with a zero of that same dtype (False for bool, 0 for numeric). Numeric int/float brackets are unchanged; empty scales still return int64 [0]. Fixes PolicyEngine/policyengine-us#8780 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Author
|
Lead review: root cause exactly identified — the Python int sentinels in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the framework-level root cause of PolicyEngine/policyengine-us#8780.
SingleAmountTaxScale.calc()returned an int640/1array for booleansingle_amountbrackets (amount_unit: boolwithamount: true/false), not a boolean array. Applying~(logical NOT) to that int result performed a bitwise negation (~1 == -2,~0 == -1, both truthy) instead of a logical one, silently producing wrong values in consuming formulas. The bug was masked whenever the surrounding expression happened to&/|the result with a genuine bool term, so it shipped undetected until a masking term was removed during cleanup in policyengine-us (#8763).Root cause
In
single_amount_tax_scale.py, the amounts array was built as:Even when
self.amountsare Python booleans ([True, False]), prepending/appending the Python int0sentinels forces numpy to inferint64for the whole array, dropping the boolean dtype.~int64is then a bitwise negation.Fix
Build the amounts array from the bracket amounts first, so numpy infers their natural dtype (bool for boolean brackets, int/float otherwise), then pad the two out-of-range sentinel cells with a zero of that same dtype (
Falsefor bool,0for numeric):Why this is low-risk (no ripple into numeric consumers)
int64, float staysfloat64).int64 [0](historical behavior preserved via thelen == 0guard).0/1array and a bool array is~— which is exactly the bug being fixed. Every other combinator used by consuming formulas (&,|,where(),==,*,+) behaves identically on bool and int 0/1, because numpy auto-promotes bool in arithmetic. So switching boolean brackets from int64 to bool is strictly a correctness improvement with no numeric regression.With this fix, the per-site
.astype(bool)workarounds already merged in policyengine-us (#8919 and theaca_required_contribution_percentagefix) become redundant but remain harmless; they are intentionally left in place.Tests
Added to
tests/core/tax_scales/test_single_amount_tax_scale.py:test_calc_preserves_boolean_dtype— bool brackets return aboolarray.test_calc_boolean_result_supports_logical_not—~on the result is logical NOT ([False, True]), the exact regression guarded.test_calc_preserves_integer_dtype/test_calc_preserves_float_dtype— numeric dtypes and values unchanged.test_calc_out_of_range_bool_returns_false_sentinel— out-of-range sentinel isFalse, not int0.test_calc_empty_scale_stays_int— empty scale still returnsint64 [0].test_calc_boolean_dtype_end_to_end_via_parameter_scale— fullParameterScale→get_at_instant→SingleAmountTaxScalepath with anamount_unit: boolscale (mirroring how policyengine-us declares these), asserting bool dtype and correct~.Checks run (Python 3.13, editable install in a clean worktree):
tests/core/tax_scales/test_single_amount_tax_scale.py: 12 passed (5 pre-existing + 7 new).tests/core/tax_scales/+tests/core/parameters/: 89 passed.tests/: 615 passed, 4 skipped, 1 xfailed — no regressions.ruff formatandruff checkclean on both changed files.Changelog fragment:
changelog.d/bool-bracket-dtype.fixed.md.