Skip to content

Preserve bool dtype in SingleAmountTaxScale.calc()#508

Merged
MaxGhenis merged 1 commit into
masterfrom
codex/bool-bracket-dtype
Jul 6, 2026
Merged

Preserve bool dtype in SingleAmountTaxScale.calc()#508
MaxGhenis merged 1 commit into
masterfrom
codex/bool-bracket-dtype

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

Summary

Fixes the framework-level root cause of PolicyEngine/policyengine-us#8780.

SingleAmountTaxScale.calc() returned an int64 0/1 array for boolean single_amount brackets (amount_unit: bool with amount: true/false), not a boolean array. Applying ~ (logical NOT) to that int result performed a bitwise negation (~1 == -2, ~0 == -1, both truthy) instead of a logical one, silently producing wrong values in consuming formulas. The bug was masked whenever the surrounding expression happened to &/| the result with a genuine bool term, so it shipped undetected until a masking term was removed during cleanup in policyengine-us (#8763).

Root cause

In single_amount_tax_scale.py, the amounts array was built as:

guarded_amounts = numpy.array([0] + self.amounts + [0])

Even when self.amounts are Python booleans ([True, False]), prepending/appending the Python int 0 sentinels forces numpy to infer int64 for the whole array, dropping the boolean dtype. ~int64 is then a bitwise negation.

Fix

Build the amounts array from the bracket amounts first, so numpy infers their natural dtype (bool for boolean brackets, int/float otherwise), then pad the two out-of-range sentinel cells with a zero of that same dtype (False for bool, 0 for numeric):

if len(self.amounts) == 0:
    guarded_amounts = numpy.array([0, 0])
else:
    amounts = numpy.asarray(self.amounts)
    sentinel = numpy.zeros(1, dtype=amounts.dtype)
    guarded_amounts = numpy.concatenate([sentinel, amounts, sentinel])

Why this is low-risk (no ripple into numeric consumers)

  • Numeric int/float brackets are untouched — same dtype, same values (verified: int stays int64, float stays float64).
  • Empty scales still return int64 [0] (historical behavior preserved via the len == 0 guard).
  • The only operator whose semantics differ between an int 0/1 array and a bool array is ~ — which is exactly the bug being fixed. Every other combinator used by consuming formulas (&, |, where(), ==, *, +) behaves identically on bool and int 0/1, because numpy auto-promotes bool in arithmetic. So switching boolean brackets from int64 to bool is strictly a correctness improvement with no numeric regression.

With this fix, the per-site .astype(bool) workarounds already merged in policyengine-us (#8919 and the aca_required_contribution_percentage fix) become redundant but remain harmless; they are intentionally left in place.

Tests

Added to tests/core/tax_scales/test_single_amount_tax_scale.py:

  • test_calc_preserves_boolean_dtype — bool brackets return a bool array.
  • test_calc_boolean_result_supports_logical_not~ on the result is logical NOT ([False, True]), the exact regression guarded.
  • test_calc_preserves_integer_dtype / test_calc_preserves_float_dtype — numeric dtypes and values unchanged.
  • test_calc_out_of_range_bool_returns_false_sentinel — out-of-range sentinel is False, not int 0.
  • test_calc_empty_scale_stays_int — empty scale still returns int64 [0].
  • test_calc_boolean_dtype_end_to_end_via_parameter_scale — full ParameterScaleget_at_instantSingleAmountTaxScale path with an amount_unit: bool scale (mirroring how policyengine-us declares these), asserting bool dtype and correct ~.

Checks run (Python 3.13, editable install in a clean worktree):

  • tests/core/tax_scales/test_single_amount_tax_scale.py: 12 passed (5 pre-existing + 7 new).
  • tests/core/tax_scales/ + tests/core/parameters/: 89 passed.
  • Full suite tests/: 615 passed, 4 skipped, 1 xfailed — no regressions.
  • ruff format and ruff check clean on both changed files.

Changelog fragment: changelog.d/bool-bracket-dtype.fixed.md.

Boolean single_amount brackets (amount_unit: bool, amount: true/false)
previously returned an int64 0/1 array from .calc() because the sentinel
padding [0] + self.amounts + [0] coerced the whole array to int64. Applying
~ (logical NOT) to that int array performed a bitwise negation (~1 == -2,
~0 == -1, both truthy) instead of logical NOT, silently producing wrong
results in consuming formulas.

Build the amounts array from the bracket amounts first so numpy infers their
natural dtype, then pad the out-of-range sentinel cells with a zero of that
same dtype (False for bool, 0 for numeric). Numeric int/float brackets are
unchanged; empty scales still return int64 [0].

Fixes PolicyEngine/policyengine-us#8780

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Lead review: root cause exactly identified — the Python int sentinels in calc() coerced boolean bracket amounts to int64, making ~ bitwise (both ~0 and ~1 truthy) instead of logical. Fix preserves amounts dtype with same-dtype sentinel padding and keeps the empty-scale int64 fallback. Numeric brackets are byte-identical before/after; verified via full core suite (615 passed) and a 1,004-test pe-us consumer sweep with this branch installed. The three previously-fixed pe-us .astype(bool) sites stay as harmless belt-and-suspenders.

@MaxGhenis MaxGhenis marked this pull request as ready for review July 6, 2026 18:51
@MaxGhenis MaxGhenis merged commit 25111c1 into master Jul 6, 2026
22 checks passed
@MaxGhenis MaxGhenis deleted the codex/bool-bracket-dtype branch July 6, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant