Skip to content

UK: household-wealth imputation stage with cash / S&S ISA split#181

Draft
vahid-ahmadi wants to merge 2 commits into
mainfrom
uk-wealth-isa-imputation
Draft

UK: household-wealth imputation stage with cash / S&S ISA split#181
vahid-ahmadi wants to merge 2 commits into
mainfrom
uk-wealth-isa-imputation

Conversation

@vahid-ahmadi

Copy link
Copy Markdown

What

Adds a WAS-donor household_wealth source stage — a declarative manifest (uk/wealth_source_stages.json) plus a plan module (uk/wealth_imputation.py) mirroring uk_bus_plan/us_plan — that imputes UK household wealth holdings and surfaces cash_isa and stocks_and_shares_isa as standalone outputs. The investment-ISA component is also fold_into corporate_wealth for back-compatibility.

Why

Closes #180. The archived UK-data wealth imputation summed investment ISAs into corporate_wealth (so they weren't separable) and never represented cash ISAs, leaving ISA balances unusable. Model-side input variables added in PolicyEngine/policyengine-uk#1791.

For review (@MaxGhenis)

  • Draft. The executable WAS donor transform is injected by the build runtime — the manifest is declarative by design (no stubs/loaders), so this PR declares the stage contract, not the loader.
  • Placement: the UK build module (uk/__init__.py, bus imputation, geography) currently lives on uk-bus-fare-imputation-calibration, not main. I added a minimal uk/__init__.py exporting only the wealth symbols; this will need reconciling/stacking with that branch when it lands — guidance welcome.
  • WAS mapping is documented in the manifest notes: cash_isaDVCISAVR8 (not mapped in the legacy imputation), stocks_and_shares_isaDVIISAVR8.
  • Tests: test_uk_wealth.py (manifest validity, plan assembly + donor citations, ISA split present/nonnegative, fold-into corporate_wealth), mirroring test_uk_bus.py. Passing locally.

Declare a WAS-donor `household_wealth` source stage (manifest + plan module,
mirroring the bus imputation pattern) that surfaces `cash_isa` and
`stocks_and_shares_isa` as standalone outputs, with the investment-ISA
component also folded into `corporate_wealth` for back-compatibility.

Addresses the gap from the archived UK data wealth imputation, where investment
ISAs were summed into corporate_wealth (not separable) and cash ISAs were never
represented. The matching model-side input variables are in
PolicyEngine/policyengine-uk#1791.

Draft: the executable WAS donor transform is injected by the build runtime (not
in this manifest by design); requesting review on stage placement given the UK
build module (bus, geography) currently lives on uk-bus-fare-imputation-calibration.

Closes #180

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vahid-ahmadi vahid-ahmadi requested a review from MaxGhenis June 24, 2026 12:28
The uk/ build package is a spec-only country package (JSON resources listed in
country_package.json; runtime Python lives in shared modules). The first cut
added a plan module and __init__.py into uk/, violating the spec-only contract.

- Drop uk/wealth_imputation.py and uk/__init__.py.
- Register wealth_source_stages.json in uk/country_package.json resources.
- Rewrite test_uk_wealth.py to load the manifest via the shared
  load_source_manifest runtime and assert the cash/S&S ISA split, the
  fit+clip operations, and the fold into corporate_wealth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@MaxGhenis MaxGhenis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right direction, and the manifest passes the real load_source_manifest validator (operation vocabulary, nonnegative_outputs ⊆ outputs with net_financial_wealth correctly excluded, no incumbent deps) with meaningful tests. A few things before this is mergeable (agent-assisted review):

  1. The description is stale. It describes a Python uk/wealth_imputation.py plan module mirroring us_plan/uk_bus_plan and a stacking concern against the bus-fare branch. The head commit made this spec-only — the actual diff is wealth_source_stages.json + a country_package.json resource line + contract tests, it applies cleanly to main, and touches no __init__.py. Please rewrite the body to match what's here.

  2. As merged this is inert — nothing loads it. us_runtime wires US_SOURCE_MANIFEST = load_source_manifest(...) and us_plan requires a transform per stage; uk_runtime has no equivalent and doesn't reference source_manifest, source_stages, or wealth at all. So this declares the contract but produces no cash_isa/stocks_and_shares_isa columns until a follow-up wires uk_runtime. Fine as a first step — just say so and link the follow-up.

  3. Naming/placement (you asked): US keeps one source_stages.json per country with many stages. I'd match that — uk/source_stages.json with a household_wealth stage — so the eventual UK manifest loader mirrors the US one and bus/geography stages append rather than spawning more *_source_stages.json files.

  4. No calibration. It's a donor QRF clipped to donor range, no target. HMRC's 2022-23 adult ISA market is ~£726bn (S&S ~£430bn, cash ~£294bn) and WAS under-captures the top tail, so a raw WAS-donor impute won't hit those totals. Either add a calibration target downstream or document that ISA totals are uncalibrated.

  5. Minor: the WAS mapping (cash_isa ← DVCISAVR8, stocks_and_shares_isa ← DVIISAVR8_aggr) lives only in notes and isn't test-checked — worth a line confirming DVCISAVR8 is the cash-ISA value field. fit_weighted_qrf has no seed; set one for reproducible builds. And the body says DVIISAVR8 while the manifest says DVIISAVR8_aggr.

No pin issue with the model PR — the manifest is just column names, so nothing here depends on policyengine-uk#1791 at build time. But the runtime PR that actually emits these columns should bump the uk extra from policyengine-uk>=2.88 to >=2.90 (the release carrying the input variables) so the columns land instead of being dropped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UK: surface cash ISA and stocks & shares ISA holdings in the wealth imputation

2 participants