Derive hours-worked inputs from ASEC reported hours#293
Conversation
Add a hours_worked source stage (manifest entry, runtime handler, frame transform, release gate, builder wiring) following the immigration template. Direct-maps measured ASEC variables (HRSWK, A_HRS1, WKSWORK), heals the published constant-40 landmine, and hands weekly-hours ownership off from org_wages. Closes #242; unblocks #248.
daphnehanse11
force-pushed
the
hours-worked-source-stage
branch
from
237a841 to
7c797c4
Compare
|
Rebased onto current main ( Beyond conflict resolution, one intended change: dropped the now-stale Conflict resolutions were all additive interleaves with main's newly merged take-up (#315) and eCPS-parity (#316) wiring: both gate sets now run in sequence in Local verification: full 🤖 Generated with Claude Code |
Add a snap_take_up source stage (manifest entry, runtime handler, frame transform, release gate, builder wiring) following the immigration and hours templates. Reported SPM_SNAPSUB recipients always take up; non-reporters fill via seeded stable-identity draws to the cited FNS 0.82 participation rate. Heals the published all-True landmine. Closes #243; second half of #248 alongside #293.
MaxGhenis
left a comment
MaxGhenis
left a comment
There was a problem hiding this comment.
Approving (agent-assisted review). We verified this against the retired pipeline's final state rather than its old tagged releases, since that's what the parity claim is really about. The last-published policyengine-us-data surface (package 1.115.x — the direct mapping landed in 9f8db56 on 2026-05-23 and carried through the 2026-07-02 archival) maps weekly_hours_worked = HRSWK, hours_worked_last_week = A_HRS1, and weeks_worked = clip(WKSWORK, 0, 52). This PR reproduces those exactly, so "parity by construction" holds. (The older 1.17/1.18 tags annualized by WKSWORK/52, but that's not the retired state.)
The one real subtlety is a strength, not a gap: the retired pipeline set the aggregate weekly_hours_worked, never the leaf weekly_hours_worked_before_lsr — which is why the leaf fell back to its 40-hour default and the SNAP 30/20-hour tests passed for everyone (#242). Writing the leaf with the same HRSWK value both preserves the retired aggregate and fixes the variable the work-requirement rules actually read. Sentinel handling (A_HRS1 = -1/NaN floored to 0, weeks clipped to [0, 52]) is if anything a touch more robust than the retired assignment.
Gate bands, the constant-40 healing path, the org_wages ownership handoff, stage ordering, and the 18 tests all check out, and the rebase is a clean single commit on the current base. The end-to-end result (worked share 0.512, mean weekly hours 37.3) matches the retired surface's ~50.4% positive share.
Sequencing: no logical dependency on #294, but both PRs thread gate params through the same release-builder signatures and edit the same exclusions dict, so whichever lands second needs a mechanical rebase. Note I removed the stale SNAP parity-register entry on #294; this PR's hours columns aren't in the register, so nothing equivalent is needed here.
Optional nit: hours_worked.py:81 — US_HOURS_WORKED_NONCONSTANT_PERSON_COLUMNS = (US_HOURS_WORKED_OUTPUT_COLUMNS) isn't a tuple wrap, just an alias; drop the parens.
|
Follow-through from Max's #294 review, which named this PR directly: "Same pattern applies to #293's hours entries — cleanest if each producing PR removes its own register line." This branch's stage outputs |
Add a snap_take_up source stage (manifest entry, runtime handler, frame transform, release gate, builder wiring) following the immigration and hours templates. Reported SPM_SNAPSUB recipients always take up; non-reporters fill via seeded stable-identity draws to the cited FNS 0.82 participation rate. Heals the published all-True landmine. Closes #243; second half of #248 alongside #293.
…ate (#294) * Assign SNAP take-up from reported receipt and the FNS rate Add a snap_take_up source stage (manifest entry, runtime handler, frame transform, release gate, builder wiring) following the immigration and hours templates. Reported SPM_SNAPSUB recipients always take up; non-reporters fill via seeded stable-identity draws to the cited FNS 0.82 participation rate. Heals the published all-True landmine. Closes #243; second half of #248 alongside #293. * Remove the takes_up_snap_if_eligible parity exemption: this stage produces it The register reason ('Populace does not yet produce it') becomes false once this PR's SNAP stage populates the layer, and the entry would fail the stale-exemption check when #337 lands. #337 removes the four entries rotted by already-merged work; each producing PR removes its own. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Max Ghenis <mghenis@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…he guard The #293 hours stage merged without removing its parity register exemptions, so hours_worked_last_week and weekly_hours_worked_before_lsr carried false "not yet on the candidate frame" reasons — the stage runs before the parity gate and its signal gate requires both non-constant, so the first release build after the staleness enforcement lands would have failed on them. Both confirmed as engine input variables (its third output, weeks_worked, has no register entry). hourly_wage stays: it is an org_wages output and that stage is not executed anywhere yet. The static register guard now also pins the deterministic stage output constants (hours, SNAP take-up, eligibility inputs) alongside the immigration outputs and contract-produced take-up flags, so the next producing stage that forgets its register line fails in CI rather than on a release build — #293 slipped through exactly because the guard did not know stage outputs, only take-up programs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ies (#337) * Fail the eCPS parity gate on stale exemptions; remove four rotted entries parity_gate skipped known-gap names unconditionally, unlike its sibling gates (default_valued_columns_gate at gates.py:707 and source_stage_input_coverage_gate at gates.py:1499), which both fail on stale reviewed exclusions. That let register entries rot silently once the candidate started producing the exempted layer (the takes_up_medicaid_if_eligible case surfaced in PR #334). The gate now fails on any exemption whose candidate nonzero share is positive, reports exemptions the reference never populates as dormant, and records both in the gate details. Auditing the register against the current release pipeline found four entries stale on arrival — the pipeline deterministically materializes them before the gate runs: takes_up_tanf_if_eligible and takes_up_eitc (contract-seeded take-up, PR #315), ssn_card_type and immigration_status_str (immigration stage, #266). A new register test pins that statically-knowable subset so a stale entry is caught at test time, not first at release-build time. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Extend the register staleness guard to count-calibrated take-up programs Max's review of #294 (SNAP take-up) spotted that the static guard covered immigration outputs plus seeded_take_up_programs() only, so a program whose stage produces its flag under a different contract treatment sat outside it — SNAP (out_of_scope) was hand-fixed on #294, and count_calibrated programs (Medicaid in #334, CHIP next per the contract followup) had the same exposure: the staleness failure would only surface on the first release build, not in CI. The guard now derives the produced set from contract treatments ({seed, count_calibrated}) instead of the seeded helper, making it the CI enforcement of the rule that each producing PR removes its own register line. Keyed on the treatment string rather than #334's count_calibrated_take_up_programs() helper so the test imports nothing that does not exist on main yet; it picks up the Medicaid flip (and any future CHIP flip) automatically at merge, in either merge order. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Remove the two hours-stage register entries #293 left behind; widen the guard The #293 hours stage merged without removing its parity register exemptions, so hours_worked_last_week and weekly_hours_worked_before_lsr carried false "not yet on the candidate frame" reasons — the stage runs before the parity gate and its signal gate requires both non-constant, so the first release build after the staleness enforcement lands would have failed on them. Both confirmed as engine input variables (its third output, weeks_worked, has no register entry). hourly_wage stays: it is an org_wages output and that stage is not executed anywhere yet. The static register guard now also pins the deterministic stage output constants (hours, SNAP take-up, eligibility inputs) alongside the immigration outputs and contract-produced take-up flags, so the next producing stage that forgets its register line fails in CI rather than on a release build — #293 slipped through exactly because the guard did not know stage outputs, only take-up programs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2 participants
Summary
Closes #242; unblocks the SNAP work-requirements exposure channel tracked in #248.
The published dataset either omits
weekly_hours_worked_before_lsror ships it constant at the engine's 40-hour default, so PolicyEngine-US treats everyone as working 40 hours/week — SNAP's 30-hour general and 20-hour ABAWD tests pass universally and work-requirement reforms score ~zero through the hours channel.This adds a
hours_workedsource stage following the #266 (immigration) template: manifest entry + shared runtime handler + frame transform + release gate + builder wiring.What it does
Direct mappings of measured ASEC variables — identical to the retired enhanced-CPS pipeline, so parity is by construction, nothing imputed:
weekly_hours_worked_before_lsr←HRSWK(usual weekly hours last year; 0 for non-workers)hours_worked_last_week←A_HRS1weeks_worked←WKSWORKclipped to [0, 52]Ownership handoff: the
org_wagesmanifest entry no longer claimsweekly_hours_worked_before_lsr— it never produced it (the retired pipeline sourced weekly hours from ASECHRSWKdirectly, not from the ORG fit) — and consumes hours as predictors instead.Healing behavior: the frame transform is idempotent when the hours surface carries signal, and recomputes from raw columns when weekly hours are constant — the published constant-40 landmine is repaired, not trusted.
Release gate (
hours_worked_signal): weighted positive-hours share must land in [0.35, 0.62] (CPS ~50% of all persons worked last year) and mean weekly hours among workers in [30, 45] (BLS ~38.5). Threaded through release gate failures, calibration diagnostics, and both manifests, mirroring the immigration gate. The L0 refit export now requires all three columns nonconstant.Verification
Documented semantics caveat
SNAP's ABAWD standard is a monthly 80-hour test;
HRSWKis an annual usual-hours measure. A person who worked half the year at 40 hrs/week carries 40 here. This matches how the retired pipeline fed the engine and is stated in the stage notes — week-by-week volatility is not observable in the CPS and is out of scope. The full ORG labor-market fit (tier 2) is tracked in #291.Merge-order note
After this merges, the
weekly_hours_worked_before_lsrreviewed exclusion in #286's degenerate-input gate becomes stale and should be removed (the gate will then permanently enforce non-degenerate hours). If #286 merges first, that's a one-line follow-up here; if this merges first, I'll drop the exclusion in #286.🤖 Generated with Claude Code