UK AI-exposure analysis stack: exposure crosswalk, occupation targets, exposure imputation, and ESRI-style AI shock scenarios

[vahid-ahmadi]

Summary

Adds the four building blocks for a UK analysis of generative AI's fiscal and distributional impact by age/seniority (ESRI JR16 replication for the UK, extended along the age dimension), all under populace.build.uk_runtime:

1. AI-exposure crosswalk (ai_exposure.py + packaged CSVs)

• All 412 SOC2020 unit groups scored on six measures: c_aioe (primary, Felten AIOE × Pizzinelli complementarity adjustment), complementarity (θ), dsit_aioe, dsit_llm, eloundou_beta, felten_aioe.
• θ reconstructed from public O*NET 27.3 (CC BY 4.0) per the IMF WP/23/216 recipe; validated against the paper's published min/max anchor occupations (reproduced exactly, uniform +0.035 vintage offset documented).
• UK-native DSIT Nov-2023 Annex 1 scores (OGL v3) mapped SOC2010→SOC2020 via the ONS coding index; US measures chained US-SOC2018→SOC2010→ISCO-08→SOC2020 (BLS + ONS), mapping_quality + fan-out recorded per row (31 direct / 380 chained / 1 parent-imputed).
• exposure_for_soc() with 4→3→2-digit fallback; exposure_for_major_group() (1-digit, ASHE-2021-employment-weighted) for microdata that carries only FRS major-group SOC.

2. Occupation calibration targets (occupation_targets.py + packaged CSVs)

• ASHE Table 14 (2025 provisional, SOC2020 4-digit): 340 employment targets (suppressed cells skipped-and-reported).
• APS via Nomis (Jan–Dec 2025): 26 sub-major occupation targets + 6 age-band targets. Occupation×age cross-tab is not published; margins only, documented.
• Returns TargetSets (entity=person) with full provenance; bad rows skipped-and-reported, never silently dropped.

3. Exposure imputation stage (exposure_imputation.py)

• QRF is numeric-only (targets coerced to float, quantile interpolation), so the stage imputes the numeric exposure score, never occupation codes — with observed FRS soc_major_group as the lead predictor (refine-within-group), a warned blind fallback, and a zero-model baseline (exposure_from_major_group).
• Ships as a plan.Stage whose consumes includes soc_major_group, so runs are blocked before the FRS adult.tab SOC merge.
• Donor contract documents the UKDS EUL 5-quarter longitudinal LFS (4-digit SOC2020) column mapping and EUL constraints; donor microdata is never committed.

4. AI shock scenarios (ai_shock_scenarios.py)

• ESRI JR16 mechanics: employment shock per eq 3.4 (group quotas ∝ EMP×C-AIOE, weighted random within-group selection, summaries averaged over 50 seeded draws); wage shock per eq 3.5 (distributed by θ, not exposure; uniform fallback warns); capital shock parameterised as return 1.005%→1.405% (≈+39.8%) on interest+dividends, rent excluded.
• Presets: central (7% displacement, +2.6% wages), low (~1%, Acemoglu 2025), high (13%) — documented as overridable literature anchors.
• Extension beyond ESRI: youth_displacement_multiplier (default 1.0 = pure replication) tilts within-group displacement toward 16–24 while preserving eq-3.4 group totals; all summaries resolved by age band. ESRI's stated limitation (random within-group selection ignores younger workers' higher transition risk) is quoted in the docstring.

Tests

48 new tests across the four modules, all passing (uv run --package populace-build --group dev pytest); ruff check/format clean; uk_runtime import verified (171 exports, all __all__ names resolve). Packaged data files are all <50 KB, open-licensed (OGL v3 / MIT / CC BY 4.0), with sources cited in module docstrings.

Not included (follow-ups)

• FRS adult.tab SOC merge script (attaches observed 1-digit SOC to the person table)
• Analysis driver (attach exposure → scenario grid → PolicyEngine UK → Exchequer/poverty/Gini by age band)

References

The design replicates and extends the methodology of these papers:

• Doorley, K., O'Connor, S., O'Shea, R. & Tuda, D. (2026), Artificial intelligence and income inequality in Ireland, ESRI/Dept. of Finance Jointly-published Report No. 16 — the method being replicated for the UK: C-AIOE exposure -> employment/wage/capital shock scenarios (eq 3.4/3.5) -> tax-benefit microsimulation (SWITCH). PDF
• Klein Teeselink, B. (2025), Generative AI and Labor Market Outcomes: Evidence from the United Kingdom, SSRN 5516798 — UK evidence that GenAI-exposed firms cut employment (-4.5%), concentrated in junior roles (-5.8%); anchors the displacement-scenario calibration. SSRN
• Hosseini, S.M. & Lichtinger, G. (2026), Generative AI as Seniority-Biased Technological Change: Evidence from U.S. Resume and Job Posting Data — motivates the youth_displacement_multiplier extension: GenAI adoption reduces junior employment (~9% after six quarters) while senior employment is unaffected. PDF / SSRN 5425555

🤖 Generated with Claude Code

[MaxGhenis]

Strong work — clean module boundaries, provenance in every docstring, and behavioral tests that assert the actual equations (eq 3.4 group quotas, eq 3.5 θ-graded uplift, the +39.8% capital ratio) rather than just importing. The donor-weight test and the primary-vs-blind conditioning test are exactly the contracts DESIGN.md was written around. CI is green, it's mergeable, and it's additive with no downstream consumers yet, so nothing here blocks. A few methodology and provenance notes (agent-assisted review; data spot-checks reconciled — APS bands sum to the 16+ total, mapping_quality is 380/31/1, ASHE yields exactly 340 targets):

1. Crosswalk reproducibility. Only the output CSVs are committed; the ONET θ reconstruction and the US-SOC→ISCO→SOC2020 chaining live outside the repo. That's a documented choice, but the θ reconstruction is elaborate enough — and DESIGN.md's "artifacts carry their environment" pointed enough — that the builder (or a linked script) should be committed the way geography_sources ships its build_ functions. A follow-up issue is fine.

2. Two age-band schemes. The APS age targets use 16-19/20-24/25-34/35-49/50-64/65+; the scenario summaries use 16-24/25-34/35-44/45-54/55-64/65+. 16-24 reconciles to the first two APS bands, but 35-49/50-64 don't decompose into the scenario bands, so a driver can't calibrate age employment to APS and report shock incidence on the same bands above 35. Worth settling before the analysis driver lands.

3. Wage anchor attribution. The 7% displacement checks out as Goldman's (Briggs & Kodnani) 6-7% base case. I couldn't find the +2.6% wage figure in their work — their headlines are +15% productivity / +7% GDP — so please confirm where +2.6% comes from (ESRI's own calibration?) and cite that source in the module docstring and PRESETS comment. Low stakes since presets are labeled overridable anchors, but the citation should corroborate the value.

4. Self-employed sit outside every shock (the employed mask is employment_income > 0, and self-employment income isn't in the capital columns either). That mirrors ESRI, but state it as a scope limitation in any writeup.

5. Major-group coding seam: exposure_for_major_group wants 1-9 but the FRS side documents 1000-9000. It warns on unmatched rather than failing silently, but the integration step has to map codes or it gets all-NaN.

One unverified item, flagged as such rather than as a problem: the θ min/max anchor occupations vs IMF WP/23/216 (their site 403s). Happy to approve once the +2.6% citation is confirmed or corrected.

[MaxGhenis]

Review — provenance and conventions are excellent; two maintainer calls before merge (scope placement + rebase). Not approving yet on those grounds, not on any correctness defect.

Exposure-crosswalk provenance is exemplary. Every measure in uk_soc2020_ai_exposure.csv (all 412 SOC2020 unit groups) is attributed to a specific, real, correctly-characterized source:

• felten_aioe — Felten, Raj & Seamans (2021) standardised AIOE (AIOE-Data/AIOE GitHub).
• eloundou_beta — Eloundou et al. (2023) "GPTs are GPTs" human-annotated beta (openai/GPTs-are-GPTs, MIT).
• dsit_aioe/dsit_llm — DSIT/DfE "The impact of AI on UK jobs and training" (Nov 2023), Annex 1, the official UK SOC2010 mapping of the Felten scores.
• c_aioe (primary) — complementarity-adjusted AIOE per Pizzinelli et al. (2023, IMF WP/23/216) and the Williamson et al. (2025, J Labour Market Res 59:30) UK/Ireland application.
• The US-SOC-2018 → SOC-2010 (BLS Nov 2017) → ISCO-08 (BLS 2012) → SOC-2020 (ONS SOC2020 Vol 2 coding index) chain is fully documented, with many-to-many links flagged chained/direct/imputed-from-parent in mapping_quality.

The one reconstructed measure, complementarity_theta, is handled with real rigor and full disclosure: theta isn't openly published at occupation level (IMF/IGEES request-only), so it's rebuilt from the WP/23/216 recipe on ONET 27.3 and validated to reproduce the paper's published min/max occupations (US SOC 51-9031 / 29-1022) with a documented ~+0.03–0.04 vintage offset. No licensing issue — it derives from CC BY 4.0 ONET plus the published method, not the request-only data itself. Worth a note to swap in the exact IMF/IGEES theta if it ever becomes available.

Occupation-target provenance is equally clean. ASHE Table 14.7a → employment_jobs per SOC2020 4-digit (median pay carried for diagnostics only — correctly not a sum target); APS via Nomis NM_17_1, table T09b (sub-major occupation) and T01 (age band) declared as independent margins because the APS publishes no occupation×age cross-tab. CSV schemas match the docstrings and the magnitudes are right (16+ employment 33.3m ≈ UK total; SOC 11 2.62m). Bad rows are skipped-with-reason, structural problems raise.

Conventions match the UK build. exposure_imputation uses a proper DonorSpec (LFS Five-Quarter Longitudinal EUL) with a documented donor contract and fit_weighted predictors; occupation_targets compiles a populace.calibrate.TargetSet. Consistent with the DonorSpec/Stage + TargetSet patterns the rest of the UK stack uses.

Neutrality on the scenarios is right. ai_shock_scenarios is framed explicitly as scenario analysis, not causal/predictive estimation, implementing ESRI JR16 (Doorley et al. 2026) eqs 3.4/3.5 with parameters presented as overridable literature anchors (Briggs & Kodnani 2023 / Goldman central, Acemoglu 2025 low). It sets employment_income=0 + ai_displaced=True and leaves benefit/unemployment modelling to the downstream PE-UK run — clean separation. The by-age-band resolution and youth_displacement_multiplier are a documented, principled extension of ESRI's random within-group selection.

Two things for a maintainer to decide before merge:

1. Scope/placement. This is ~+3980 lines of research-analysis infrastructure (an exposure crosswalk + shock-scenario engine) landing in populace-build/uk_runtime. occupation_targets unambiguously belongs (population calibration). The exposure crosswalk + scenario engine are an analysis layer riding on the microdata — reasonable to host here, but big enough that whether it lives in core populace-build vs. a separate research module is a call for the maintainers, not something to wave through in a routine review.
2. Rebase. CONFLICTING/DIRTY — a large branch touching uk_runtime/__init__.py; it needs a rebase onto current main and the conflict resolution won't be purely mechanical. CI was green on the last push (test + wheels, 3.13/3.14).

Minor: the exposure major-group aggregation weights by ASHE Table 14 (2021 provisional) while the calibration targets use ASHE 2025 — different vintages for different purposes, defensible but worth aligning to the latest release; and the shipped CSVs are tidy derivations whose raw→tidy step lives outside the repo, so a checked-in derivation script would make the crosswalk fully reproducible from primary sources.

Not merging (review-only per request).