How the pipeline works

The emburdensynth pipeline turns open data into cell-level household energy expenditure estimates in five phases plus a post-processing step. Each stage is implemented as a single exported function; each can be run independently for debugging.

The five phases plus post-processing

Phase 1

impute_physical_characteristics()

Probabilistic mapping from NREL ResStock building physical attributes onto ACS PUMS demographic households. Produces per-household dwelling area, vintage, heating fuel, insulation profile.

Phase 2

calibrate_expenditures()

Quantile mapping via RECS empirical CDF stratified by income bracket and building type. Assigns utility-expenditure quantiles to each household.

Phase 3

optimize_spatial_allocation()

Multi-level simulated annealing across ~2,300 PUMAs / GADM level-2 units, matching cell-level marginals to national totals. Runs in parallel via furrr.

Phase 4

reverse_engineer_consumption()

Piecewise inverse-tariff root-finding via uniroot. Converts household dollar expenditure to physical kWh consumption using per-country utility rate structures from URDB.

Phase 5

generate_load_profiles()

Load-profile morphing to 8,760 hourly bins, weighted by end-use and season using NREL EULP baselines. Output: l1l8760 kWh/hour columns per household in parquet.

Post

calculate_energy_insecurity_gap()

Joins the physics baseline (degree-day derived required kWh) with the pipeline’s estimated consumption. Flags cells where estimated < 0.9 × physics-required.

Net Energy Burden aggregation

Aggregating household-level burden to country-level requires a well-formed weighted mean. The naive choice — weighted.mean(spending/income, weights) — is biased. The mathematically correct formulation, derived from the Net-Energy-Burden framework in the emburden package, computes Nh = (G-S)/Se per cell (where Se is energy spending), takes the weighted mean of Nh, and inverts as:

NEB = 1 / (1 + weighted_mean(Nh))
Why this matters

Across the 128 countries covered in our release, naive aggregation over-estimates country burden by a median 5.72%. The bias is small for stable economies but becomes very large under hyperinflation. For VEN, where cell-level energy spending/income ratios span several orders of magnitude, the naive-vs-proper divergence reaches 1414%. This result establishes that any prior cross-country comparison of household energy burden using weighted-mean-of-ratios has been systematically biased in a direction that inflates estimates in the world’s most economically volatile settings.

The physics baseline

Cell-level physics-required kWh is derived from:

  • HDD × UA × area / COP_heat — heating demand
  • CDD × UA × area / COP_cool — cooling demand
  • 1,800 kWh/yr baseload — appliances, lighting, hot water

with country-specific HDD and CDD from World Bank Climate Change Knowledge Portal and a global-average building envelope (60 m², COP 2.5 for cooling, 2.8 for heating). Per-country building-stock corrections are on the roadmap.

Uncertainty quantification

Bootstrap 90% confidence intervals via 500 Beta-perturbation replicates on GCD sector shares (n_eff = 100) plus boundary-jitter sensitivity across GADM level-2 boundaries. Cell-level CI width is aggregated to country-mean CI width via variance decomposition.

Cross-signal integration

Country-level burden is joined to 11 external signals for the PCA + k-means archetype analysis:

  • WB WDI 1960–2024 (7 indicators including electricity access)
  • Ember 2000–2025 (aggregate + 9-fuel generation mix)
  • OWID Energy 1900–2025 (per-capita electricity + primary energy)
  • OWID CO2 1750–2025 (per-capita + cumulative + by-fuel)
  • EIA International 1980–2023 (222 countries)
  • WHO PM2.5 ambient air quality (population-weighted)
  • WB clean-fuel cooking access
  • FAOSTAT undernourishment 2000–2023
  • UNDP HDI 1990–2022
  • NASA GISTEMP 1880–2025
  • WB Worldwide Governance Indicators 1996–2022

See data provenance for full URLs + retrieval dates.