From Expansion to Elimination, DATA

This project performs a Bayesian hierarchical analysis to investigate the factors influencing energy cost burden across different ZIP codes and years. Using panel data from multiple Excel files spanning several years (2012-2022), the project aims to model the relationship between energy cost burden and various predictors including tax_returns, uptake (presumably related to program participation or energy efficiency measures), and percent_white.

The core of the analysis involves:

Data Loading and Preprocessing: Combining data from multiple years, handling missing values, and standardizing predictor variables.
Hierarchical Modeling: Building a Bayesian hierarchical model using PyMC that accounts for variation across both ZIP codes and years through the use of random effects.
Inference: Performing inference using both variational inference (ADVI) and Markov Chain Monte Carlo (MCMC) methods, specifically the No-U-Turn Sampler (NUTS), to estimate the posterior distributions of the model parameters.
Diagnostics and Comparison: Analyzing the convergence diagnostics (R-hat, ESS, divergences) for the MCMC samples and comparing the results obtained from ADVI and NUTS to understand the reliability of the different inference methods for this model and dataset.
Exploratory Analysis: Including steps for basic data exploration such as summary statistics, correlation analysis, and time trends of key variables.
The project highlights the importance of using robust MCMC methods like NUTS for complex models, especially when simpler approximations like ADVI might yield conflicting conclusions, and includes steps to improve sampler performance and assess convergence.