Fixed-Effect Suppression in Mixed-Effects Gradient Boosting: An Empirical Study of Unseen-Group Prediction

# Fixed-Effect Suppression in Mixed-Effects Gradient Boosting Data, code and recorded outputs for the study of unseen-group prediction inmixed-effects gradient boosting. Supports the paper "Fixed-Effect Suppression in Mixed-Effects GradientBoosting: An Empirical Study of Unseen-Group Prediction" by Abha Vasal,Preeti Rathi and Rupesh Kumar. The article reference will be added here onceit is published. The paper asks what remains of the population-level predictor when amixed-effects boosted model is applied to a group absent from training. Therandom effect of an unseen group cannot be estimated, so prediction falls tothe tree ensemble alone. This deposit holds the three panels, the code as it was run, and one recordedoutput file for every table. What can be reproduced from it, and what cannot,is set out below. In short: the recorded outputs reproduce, the standalone scripts run, andevery stage that produced a reported number is here, including the syntheticexperiment and the three-comparator benchmark. Two things a reader cannot dofrom this folder: execute a full refit end to end, because the fitting stagesrun inside the notebook that holds the panels in memory, and rebuild Figure3(b), because the absolute predicted spread was never written to file. --- ## Layout ```data/ europe/ p2_region_year_panel.csv NUTS2 by year, source panel regional_pm25_W.csv region-constant PM2.5 level and trend eurostat_series/ seven regional covariate series india/ harmonized_city_year_panel.csv acag_city_year_pm25.csv radon/ srrs2.csv EPA residential radon surveycode/ 01 to 21, 04 and 17 withdrawn, so 19 files: construction, experiments, fitting stages, audits and figuresresults/ one file per table, named for the table it supportsMANIFEST.csv every file with its SHA-256 digest and what it supportsVERIFICATION.txt the reproduction check, with its findingsAUDIT.txt the record of how this folder was assembled``` Notebook outputs, execution counts and cell identifiers have been stripped, sothe notebooks carry source only. The audit record that earlier appeared as anappendix table in the paper is held here, in `AUDIT.txt` and `VERIFICATION.txt`. --- ## The three panels | panel | rows | groups | group unit | predictors | outcome ||---|---|---|---|---|---|| Europe | 864 | 219 | NUTS2 region, in 29 countries | 56 | all-cause death rate per 100 000 || India | 1,400 | 66 | city | 39 | annual mean PM2.5, µg m⁻³ || Radon | 12,687 | 471 | county | 11 | log radon, log(pCi L⁻¹) | The Indian outcome is an exposure measure, not mortality. It tests modelbehaviour on a second panel; it does not confirm the European mortalityresult. Two columns were removed before fitting and both removals matter. **Regional life expectancy, Europe.** Its correlation with the all-cause deathrate is −0.939 on the fitted years. The two measures describe the samemortality experience, so a model given it would be re-deriving the outcomerather than predicting it. **Reported measurement error, radon.** `pcterr` is the percent error of eachradon reading. Relative error on a counting measurement falls as the activityrises, so the column is a transformation of the outcome. On this file`corr(1/pcterr, log activity)` is +0.871, while the raw correlation is only−0.384 and the rank correlation −0.088. A linear or rank screen does not findit. Predicting the outcome from that one column with a 40-bin lookup table andno learner scores 0.4983 mean per-county RMSE under county-blockedcross-validation, against 0.8173 for a predictor handed every held-out countymean. One column beat an oracle, which is how the problem was identified.`code/11_leakage_screen.py` is the screen; it reports no comparable column onthe European or Indian panels. --- ## Reproducing the results Fitting requires `gpboost` 1.7.4 (Python), `scikit-learn`, `numpy` and`pandas`. The audit and bootstrap scripts need only `numpy` and `pandas`. The three radon scripts write beside themselves unless `P2_OUT` is set. The code comes in two kinds, and the difference matters. **Notebooks, and cell-scripts that belong to them.** `code/01`, `code/02`,`code/03` and `code/05` are notebooks. They were written and run on GoogleColab with the panels held on Drive. Each now mounts Drive only when`google.colab` imports, and otherwise falls back to the working directory orto `P2_BASE`. That fallback lets them start outside Colab; it has not beenexercised end to end, because a full run refits every model and needs`gpboost`. Treat the notebooks as the record of what was run, and thestandalone scripts below as the part you can execute directly. `code/06`, `code/07`, `code/08` and `code/14` are notstandalone programs: each is one cell lifted out of a notebook, and eachreuses variables that the notebook has already put in scope, such as thefitted panel and the fold generator. Their headers say which notebook andwhich variables. To run them, paste each into the notebook named in itsheader, at the point the header gives. They will fail with a name error ifrun on their own. Run the notebooks in this order: 1. `code/01_build_panels.ipynb` builds all three panels and prints the counts in Table 2.2. `code/21_published_baselines.ipynb` constructs `PANELS` for all three panels. Notebook 02 reads two cells out of it rather than requiring it to be run separately.3. `code/02_sensitivity_and_benchmark.ipynb` runs the 18-configuration sensitivity analysis and the tuned benchmark for Europe and India. It also records the linear mixed model fitting outcome reported in Section 5.6. `code/14`, `code/18`, `code/19` and `code/20` are its stage scripts.4. `code/05_radon_leakage_and_refit.ipynb` establishes the radon leakage and refits that panel without `pcterr`. `code/06`, `code/07` and `code/08` are cells of this notebook and produce the radon rows of Tables 3, 4, 5, 7, 9 and 11.5. `code/03` runs the robustness arms. **Standalone scripts.** These run on the deposited files and need nothing inscope. Run each from inside `code/`. The default paths already point at thisdeposit, so the bare command is enough. ```cd codepython 09_country_clustered_bootstrap.py # Table 10 clustered intervalspython 10_design_matrix_audit.py # the 16 European design claimspython 11_leakage_screen.py # the radon leakage screenpython 13_interface_test.py # Table 6, needs gpboostpython 15_make_figure1.py # rebuilds Figure 1python 16_make_figure2.py # rebuilds Figure 2``` `09`, `10`, `11`, `15` and `16` need only `numpy` and `pandas`, with`matplotlib` for the two figure scripts. Each writes beside itself unless youpass `--out`. `code/12` is a record of a recalculation run during theanalysis and cannot be rerun here; its header explains why. Boosting with histogram splitting and subsampling is not bit-reproducible, soindividual fits vary slightly between runs. Aggregate values reproduce to theprecision printed in the paper. --- ## Which file supports which table | table | file ||---|---|| 1 | `results/table01_literature_audit.csv` || 2 | `results/panel_descriptives.csv`, and `code/01` || 3, 4, 5, 7 Europe and India | `results/table03_05_07_sensitivity_europe_india.csv` || 3, 4, 5, 7 radon | `results/table03_05_07_sensitivity_radon_fits.csv` and `..._byconfig.csv` || 6 | `results/table06_interface_test.csv` || 8 | `results/table08_synthetic_dose_response.csv`. See the note on the synthetic experiment below. || 9, 10 Europe and India | `results/table09_10_per_group_rmse.csv` || 9 radon | `results/table09_radon_tuned_folds.csv`, `results/table09_11_radon_per_county.csv` || 10 clustered intervals | `results/table10_country_clustered_intervals.csv` || 10, 11 Europe and India | `results/table09_10_11_paired_differences.csv` || 11 radon | `results/table09_11_radon_per_county.csv` || Figures 1 and 2 | `code/15_make_figure1.py`, `code/16_make_figure2.py`, `results/figure01_values.csv`, `results/figure02_values.csv`, `results/figure01.png`, `results/figure02.png` || Figure 3(a) | `results/table08_synthetic_dose_response.csv`, produced by `code/20` || Figure 3(b) | not reproducible from the deposited output; see the Table 8 and Figure 3 section || Section 4.2 split audit | `results/split_integrity_fold_balance.csv` || Section 4.1 tuned hyperparameters | `results/hyperparameter_selections.csv` || Section 5.2 failure case | `results/section52_failure_case_europe_india.csv`, `..._radon.csv` | One caution on `results/table03_05_07_sensitivity_europe_india.csv`. Besidesthe Europe and India arms it carries a Radon arm from the run before `pcterr`was removed, and five synthetic arms. Only rows with `arm == real` and panelEurope or India are current. The radon values in that file are superseded by`results/table03_05_07_sensitivity_radon_byconfig.csv`. The two figure scripts filteron this and stop if the filter fails. `MANIFEST.csv` repeats this mapping for every file, with digests. Its ownrow carries an empty size and digest, because a file cannot contain its ownhash. The other 59 rows are checkable. --- ## Table 8 and Figure 3 `results/table08_synthetic_dose_response.csv` is the recorded output of thesynthetic experiment: 30 rows, six target variance shares by five replicates,with the realised share, the spread ratio and the RMSE for each. The code thatproduced it is `code/20_tasks_2_and_3.py`. Table 8 and Figure 3(a) can be recomputed from that file in full. **Figure 3(b) cannot.** Panel (b) plots the predicted standard deviationitself, indexed within each replicate, and the recorded file keeps only theratio. The ratio stored for a level is the mean of five fold-specific ratios,each divided by the standard deviation of its own held-out fold. Those fivedenominators do not grow at a common rate as the group variance rises: acrossthe deposited replicates they diverge by as much as a factor of two betweenfolds at the highest level. So the mean of the ratios cannot be multipliedback by any single outcome standard deviation to recover the mean predictedspread. An earlier version of this deposit carried a script that attemptedexactly that reconstruction. It has been withdrawn, because it rested on anassumption the recorded data do not support. Recording the quantity is a one-line change, marked in `code/20` at the pointwhere the ratio is formed. Until that experiment is rerun, Figure 3(b) issupported by the figure in the paper and not by a deposited series. --- ## The four fitting stages Notebook 02 runs the Europe and India work in four stages, each a separatescript. All four are deposited. | Stage, as notebook 02 names it | Deposited as | Produces ||---|---|---|| sensitivity | `code/18_gpboost_tuning_sensitivity.py` | the 18-configuration sweep, `results/table03_05_07_sensitivity_europe_india.csv`, behind Tables 3, 4, 5 and 7 || failure_fix | `code/19_gpboost_and_mixedlm_fix.py` | the Section 5.2 failure case, `results/section52_failure_case_*.csv`, and the Section 5.6 mixed model record || robustness | `code/14_configuration_robustness.py` | the configuration-robustness check of Section 6.3 | One thing to expect when `PANELS` is built. It gives Radon 12 predictors,`pcterr` among them, because that is the state the panel was in before theleakage was found. The 11-predictor radon panel the paper reports comes fromthe refit in `code/05`, not from `PANELS`. Europe and India are unaffected:`PANELS` gives them 56 and 39 predictors, which is what the paper reports. Before any of them runs, notebook 02 builds `PANELS`, the object all fourstages depend on. It does that by lifting two code cells out of`code/21_published_baselines.ipynb`, which it finds by the markers`REQUIRE_REVISED_PANEL` and `PANELS = {}`. That notebook is thelife-expectancy-removed version, the one the paper uses.`code/01_build_panels.ipynb` builds the source panels from the raw series butdoes not define `PANELS`, so it is not the notebook the markers live in. It ishowever the panel source that `code/21` itself resolves: under its originalname it was `P2_DESC_panels-LIFEEXP_REMOVED.ipynb`, and it carries the fourmarkers `code/21` requires plus `EXCLUDE_NEAR_DEFINITIONAL`, which is whatmakes the Europe panel the 56-predictor one. `code/21` now looks for thedeposited name first, so the chain 02 to 21 to 01 resolves inside this folderwith nothing external. That notebook was run on 13 October 2026. The executed copy was comparedagainst the deposited one cell by cell: 15 of its 17 code cells arebyte-identical, and the two that differ are path lists only. Its recordedoutput gives Europe 864 rows and 219 regions, India 1,400 and 66, Radon12,687 and 471, which are the figures in Table 2.| tasks23 | `code/20_tasks_2_and_3.py` | Table 8 and Figure 3 from the synthetic dose response, and Tables 9, 10 and 11 from the benchmark | None of the four is standalone. Each reuses `PANELS` and the fold helpers thatnotebook 02 has already built, which is why notebook 02 locates them by nameand runs them in order rather than importing them. Notebook 02 tries thedeposited name first and the original working name second, so it finds themwhichever layout it is run in. `code/20` is the source of the synthetic experiment. An earlier version ofthis deposit said that code was missing, because the notebook then carryingthat description ran a different experiment and was withdrawn. The actualsource is here. What still cannot be done from this folder is a full end-to-end refit without`gpboost` and a run of notebook 02 from the top. The recorded outputs arecomplete, and `VERIFICATION.txt` checks the reported values against them. --- ## Two files carry superseded radon rows, and say so `results/table09_10_per_group_rmse.csv` and`results/table09_10_11_paired_differences.csv` were written before themeasurement-error predictor `pcterr` was removed from the radon panel. TheirEurope and India rows are current and match the manuscript. Their Radon rowsare not. Both files now carry a `status` column and a `note` column. Every Radon rowreads `superseded` and names the file that replaces it. No row was altered ordeleted, because these are recorded outputs and rewriting a recorded numberwould be worse than labelling it. | Quantity | These files | Manuscript, from the current source ||---|---|---|| Radon GPBoost mean RMSE | 0.2980 | 0.9279 || Radon pooled minus GPBoost | +0.7856 | +0.1553 || Radon ridge minus GPBoost | +0.6347 | +0.1134 | The current radon source is `results/table09_11_radon_per_county.csv`. Takingthe mean per county over its 471 counties and then averaging gives GPBoost0.9279, ridge 1.0413 and pooled 1.0833, and the paired differences +0.1553[+0.1356, +0.1750] and +0.1134 [+0.0951, +0.1316]. Those are the Table 9 andTable 11 radon values exactly. `code/09` reads only the Europe rows of the per-group file, so the clusteredintervals in Table 10 are unaffected. --- ## Audits included `results/audit_european_design_matrix.csv` is a per-predictor inventory of theEuropean fitting matrix: name, origin, type, unique values, unique valueswithin region, missing count, range and correlation with the outcome.`results/audit_european_design_verdict.csv` checks sixteen claims the papermakes about that matrix. All sixteen hold. `results/audit_radon_leakage_screen.csv` records the screen that found`pcterr`. A recalculation was also run during the analysis, by `code/12`. Its output isnot included. It was produced before the measurement-error predictor wasremoved from the radon panel, so its radon figures are superseded, and two ofthe intermediate files it reads are not redistributed here, so it cannot bererun from this deposit. `VERIFICATION.txt` replaces it and is computed fromthe deposited files alone. `VERIFICATION.txt` compares selected manuscript values against quantitiesrecomputed from the deposited files. Fifty-one checks pass and none fails.They are not a rerun of the model fitting, and they do not cover every numberin the paper. `VERIFICATION.csv` holds the same checks in machine-readableform. One point of possible confusionis worth stating: the radon file holds nine FIPS codes but only eight distinctstrings in its state column, because Michigan, Wisconsin and part of Minnesotaare labelled with an EPA region code. The paper reports nine states, which iswhat the file supports. --- ## Sources Mortality: Eurostat table `hlth_cd_asdr2`, standardised death rate by NUTS 2region of residence, age-standardised by the direct method against theEuropean Standard Population 2013, per 100 000 inhabitants. Retrieved throughthe SDMX 2.1 dissemination API. Regional covariates: Eurostat `demo_r_d3dens`, `demo_r_mlifexp`,`demo_r_pjanind2`, `edat_lfse_04`, `hlth_rs_bdsrg`, `lfst_r_lfu3rt`,`nama_10r_2gdp`, each averaged over 2011 to 2022 and cascading to NUTS1 orcountry where regional values are absent. Air quality: ACAG V5.GL.04 surface PM2.5 at 0.1 degree with a 15 km buffer. Meteorology: NASA POWER daily fields. Radon: EPA `srrs2` residential survey. Archive timestamps are 30 July 2026 for mortality, 21 September 2026 for thecovariates and 16 August 2026 for radon. These are file timestamps rather thanpublisher receipts. Eurostat revises these tables, so a fresh extraction maydiffer. --- ## Licence Code is released under the MIT Licence, the text of which is in `LICENSE`. The CC BY 4.0 release covers only the panels we derived: the Europeanregion-year panel, the harmonised Indian city-year panel and theregion-constant PM2.5 summaries. It does not cover the files redistributed close to their source form. Theseven Eurostat series are reusable under the European Commission's reusepolicy with attribution. `data/radon/srrs2.csv` is the EPA residential radonsurvey. `data/india/acag_city_year_pm25.csv` is derived from ACAG V5.GL.04,which is released for non-commercial research use with attribution, and themeteorological fields come from NASA POWER. Each of those carries the termsset by its publisher, and those terms govern, not ours. Anyone reusing themshould go to the publisher.