Public Utility Data Liberation Project (PUDL) Data Release

v2025.8.0 (2025-08-14) This is a regular quarterly release of PUDL. It includes new 2024 annual updates for a number of datasets (FERC Forms 2, 6, 60, & 714), and a minor update to the 2024 FERC Form 1 data that includes late filings & revisions. It also includes year-to-date updates for the monthly and quarterly datasets, including EIA-860M, EIA-923, EIA-930, and the EPA CEMS hourly emissions. There were also a number of data processing bug fixes and data usability improvements. See the full notes below for details. New Data Thanks to contributions from @alexclippinger, we’ve added cleaned EIA923 Schedule 8A Byproduct Disposition to the PUDL database as _core_eia923__yearly_byproduct_disposition. Once harvested, this table will be replaced with a well-normalized version of the same data, but it is being published in this form until then. See #4100 and #2448, and #4502. Expanded Data Coverage EIA-860M Updated EIA-860M monthly generator report with newly published data for May and June of 2025. See issue #4379 and PR #4536. EIA-923 Added EIA-923 data through May 2025. See #4516 and #4538. EIA 930 Updated EIA 930 data published up through the beginning of August 2025. See #4517 and PR #4523. EIA Bulk Electricity API Updated the EIA Bulk Electricity data to include data published up through the beginning of August 2025. See #4519 and PR #4523. EPA CEMS Added EPA CEMS data through June 2025. See #4518 and #4531. FERC Form 1 Updated FERC Form 1 2024 data to include late respondents. See #4493 and #4507. FERC Forms 2, 6 and 60 Updated our extraction of FERC Forms 2, 6, and 60 to raw SQLite databases to include 2024 data. See #4418 and #4433. FERC Form 714 Integrated 2024 data for FERC Form 714. See issue #4409 and PR #4530. PHMSA Gas Data Extracted 2023 and 2024 PHMSA distribution and transmission data to raw assets. This data is not currently published to the PUDL database. See #4449 and #4470. Extracted 1970 through 1989 PHMSA transmission data to raw assets. This data is not currently published to the PUDL database. See #3290 and #4500. Quality of Life Improvements The output of dbt_helper update-tables now conforms to the format that our pre-commit hooks expect, reducing annoying back-and-forth and diffs. See #4119 and #4401. Improved behavior of dbt_helper when interacting with row count test definitions as well as updating the row counts stored in dbt seed tables: the logic for writing a new table dbt schema no longer includes automatically adding a row count test. Also, the logic for updating row counts now depends on whether a test has been defined in the dbt schema, whether any existing row counts for that table are present in the seed table, as well as user provided settings such as --clobber. Stopped running code checks in CI when only the documentation has changed. See issue #4410 and PR #4429. Added utility_id_ferc1_dbf and utility_id_ferc1_xbrl columns into all ferc1 output tables. See #4365 and PR #4528. Bug Fixes Fixed bug in how we were labeling the data_maturity of EIA 923. See issue #4328 and PR #4392. Fixed bug in how we were repairing a misfiled EIA code in core_ferc714__respondent_id. See issue #4439 and PR #4497. Fixed bug in how we were removing duplicates in core_eia923__monthly_generation resulting in ~400 more records in this table over several years. See details in PR #4538 Documentation Migrated table description metadata into new format; see epic #4358 for issues & PRs for all source groups. This included renaming two of the preliminarily published _core tables to better conform with our table naming conventions. Table _core_eia923__cooling_system_information is now _core_eia923__monthly_cooling_system_information and _core_eia923__fgd_operation_maintenance is now _core_eia923__yearly_fgd_operation_maintenance. See #4422. Added data source pages for: EPA CAMD to EIA Power Sector Data Crosswalk; see issue #4376 and PR #4403 New Tests and Data Validations EIA-930 and FERC-714 Hourly Imputed Demand Added checks which ensure that only hourly electricity demand values which are flagged for imputation change significantly from their reported values before and after the imputation. Check that the missingness of various columns in the hourly reported demand and imputed demand are within expected ranges. Explicitly flag years of which are dropped due to insufficient data for meaningful imputation with BAD_YEAR. Affected tables include out_eia930__hourly_operations, out_eia930__hourly_subregion_demand, and out_ferc714__hourly_planning_area_demand. See PR #4334. Check for entirely null column-years Previously we had a data validation check that ensured there were no entirely null columns applied to a handful of tables. Such columns were typically the result of typos or failures to update column names, or application of an incompatible dtype, e.g. casting an uncleaned column containing Y or N to boolean. A similar check has been implemented in our dbt data validation checks and is now applied to all tables. See issue #4105 and PR #4382. As a result of more broadly applying this check, we found and fixed a few data quality and column naming issues resulting in minor changes to the database schema: id_dc_coupled_tightly was renamed to is_dc_coupled_tightly (typo). switch_operating was consolidated with the existing can_switch_when_operating column found in the multi-fuel generator tables. The model_tax_credit_case_nrelatb column had its allowable enumerated values corrected, resulting in real non-null contents. See PR #4384. Three previously entirely null boolean columns in the multifuel generator table now contain real values, they are: can_fuel_switch, has_regulatory_limits, and can_cofire_oil_and_gas. Unusual patterns of null values were identified and investigated in issue #4407 with some additional explanations added in PR #4442. Other PUDL v2025.8.0 Resources PUDL v2025.8.0 Data Dictionary PUDL v2025.8.0 Documentation PUDL in the AWS Open Data Registry PUDL v2025.8.0 in a free, public AWS S3 bucket: s3://pudl.catalyst.coop/v2025.8.0/ PUDL v2025.8.0 in a requester-pays GCS bucket: gs://pudl.catalyst.coop/v2025.8.0/ Zenodo archive of the PUDL GitHub repo for this release PUDL v2025.8.0 release on GitHub PUDL v2025.8.0 package in the Python Package Index (PyPI) Contact Us If you're using PUDL, we would love to hear from you! Even if it's just a note to let us know that you exist, and how you're using the software or data. Here's a bunch of different ways to get in touch: Follow us on GitHub Use the PUDL Github issue tracker to let us know about any bugs or data issues you encounter GitHub Discussions is where we provide user support. Watch our GitHub Project to see what we're working on. Email us at [e-mail] for private communications. On Mastodon: @[e-mail] On BlueSky: @catalyst.coop On Twitter: @CatalystCoop Connect with us on LinkedIn Play with our data and notebooks on Kaggle Combine our data with ML models on HuggingFace Learn more about us on our website: https://catalyst.coop Subscribe to our announcements list for email updates.