Public Utility Data Liberation Project (PUDL) Data Release

v2025.9.1 (2025-09-05) This is a monthly release primarily focused on updating the EIA-860M, with other incremental changes coming along for the ride. A couple of things to be aware of: @mfripp identified a bug in how we were constructing detailed utility asset/liability and income/expense tables from FERC Form 1. This has been partially addressed, but the fix needs to be applied to a couple of additional tables. See #4593 to track our progress. We are now producing GeoParquet outputs for tables that contain spatial data. This is a great new feature! But also potentially a breaking change, depending on what tools you’ve been using to read our Parquet outputs. GeoPandas and DuckDB’s spatial extension both work well. Enhancements Geospatial outputs with GeoParquet We’ve started producing GeoParquet outputs that include explicit geometries for use with GeoPandas and other mapping and geospatial analysis packages. See geopandas.read_parquet() for dcoumentation on how to read them. We’ve also tested it with the DuckDB Spatial extension. This is still experimental and there are only a handful of tables that currently include geometries, but we hope to apply it more widely in the future for any tables with geospatial information. See PR #4546. We’ve started by writing the Census DP1 – Profile of General Demographic Characteristics state, county, and tract level data out as GeoParquet files, so they can be used alongside the other Parquet data without needing to read the Census DP1 SQLite DB. This will allow us to point our Kaggle (and other) notebooks that make maps directly at the Parquet files in S3 rather than depending on the (somewhat chonky) Kaggle PUDL dataset. For now the only tables with a valid geometry column are: out_censusdp1tract__states out_censusdp1tract__counties out_censusdp1tract__tracts out_ferc714__georeferenced_respondents Expanded Data Coverage EIA-860M Updated EIA-860M monthly generator report with newly published data for July of 2025. See issue #4590 and PR #4594. Quality of Life Improvements We updated our Kaggle notebooks to read PUDL data from our AWS Open Data Registry S3 bucket instead of relying on the PUDL Kaggle Dataset, since copying all of the PUDL data into the notebook workspace was taking more than 5 minutes, which made it frustrating for users to get started working with the data. This also means it should be easier to run the notebooks locally (in an appropriate Python environment) since the data doesn’t need to be present locally. The notebooks are also pushed to our PUDL Examples GitHub repo. See issue #4381. When running dbt_helper update-tables without the --clobber flag, existing schema tests, descriptions and other metadata are now preserved. Furthermore, the --update flag has been removed, with the default schema update logic behaving as follows: if columns are added or removed, updates are allowed to pass. However, if any metadata is removed, such as tests or descriptions, the update fails unless --clobber is used. See issue #4466 and PR #4525. Bug Fixes Stopped nulling values in columns with ENUM constraints when the value was not found in the ENUM. Previously we logged a warning, and now it will raise an error. There were a couple of trivial cases in which we were losing values that violated the constraints, but nothing serious. See PR #4548. Fixed a user identified bug within the out_ferc1__yearly_detailed_income_statements table unnecessarily dropping records. See PR #4580. Documentation Added data source pages for: EIA Bulk API Data; see issue #4372 and PR #4567. Other PUDL v2025.9.1 Resources PUDL v2025.9.1 Data Dictionary PUDL v2025.9.1 Documentation PUDL in the AWS Open Data Registry PUDL v2025.9.1 in a free, public AWS S3 bucket: s3://pudl.catalyst.coop/v2025.9.1/ PUDL v2025.9.1 in a requester-pays GCS bucket: gs://pudl.catalyst.coop/v2025.9.1/ Zenodo archive of the PUDL GitHub repo for this release PUDL v2025.9.1 release on GitHub PUDL v2025.9.1 package in the Python Package Index (PyPI) Contact Us If you're using PUDL, we would love to hear from you! Even if it's just a note to let us know that you exist, and how you're using the software or data. Here's a bunch of different ways to get in touch: Follow us on GitHub Use the PUDL Github issue tracker to let us know about any bugs or data issues you encounter GitHub Discussions is where we provide user support. Watch our GitHub Project to see what we're working on. Email us at [e-mail] for private communications. On Mastodon: @[e-mail] On BlueSky: @catalyst.coop On Twitter: @CatalystCoop Connect with us on LinkedIn Play with our data and notebooks on Kaggle Combine our data with ML models on HuggingFace Learn more about us on our website: https://catalyst.coop Subscribe to our announcements list for email updates.v2025.9.1 (2025年9月5日) 本版本为月度发布，核心更新围绕EIA-860M数据集展开，同时附带其他增量变更。以下几点请注意： @mfripp 发现了我们在从FERC Form 1构建详细公用事业资产负债及收支表格时存在的缺陷，目前该问题已得到部分修复，但仍需对额外两张表格应用该修复方案。可追踪议题#4593以了解修复进度。 我们现已为包含空间数据的表格生成地理参数化Parquet（GeoParquet）输出格式。这是一项极具价值的新功能，但同时也可能带来破坏性变更，具体取决于您此前用于读取Parquet输出文件的工具。GeoPandas与DuckDB的空间扩展工具均可完美适配该格式。 ## 增强功能 ### 地理空间输出支持（GeoParquet格式） 我们已开始生成包含显式几何信息的GeoParquet输出文件，以供GeoPandas及其他制图与地理空间分析工具使用。有关读取方式的说明，请参考geopandas.read_parquet()文档。我们同时已在DuckDB Spatial扩展工具中完成适配测试。目前该功能仍处于试验阶段，仅少数表格包含几何信息，但我们计划未来为所有带有地理空间信息的表格全面推广该格式。相关实现可参考PR #4546。 我们首先将Census DP1——通用人口特征概况的州、县及普查区层级数据导出为GeoParquet文件，如此一来用户无需再读取Census DP1的SQLite数据库，即可直接使用该数据与其他Parquet格式数据协同工作。这将使我们可以在Kaggle（及其他平台）的制图笔记中直接指向S3存储桶中的Parquet文件，而无需依赖体积相对庞大的Kaggle PUDL数据集。目前仅以下表格包含有效的几何列： - out_censusdp1tract__states - out_censusdp1tract__counties - out_censusdp1tract__tracts - out_ferc714__georeferenced_respondents ## 扩展数据覆盖范围 ### EIA-860M 更新了EIA-860M月度发电机报告，纳入了2025年7月最新发布的数据。相关信息可参考议题#4590与PR #4594。 ## 用户体验优化 我们更新了Kaggle笔记，使其从AWS Open Data Registry的S3存储桶中读取PUDL数据，而非依赖Kaggle平台上的PUDL数据集。此前将全部PUDL数据复制到笔记工作区需耗时5分钟以上，给用户初次使用数据带来了不便。该变更同时也让用户更易在本地（适配Python环境中）运行笔记，因为无需提前在本地存储数据。相关笔记已同步推送至PUDL Examples GitHub仓库。相关信息可参考议题#4381。 当不带--clobber标志运行dbt_helper update-tables命令时，原有模式测试、描述及其他元数据将被保留。此外，--update标志已被移除，默认的模式更新逻辑如下：若仅新增或移除列，则允许执行更新操作；但若存在元数据（如测试或描述）被删除的情况，则更新操作将失败，除非使用了--clobber标志。相关信息可参考议题#4466与PR #4525。 ## 漏洞修复 修复了ENUM约束列在枚举值未匹配时被置空的问题。此前我们仅会记录警告，现在将直接抛出错误。此前存在少量微不足道的场景会导致违反约束的数值丢失，但并未造成严重影响。相关实现可参考PR #4548。 修复了用户反馈的out_ferc1__yearly_detailed_income_statements表格不必要地丢弃记录的问题。相关实现可参考PR #4580。 ## 文档更新 新增了以下数据源页面： - EIA Bulk API数据；相关信息可参考议题#4372与PR #4567。 ## PUDL v2025.9.1 其他资源 - PUDL v2025.9.1 数据字典 - PUDL v2025.9.1 官方文档 - AWS开放数据注册表中的PUDL - 公共AWS S3存储桶中的PUDL v2025.9.1：s3://pudl.catalyst.coop/v2025.9.1/ - 请求者付费模式GCS存储桶中的PUDL v2025.9.1：gs://pudl.catalyst.coop/v2025.9.1/ - 本版本PUDL GitHub仓库的Zenodo存档 - GitHub上的PUDL v2025.9.1发布页 - Python包索引（PyPI）中的PUDL v2025.9.1软件包 ## 联系我们 如果您正在使用PUDL，我们期待收到您的反馈！哪怕只是告知我们您正在使用该软件或数据集，以及您的使用场景，我们都十分欢迎。您可以通过以下多种方式与我们取得联系： - 在GitHub上关注我们 - 使用PUDL GitHub议题追踪器反馈您遇到的漏洞或数据问题 - 通过GitHub Discussions获取用户支持 - 关注我们的GitHub项目以了解最新工作动态 - 发送邮件至[e-mail]进行私下沟通 - Mastodon平台：@[e-mail] - BlueSky平台：@catalyst.coop - Twitter平台：@CatalystCoop - LinkedIn上与我们建立联系 - 在Kaggle上体验我们的数据与笔记 - 在HuggingFace上将我们的数据与机器学习模型结合使用 - 访问我们的官网了解更多信息：https://catalyst.coop - 订阅我们的公告列表以获取邮件更新。