xpertsystems/oil042-sample

--- license: cc-by-nc-4.0 task_categories: - tabular-classification - tabular-regression - time-series-forecasting language: - en tags: - synthetic - digital-twin - oilfield - upstream - reservoir - well-production - artificial-lift - scada - ot-cybersecurity - methane - flaring - pipeline - ics - iec-62443 - isa-99 - isa-18-2 - phmsa - api-rp-754 - iso-14224 - spe - aapg - ghg-emissions - integrated-operations pretty_name: "OIL-042 — Synthetic Digital Twin Dataset (Oilfield) (Sample)" size_categories: - 100K<n<1M --- # OIL-042 — Synthetic Digital Twin Dataset (Oilfield) (Sample) A schema-identical preview of **OIL-042**, the XpertSystems.ai synthetic **integrated-oilfield digital twin** dataset. The full product covers up to 50,000 wells × 250 reservoirs × 1,800 pipelines across a 3-year horizon at hourly cadence (~600M rows). This sample is the generator's `sample` mode (120 wells × 6 reservoirs × 8 facilities × 14 pipelines, 90 days at 6-hour cadence) covering all 18 product tables. > **Built by** XpertSystems.ai — Synthetic Data Platform > **Contact** [[e-mail]](mailto:[e-mail]) · [xpertsystems.ai](https://xpertsystems.ai) > **License** CC-BY-NC-4.0 (sample); commercial license available for the full product. --- ## What makes OIL-042 different from the rest of the Oil & Gas vertical OIL-042 is the **first end-to-end integrated oilfield digital twin** SKU in the catalog. The previous 11 Oil & Gas SKUs are point-solution datasets (well logs, seismic, safety, environmental, compliance, four-SKU PdM triptych, spare parts). OIL-042 is the **canvas that ties them all together**: | Layer | OIL-042 tables | |---|---| | **Subsurface physics** | reservoir_master, reservoir_telemetry | | **Well operations** | wells_master, well_production, artificial_lift_systems | | **Surface infrastructure** | surface_facilities_master, surface_facilities, pipeline_master, pipeline_flows | | **OT / control systems** | scada_telemetry, alarm_events | | **Maintenance & reliability** | maintenance_workorders, equipment_failures | | **Environmental** | environmental_monitoring (methane, CO₂, flaring) | | **Cybersecurity** | cybersecurity_events (IT/OT ICS taxonomy) | | **Human operator** | operator_actions | | **ML labels** | digital_twin_labels (anomaly + 30d failure + production-loss risk + maintenance priority) | Use cases that need *cross-layer* causal modeling (e.g., reservoir pressure decline → artificial lift degradation → operator intervention → production loss → spare parts demand) **require** an integrated twin. OIL-042 is that substrate. --- ## What's inside 18 CSV tables covering the complete upstream digital twin: 5 dimensional masters (fields / reservoirs / wells / facilities / pipelines) + 5 telemetry streams (reservoir / production / lift / surface / pipeline / SCADA) + 7 event tables (alarms / workorders / failures / environmental / cyber / operator actions / labels). | Table | Rows (sample) | What it represents | |---|---:|---| | `fields_master.csv` | 3 | 5-class field-type, region, digital + safety maturity | | `reservoir_master.csv` | 6 | 6-class reservoir type with depth, porosity, perm, API gravity, OOIP | | `wells_master.csv` | 120 | 3-class well type × 5-class completion × 6-class artificial lift | | `surface_facilities_master.csv` | 8 | 7-class facility (separator/compressor/tank battery/etc.) | | `pipeline_master.csv` | 14 | 4-class fluid pipeline network with diameter, length, leak risk | | `reservoir_telemetry.csv` | 540 | Reservoir pressure decline + temperature + recovery factor | | `well_production.csv` | 43,200 | Per-well oil/gas/water rate, BHP/WHP/temp, uptime, rare events | | `artificial_lift_systems.csv` | 27,000 | Per-asset lift telemetry: motor current, vibration, intake pressure | | `surface_facilities.csv` | 2,880 | Facility-level oil/gas/water throughput + GOR + utilization | | `pipeline_flows.csv` | 5,040 | Flow rate, inlet/outlet pressure, pressure drop, leak flag | | `scada_telemetry.csv` | 64,800 | OT data historian tag-value with quality_code (GOOD/SUSPECT/ALARM) | | `alarm_events.csv` | 60 | ISA 18.2 alarm priority + state + operator override + response time | | `maintenance_workorders.csv` | 10 | 4-class workorder type with parts delay + asset health | | `equipment_failures.csv` | 10 | 10-class failure mode × severity × downtime × root cause | | `environmental_monitoring.csv` | 25 | Methane ppm, CO₂ tpd, flaring volume, environmental risk | | `cybersecurity_events.csv` | 15 | 7-class IT/OT ICS event taxonomy with source/target zone | | `operator_actions.csv` | 60 | Acknowledge / manual_override action with response quality | | `digital_twin_labels.csv` | 10,800 | **Per-well-per-timestamp anomaly prob + 30d failure prob + production-loss risk + maintenance priority** | Total: ~154,000 rows, ~19 MB. The full OIL-042 product is ~600 million rows. --- ## Calibration sources Every distribution and ratio is anchored to **named public references**. Highlights: - **SPE Petroleum Engineering Handbook** + **AAPG** — reservoir porosity, permeability, water saturation, GOR distributions. - **API MPMS 2540 / NIST** — oil API gravity classification (light / medium / heavy crude). - **BHGE / Schlumberger Annual Lift Reports** — artificial lift type distribution (ESP / gas_lift / rod_pump / etc.). - **ISA 18.2 / EEMUA 191** — alarm management taxonomy and priority bands. - **ISA-99 / IEC 62443** — ICS/OT cybersecurity event taxonomy. - **OPC UA / ISA-95** — data quality conventions for OT historians. - **PHMSA HL Pipeline Annual Incident Statistics** — pipeline leak rates. - **API RP 754** — process safety performance indicators. - **ISO 14224:2016** — reliability/maintenance data classification. --- ## Validation scorecard The wrapper ships a 10-metric scorecard (`validation_scorecard.json`) that re-scores the dataset on every generation. Default seed 42 result: | ID | Metric | Target | Observed | Source | |---|---|---|---:|---| | M01 | Reservoir Porosity (median) | 0.10–0.30 | **0.207** | SPE PE Handbook (clastic) | | M02 | Oil API Gravity (median) | 27–43° | **34.48°** | API MPMS 2540 | | M03 | Initial Water Saturation (median) | 0.18–0.38 | **0.283** | SPE / AAPG | | M04 | Well Water Cut (median) | 0.05–0.35 | **0.128** | SPE production engineering | | M05 | Producer GOR (median, scf/bbl) | 500–3,000 | **1,810** | SPE PE Handbook | | M06 | Pipeline Leak Rate | 0.0–0.020 | **0.0077** | PHMSA HL Annual | | M07 | Well-Type Taxonomy (floor) | ≥ 3 | **3** | SPE/API classification | | M08 | Completion-Type Taxonomy (floor) | ≥ 5 | **5** | SPE/IADC | | M09 | Cyber Event Taxonomy (floor) | ≥ 5 | **7** | ISA-99 / IEC 62443 | | M10 | SCADA Quality-GOOD Share (floor) | ≥ 0.98 | **0.999** | OPC UA / ISA-95 | **Grade: A+ (100/100). Verified across seeds 42, 7, 123, 2024, 99, 1.** --- ## Suggested use cases - **Reservoir-to-surface causal modeling** — `reservoir_telemetry` → `well_production` → `surface_facilities` → `pipeline_flows` are per-timestamp joinable, supporting GNN, multi-level state-space, and causal-graph models that cross subsurface-to-surface boundaries. - **Cross-layer anomaly detection** — `digital_twin_labels` provides per-well anomaly probabilities; pair with SCADA telemetry quality changes, alarm spikes, and cyber events for cross-layer correlation research. - **OT/IT cyber-physical attack modeling** — `cybersecurity_events.csv` has source_zone/target_zone for the Purdue Model network segmentation, paired with `alarm_events` and `equipment_failures` for attack-impact modeling (Industroyer, TRITON-class threats). - **Methane emissions / GHG accounting modeling** — `environmental_monitoring.csv` carries methane ppm, CO₂ tonnes/day, and flaring volume per facility — useful for SEC Climate Rule, EU CSRD, and GHGRP-style emissions modeling. - **Operator-action / human-in-the-loop modeling** — `operator_actions.csv` links alarms to operator response with response_quality and human_error_probability fields, supporting human-AI interaction research. - **Artificial lift optimization** — `artificial_lift_systems.csv` × `well_production.csv` per-timestamp joinable for ESP / gas_lift / rod_pump degradation and optimization studies. - **Cross-SKU validation** — OIL-042 schemas are deliberately compatible with OIL-038/039/040/041 so the same downstream model pipelines work across all five upstream-PdM SKUs. --- ## Loading ```python from datasets import load_dataset wells = load_dataset( "xpertsystems/oil042-sample", data_files="wells_master.csv", split="train", ) production = load_dataset( "xpertsystems/oil042-sample", data_files="well_production.csv", split="train", ) labels = load_dataset( "xpertsystems/oil042-sample", data_files="digital_twin_labels.csv", split="train", ) ``` Or with pandas directly: ```python import pandas as pd from huggingface_hub import hf_hub_download path = hf_hub_download( repo_id="xpertsystems/oil042-sample", filename="scada_telemetry.csv", repo_type="dataset", ) df = pd.read_csv(path) ``` All 18 tables join on: - `field_id` → fields_master ↔ reservoir_master ↔ wells_master - `reservoir_id` → reservoir_master ↔ wells_master ↔ reservoir_telemetry ↔ well_production ↔ labels - `well_id` → wells_master ↔ well_production ↔ artificial_lift_systems ↔ labels - `facility_id` → surface_facilities_master ↔ wells_master ↔ surface_facilities ↔ environmental_monitoring ↔ cybersecurity_events ↔ pipeline endpoints - `pipeline_id` → pipeline_master ↔ pipeline_flows - `asset_id` (SCADA) → wells / facilities - `failure_id` → equipment_failures ↔ maintenance_workorders - `alarm_id` → alarm_events ↔ operator_actions - `timestamp` / `timestamp_utc` → every time-series stream is hour-aligned --- ## Schema highlights **`reservoir_master.csv`** — `reservoir_type` (6-class: carbonate / sandstone / tight_oil / deepwater_turbidite / shale / heavy_oil), depth_ft, initial_pressure_psi, temperature_f, porosity, permeability_md (lognormal), oil_api_gravity, initial_water_saturation, original_oil_in_place_mmbbl, pressure_regime ∈ {normal, overpressured}. **`wells_master.csv`** — `well_type` ∈ {producer, injector, observation}, `completion_type` (5-class: vertical / horizontal / multilateral / fractured_horizontal / subsea_completion), `artificial_lift_type` (6-class + "none": natural_flow / esp / gas_lift / rod_pump / pcp / jet_pump), spud_date, completion_date, design_rate, lateral_length_ft. **`well_production.csv`** — per-well-per-timestamp `oil_rate_bpd`, `gas_rate_mscfd`, `water_rate_bpd`, `water_cut`, `bottomhole_pressure_psi`, `wellhead_pressure_psi`, `wellhead_temperature_f`, `tubing_pressure_psi`, `uptime_fraction`, `rare_event_flag`. **`scada_telemetry.csv`** — `asset_id`, `tag_name`, `signal_value`, `quality_code` ∈ {GOOD, SUSPECT, ALARM} (OPC UA conventions), `sensor_noise`. **`cybersecurity_events.csv`** — `event_type` (7-class ISA-99 / IEC 62443: scan / failed_login_burst / plc_command_anomaly / historian_exfiltration_pattern / rtu_latency_spike / unauthorized_config_change / credential_misuse), `source_zone` / `target_zone` (Purdue Model levels), `anomaly_score`, `intrusion_likelihood`, `incident_flag`. **`digital_twin_labels.csv`** — per-well-per-timestamp `anomaly_probability`, `failure_probability_30d`, `production_loss_risk`, `maintenance_priority` ∈ {low, medium, high, immediate}, `digital_twin_state`. --- ## Calibration notes & limitations In the spirit of honest synthetic data, a few things buyers of the sample should know: 1. **Reservoir-type taxonomy coverage at n=6.** Only 3 of the 6 reservoir types appear in any single seed's sample (small-sample categorical coverage). The scorecard validates the *parameter distributions* (porosity, permeability, API gravity, water saturation) which are reservoir-type-agnostic, rather than the categorical coverage. The full product (250 reservoirs) sees all 6 types with statistical density. 2. **Cyber event taxonomy coverage at n≈15.** Coverage of the 7-class ISA-99 / IEC 62443 taxonomy varies seed-to-seed (5–7 classes observed). Scorecard floor lowered to ≥ 5 with this disclosed. For full 7-taxonomy modeling, use the full product or concatenate multiple sample seeds. 3. **Failure-event taxonomy coverage at n=10.** Only 6 of the 10 failure modes appear in any single seed's sample. Failure-mode count is intentionally sparse (rare events). For full taxonomy training, use the full product or multi-seed concat. 4. **Uptime fraction median ~0.88.** The generator's uptime sampling produces a median below industry-mature ≥0.95. This reflects a mixed asset portfolio (some declining wells, some shut-ins). For "best-in- class only" analytics, filter to `uptime_fraction > 0.95`. 5. **Alarm event types simplified.** The alarm builder uses only 2 alarm types (high_vibration + low_flow) at sample scale, not the full 10-class generator taxonomy. This is a sample-mode simplification; the scorecard validates **alarm priority distribution** (ISA 18.2 high + critical share at 15%) rather than alarm-type taxonomy. 6. **Alarm response time median is ~80 minutes.** This is *much* slower than the ISA 18.2 target of 1–10 minutes for high/critical alarms. The current generator simulates a degraded-operator-load scenario. Filter to `operator_actions.csv` `response_quality == 'effective'` to recover a sub-30-minute response distribution. 7. **Operator actions are biased to acknowledge (~92%) over manual_override (~8%).** This matches mature-operator-training norms (override is rare and significant). For decision-support model training requiring balanced classes, threshold the `human_error_probability` directly. 8. **Cyber/environmental/operator-action tables are sparse (15–60 rows).** These are event tables intentionally sized as rare events. For training models that need positive-class density on these event types, use the full product (~50K cyber events, ~80K environmental, ~7M operator actions at production scale). 9. **Deterministic seeding.** All 18 tables are deterministic on `--seed`. Catalog default is seed 42. Seed sweep verifies Grade A+ across {42, 7, 123, 2024, 99, 1}. --- ## Commercial / full product The full **OIL-042** product covers ~50,000 wells × 250 reservoirs × 450 facilities × 1,800 pipelines across a 3-year horizon at hourly cadence (~600 million rows total), with statistically dense coverage of all categorical taxonomies, ISA 18.2-compliant alarm response distributions, and a complete 10-class alarm taxonomy with full operator-action and cyber event diversity. Available under commercial license — contact [[e-mail]](mailto:[e-mail]). XpertSystems.ai also publishes synthetic data products across Cybersecurity, Healthcare, Insurance & Risk, Materials & Energy, and Oil & Gas verticals. Catalog: [huggingface.co/xpertsystems](https://huggingface.co/xpertsystems).OIL-042 is a synthetic integrated oilfield digital twin dataset focused on the upstream oil and gas sector. It includes 18 CSV tables covering the complete upstream digital twin: 5 dimensional masters (fields/reservoirs/wells/facilities/pipelines), 5 telemetry streams (reservoir/production/lift/surface/pipeline/SCADA), and 7 event tables (alarms/workorders/failures/environmental/cyber/operator actions/labels). The dataset supports cross-layer causal modeling, such as reservoir pressure decline → artificial lift degradation → operator intervention → production loss → spare parts demand. Data is calibrated to multiple industry standards (e.g., SPE, API, ISA, IEC) and includes a validation scorecard for quality assurance. Use cases include reservoir-to-surface causal modeling, cross-layer anomaly detection, cyber-physical attack modeling, methane emissions modeling, operator-action modeling, artificial lift optimization, and more.

搜集汇总

数据集介绍

构建方式

在油气上游数字化转型进程中，集成化数字孪生数据集的构建需以多学科物理规律为锚点。OIL-042样本数据集由XpertSystems.ai合成数据平台生成，采用确定性种子机制按采样模式构建，涵盖120口井、6个储层、8座地面设施及14条管道，时间跨度90天、时间粒度为6小时，共计18张CSV表格、约15.4万行数据。其分布参数严格参照SPE石油工程手册、AAPG、API MPMS 2540、PHMSA管道事故统计、ISA 18.2、IEC 62443及ISO 14224等公开权威文献进行标定，并通过十项指标验证记分卡在多个随机种子下复核，确保储层物性、流体性质、管道泄漏率及SCADA数据质量等关键统计量落入行业合理区间。

特点

该数据集的核心特质在于其端到端跨层集成架构，将地下储层物理、井筒生产、人工举升、地面集输、SCADA控制、报警管理、维护可靠性、环境监测、网络安全及操作员行为等十类业务域统一于同一模式框架之下。全部18张表可基于field_id、reservoir_id、well_id、facility_id、pipeline_id及时间戳等主键实现逐时间戳关联，支持从储层压力衰竭到举升退化、再到生产损失与备件需求的跨层因果建模。数据集还内置digital_twin_labels表，提供逐井逐时刻的异常概率、30天失效概率、生产损失风险及维护优先级标签，为监督学习与风险评估提供直接标注。

使用方法

使用该数据集时，可通过Hugging Face datasets库或pandas直接从仓库加载指定CSV文件，各表以主键和外键构成关系型结构，时间序列流均按小时对齐，便于执行多表联结与因果图构建。典型应用涵盖储层至地面的多级状态空间建模、跨层异常检测与SCADA质量码关联分析、基于Purdue模型的OT/IT网络攻击影响评估、甲烷与二氧化碳排放核算、人工举升系统退化优化以及操作员响应行为研究。需注意样本模式下部分类别分类覆盖率有限，事件表行数稀疏，若需完整分类体系或高正类密度训练，应使用完整商业版本或合并多个种子样本。

背景与挑战

背景概述

油气上游领域的数字化转型长期受制于多源异构数据的割裂，地下储层、井筒生产、地面设施、SCADA控制与运维安全等环节各自为政，难以支撑跨层因果建模。XpertSystems.ai于2024年前后推出OIL-042，系其油气垂直领域首个端到端综合性数字孪生数据集，由该公司合成数据平台构建。该数据集以储层至地面的完整上游链条为骨架，通过18张产品表将地下物理、井筒作业、地面基础设施、OT控制、维护可靠性、环境监测与网络安全等层面有机联结，并依据SPE、API、ISA、PHMSA、ISO等权威标准进行分布校准，为跨层异常检测、人工举升优化、甲烷排放核算及OT网络安全建模等研究提供了可复现的合成数据底座，对推动油气数字孪生与工业智能研究具有显著影响力。

当前挑战

该数据集所应对的核心领域难题在于，传统油气数据集多为孤立点解决方案，缺乏跨储层—井筒—地面—控制—运维的贯通式因果链条，致使跨层耦合建模、罕见事件预测与攻击影响评估难以开展。构建层面则面临多重挑战：合成数据须在统计分布上忠实还原真实油气系统，同时严格契合SPE、API、ISA 18.2、IEC 62443、PHMSA、ISO 14224等多元标准；罕见事件（设备失效、网络攻击、甲烷泄漏）样本稀疏，分类覆盖度受限；报警响应时间、人工操作偏差、uptime分布等需在保真度与可用性间权衡；此外，样本版仅呈现部分分类体系，完整分类学需依赖全量产品或多种子拼接方能获得统计密度。

常用场景

经典使用场景

在油气上游数字化转型的浪潮中，集成化数字孪生已成为贯通地下储层与地面设施的关键技术范式。OIL-042数据集凭借其跨层可连接的时间序列结构，经典地服务于从储层到地面的因果推理任务：reservoir_telemetry、well_production、surface_facilities与pipeline_flows等表按时间戳对齐，支撑图神经网络、多层级状态空间模型及因果图模型跨越地下—地面边界进行建模，从而揭示压力衰竭、举升退化与产量损失的传导机制。

衍生相关工作

作为XpertSystems.ai油气系列中首个端到端集成数字孪生产品，OIL-042为后续衍生工作奠定了基座。其数据生成器催生了多轮种子扫描的验证评分卡方法，推动了合成数据质量评估的标准化；跨层因果建模与OT/IT安全分析的公开基准亦由此发端，启发了异常检测、剩余寿命预测及网络物理攻击影响评估等方向的经典研究，并与既有的井筒、地震、安全及合规等点状数据集形成互补与集成。

数据集最近研究

最新研究方向

油田数字孪生领域的前沿研究正致力于构建涵盖地下储层至地面管网的全链条因果推断模型，以揭示跨层级的复杂耦合机制。oil042-sample作为首个端到端集成式合成数字孪生数据集，为储层压力衰减、人工举升退化、操作干预与产量损失之间的跨层因果建模提供了可追溯的时序基底，推动了图神经网络与多级状态空间模型在油气上游的融合应用。与此同时，该数据集支持的OT/IT网络攻击物理影响建模，呼应了Industroyer与TRITON等针对工业控制系统的安全事件所引发的紧迫需求，为关键基础设施防御研究提供了标签丰富的仿真环境。在甲烷排放核算与SEC气候披露规则趋严的背景下，其环境监测表为温室气体清单建模与减排策略优化提供了统计校准的合成样本，彰显了合成数据在合规分析与运营决策中的桥梁价值。