foundation-models/golden-batch-sentinel-data

---
license: cc-by-4.0
task_categories:
- time-series-forecasting
- tabular-classification
tags:
- bioprocess
- manufacturing
- anomaly-detection
- fault-detection
- batch-monitoring
- pharma
pretty_name: Golden Batch Sentinel Data
size_categories:
- 1M<n<10M
---

# Golden Batch Sentinel Data

Benchmark datasets for process monitoring and fault detection in batch manufacturing.

## Datasets

### IndPenSim (Industrial Penicillin Simulation)

A 100,000L fermentation simulation with 100 batches and rich multivariate signals.

- **Source**: [Mendeley Data](https://data.mendeley.com/datasets/pdnjz7zz5x/2)
- **Paper**: [Modern day monitoring and control challenges...](https://doi.org/10.1016/j.compchemeng.2018.05.019)
- **Batches**: 100 (90 normal, 10 faulty)
- **Variables**: 37 process variables (Raman spectra excluded for efficiency)
- **Time resolution**: 0.2 hours

**Files:**
- `indpensim/batches.parquet` - Main batch data
- `indpensim/statistics.parquet` - Batch statistics and fault labels

### Tennessee Eastman Process (TEP)

The most common benchmark for fault detection in multivariate industrial processes.

- **Source**: [Harvard Dataverse](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/6C3JR1)
- **Fault types**: 20 different fault scenarios
- **Variables**: 52 (41 measured + 11 manipulated)

**Files:**
- `tep/fault_free_train.parquet` - Normal operation (training)
- `tep/fault_free_test.parquet` - Normal operation (testing)
- `tep/faulty_train.parquet` - Faulty operation (training, all 20 faults)
- `tep/faulty_test.parquet` - Faulty operation (testing, all 20 faults)

## Usage

```python
from datasets import load_dataset

# Load IndPenSim
indpensim = load_dataset("foundation-models/golden-batch-sentinel-data", data_dir="indpensim")

# Load TEP
tep = load_dataset("foundation-models/golden-batch-sentinel-data", data_dir="tep")

# Or load specific files
import pandas as pd
from huggingface_hub import hf_hub_download

path = hf_hub_download(
repo_id="foundation-models/golden-batch-sentinel-data",
filename="indpensim/batches.parquet",
repo_type="dataset"
)
df = pd.read_parquet(path)
```

## License

The original datasets are provided under their respective licenses:
- IndPenSim: CC BY 4.0
- TEP: Public domain

This compilation is provided under CC BY 4.0.

## Citation

If you use this data, please cite the original papers:

```bibtex
@article{goldrick2019modern,
title={Modern day monitoring and control challenges outlined on an industrial-scale benchmark fermentation process},
author={Goldrick, Stephen and others},
journal={Computers \& Chemical Engineering},
year={2019}
}

@article{downs1993plant,
title={A plant-wide industrial process control problem},
author={Downs, James J and Vogel, Ernest F},
journal={Computers \& Chemical Engineering},
year={1993}
}
```