sidnarsipur/onnx-inference

---
license: mit
---

# ONNX-Predict Dataset

This dataset is derived from [ONNX-Predict](https://github.com/sidnarsipur/onnx-predict), a dataset of ONNX model inference performance measurements collected from models hosted on the ONNX Model Zoo.

It contains inference data for 2,300+ ONNX models. Each model was optimized into four variants, and inference measurements were collected across 16 machines with different CPU core counts, memory sizes, cache configurations, and CPU providers.

## Dataset Overview

Each row in the dataset combines three groups of features:

1. **Graph features** describing the ONNX model structure and estimated data movement
2. **Hardware features** describing the machine used for inference
3. **Inference measurements** describing observed runtime performance

## Features

The dataset includes:

- **Graph features:** 133 features
- Node counts: 127 features
- Data movement and compute estimates: 6 features

- **Hardware features:** 9 features

- **Inference data:** 4 target/measurement columns

## Graph Features

Graph features describe the structure and computational characteristics of each ONNX model.

### Node Count Features

There are 127 node-count features, grouped by operation type.

#### Convolution Operations

Examples include:

- `Conv`
- `QLinearConv`
- `ConvTranspose`

#### Matrix Multiplication Operations

Examples include:

- `MatMul`
- `Gemm`

#### Elementwise Operations

Examples include:

- `Relu`
- `Sigmoid`
- `Tanh`
- `Mul`
- `Div`

#### Reduction and Pooling Operations

Examples include:

- `MaxPool`
- `AveragePool`
- Other reduction-style operators

#### Normalization Operations

Examples include:

- `Softmax`
- `BatchNormalization`

#### Data Movement Operations

Examples include:

- `Transpose`
- `Reshape`
- Other tensor layout or shape transformation operations

#### Other Operations

This group includes operators related to:

- Control flow
- Constants
- Shape transformations
- Boolean logic
- Miscellaneous ONNX graph operations

### Compute and Data Movement Features

The dataset also includes estimated compute and data movement features:

- `conv_flops`
- `matmul_flops`
- `elementwise_mb`
- `reduction_mb`
- `movement_mb`
- `normalization_mb`

These features provide additional information about the expected computational and memory behavior of each model.

## Hardware Features

Hardware features describe the machine used to run inference.

Available hardware features include:

- `memory_mb`
- `l1d_cache_kb`
- `l1i_cache_kb`
- `l2_cache_kb`
- `base_clock_mhz`
- `num_cores`
- `memory_bandwidth_gbs`
- `cpu_provider`

The `cpu_provider` field identifies the CPU vendor, such as:

- `amd`
- `intel`

## Inference Data

Inference measurements summarize observed runtime latency for each model, optimization variant, and hardware configuration.

The dataset includes the following inference columns:

- `average_ms`
- `stddev_ms`
- `min_ms`
- `max_ms`

All latency measurements are reported in milliseconds.

## Source

The dataset is based on data from the ONNX-Predict project:

[https://github.com/sidnarsipur/onnx-predict](https://github.com/sidnarsipur/onnx-predict)

## License

This dataset is released under the MIT License.