Mindbyte-89/btcusdt-microbar-v2

---
language:
- en
license: mit
pretty_name: BTCUSDT Microbar v2
tags:
- finance
- cryptocurrency
- market-microstructure
- binance
- futures
- btcusdt
size_categories:
- 10M<n<100M
---

# BTCUSDT Microbar v2

Sub-candle microstructure data for Binance USD-M Futures BTCUSDT, collected continuously over six WebSocket streams. Successor to [`Torch-Trade/btcusdt-microbar`](https://huggingface.co/datasets/Torch-Trade/btcusdt-microbar).

A standard OHLCV candle compresses thousands of trades into 6 numbers. This dataset preserves the raw event-level data — every individual trade, every best bid/ask change, every depth snapshot — so the underlying microstructure features can be reconstructed at any timeframe.

## Why v2?

In April 2026 we discovered that the v1 collector had a silent routing bug. Binance USD-M Futures split market data across two WebSocket routing paths (`/public` and `/market`) — the legacy unified endpoint silently delivered streams that mapped to `/public` and dropped the rest **without error**.

Concretely, in v1:

| Stream | v1 status | v2 status |
|----------------|----------------------------------------|-----------|
| `trade` | working | working |
| `bookTicker` | working | working |
| `depth5@500ms` | partial (missing during routing migration) | working |
| `markPrice` | **broken** (subscriptions silently dropped) | **fixed** |
| `miniTicker` | **broken** (subscriptions silently dropped) | **fixed** |
| `forceOrder` | **broken** (subscriptions silently dropped) | **fixed** |

If you only need `trades` and `book_ticks`, the v1 dataset remains usable. If you need funding rate, mark price, 24h stats, liquidations, or full depth — use v2. We renamed the repo (rather than appending) so consumers get a clean discontinuity instead of a silent quality jump.

The fix is in `binance-microbar` collector commit `f88a52a` (group streams by routing path and open one WebSocket per group).

## Layout

```
data/btcusdt/
├── trades/<YYYY-MM-DD>/<HHMMSS>.parquet # individual trades
├── book_ticks/<YYYY-MM-DD>/<HHMMSS>.parquet # best bid/ask updates
├── depth/<YYYY-MM-DD>/<HHMMSS>.parquet # top-5 order book snapshots
├── liquidations/<YYYY-MM-DD>/<HHMMSS>.parquet # forced liquidations
├── mark_price/<YYYY-MM-DD>/<HHMMSS>.parquet # mark price + funding
└── mini_ticker/<YYYY-MM-DD>/<HHMMSS>.parquet # 24h rolling stats
```

Files are flushed to disk every 60 seconds. Each row carries a `timestamp_ms` (exchange clock, UTC) which is the only safe key for joins — wall-clock arrival times are not preserved.

## Schemas

### `trades/`
| column | type | description |
|-------------------|---------|-----------------------------------------------|
| `timestamp_ms` | int64 | exchange trade time, ms since epoch UTC |
| `price` | float64 | trade price (USDT) |
| `quantity` | float64 | trade quantity (BTC) |
| `is_buyer_maker` | bool | True = aggressive sell, False = aggressive buy |

Volume: ~50 trades/sec → ~4M rows/day.

### `book_ticks/`
| column | type | description |
|----------------|---------|------------------------------------------|
| `timestamp_ms` | int64 | exchange event time |
| `bid_price` | float64 | best bid price |
| `bid_qty` | float64 | best bid quantity |
| `ask_price` | float64 | best ask price |
| `ask_qty` | float64 | best ask quantity |

Volume: ~100 updates/sec → ~8M rows/day.

### `depth/`
Top 5 order book levels. Columns: `timestamp_ms`, `bid_price_0..4`, `bid_qty_0..4`, `ask_price_0..4`, `ask_qty_0..4`. Snapshots arrive every 500 ms → ~170k rows/day.

### `liquidations/`
| column | type | description |
|----------------|---------|------------------------------------------------------------|
| `timestamp_ms` | int64 | liquidation time |
| `price` | float64 | average fill price |
| `quantity` | float64 | liquidated size |
| `side` | string | `"BUY"` = short squeezed (bullish), `"SELL"` = long liquidated (bearish) |

Sparse — bursts during volatile moves.

### `mark_price/`
Updates every 3 seconds. Columns: `timestamp_ms`, `mark_price`, `index_price`, `funding_rate`, `next_funding_time_ms`. Funding rate is the most predictive macro signal in crypto futures.

### `mini_ticker/`
Updates every second. Columns: `timestamp_ms`, `open_24h`, `high_24h`, `low_24h`, `close`, `volume_24h`, `quote_volume_24h`. Distance from 24h high/low is a support/resistance signal.

## Loading

```python
from huggingface_hub import snapshot_download
import pandas as pd
from pathlib import Path

local = snapshot_download(
"Torch-Trade/btcusdt-microbar-v2",
repo_type="dataset",
allow_patterns=["trades/2026-04-29/*.parquet"],
)

trades = pd.concat(
pd.read_parquet(f) for f in sorted(Path(local, "trades/2026-04-29").glob("*.parquet"))
)
```

To compute aggregated microstructure features over arbitrary timeframes, use the [`binance-microbar`](https://github.com/TorchTrade/binance-microbar) library — `examples/build_feature_dataset.py` rebuilds 54-feature ML-ready datasets directly from these raw streams.

## Collection

- **Source**: Binance USD-M Futures public WebSocket streams (no auth required)
- **Collector**: [`binance-microbar`](https://github.com/TorchTrade/binance-microbar) at commit `f88a52a` or later
- **Host**: continuous collection on a Raspberry Pi 5 (`colony1`), uploads daily at 03:00 UTC
- **Coverage**: starts 2026-04-29; the immediate prior period (2026-04-28 → 2026-04-29) is missing because the collector was offline for the routing-fix deployment

## License

MIT. Market data is sourced from Binance's public WebSocket streams and is provided as-is. No financial advice; not affiliated with Binance.

---
语言：
- 英语
许可证：
- MIT
展示名称：
- BTCUSDT微棒v2
标签：
- 金融
- 加密货币
- 市场微观结构（market microstructure）
- 币安（Binance）
- 期货
- BTCUSDT
数据规模分类：
- 1000万<数据量<1亿
---

# BTCUSDT微棒v2

本数据集为币安USDⓈ保证金合约BTCUSDT的蜡烛线内微观结构数据，通过6路WebSocket数据流持续采集而成，是 [`Torch-Trade/btcusdt-microbar`](https://huggingface.co/datasets/Torch-Trade/btcusdt-microbar) 的升级版数据集。

标准的OHLCV（开盘价、最高价、最低价、收盘价、成交量）蜡烛图会将数千笔交易压缩为6组数值。本数据集保留了原始的事件级数据——每一笔独立交易、每一次最优买卖盘变动、每一份深度快照——因此可在任意时间粒度下重构底层微观结构特征。

## 为何推出v2版本？

2026年4月，我们发现v1版本的采集程序存在隐性路由漏洞。币安USDⓈ保证金合约的市场数据通过两路WebSocket路由路径（`/public`与`/market`）分发，而旧版统一端点会静默仅推送匹配`/public`的数据流，其余数据流则被丢弃且**无任何错误提示**。

具体而言，v1版本的表现如下：

| 数据流名称 | v1版本状态 | v2版本状态 |
|----------------|----------------------------------------|-----------|
| `trade` | 正常运行 | 正常运行 |
| `bookTicker` | 正常运行 | 正常运行 |
| `depth5@500ms` | 部分可用（路由迁移期间存在数据缺失） | 正常运行 |
| `markPrice` | **失效**（订阅请求被静默丢弃） | **修复** |
| `miniTicker` | **失效**（订阅请求被静默丢弃） | **修复** |
| `forceOrder` | **失效**（订阅请求被静默丢弃） | **修复** |

若仅需`trades`与`book_ticks`数据流，v1版本数据集仍可使用。若需要资金费率、标记价格、24小时统计数据、强平订单或完整深度数据，请使用v2版本。我们对仓库进行了重命名（而非追加版本后缀），以此让使用者清晰感知数据集的更新断点，而非隐性的质量提升。

该漏洞修复已在`binance-microbar`采集程序的`f88a52a`提交记录中实现（按路由路径分组数据流，并为每组单独建立一个WebSocket连接）。

## 数据布局

data/btcusdt/
├── trades/<YYYY-MM-DD>/<HHMMSS>.parquet # 单笔交易数据
├── book_ticks/<YYYY-MM-DD>/<HHMMSS>.parquet # 最优买卖盘更新数据
├── depth/<YYYY-MM-DD>/<HHMMSS>.parquet # 前五档订单簿快照
├── liquidations/<YYYY-MM-DD>/<HHMMSS>.parquet # 强制平仓订单数据
├── mark_price/<YYYY-MM-DD>/<HHMMSS>.parquet # 标记价格与资金费率数据
└── mini_ticker/<YYYY-MM-DD>/<HHMMSS>.parquet # 24小时滚动统计数据

数据文件每60秒刷入磁盘一次。每行数据均包含`timestamp_ms`字段（交易所时钟时间，UTC时区），这是唯一可靠的关联键——原始数据未保留数据到达的挂钟时间。

## 数据模式

### `trades/` 目录

| 字段名 | 数据类型 | 描述 |
|-------------------|---------|-----------------------------------------------|
| `timestamp_ms` | int64 | 交易所交易时间，距UTC纪元的毫秒数 |
| `price` | float64 | 交易价格（单位：USDT） |
| `quantity` | float64 | 交易数量（单位：BTC） |
| `is_buyer_maker` | bool | 字段为`True`时代表主动卖单，`False`时代表主动买单 |

数据量：约50笔交易/秒 → 日均约400万行数据。

### `book_ticks/` 目录

| 字段名 | 数据类型 | 描述 |
|----------------|---------|------------------------------------------|
| `timestamp_ms` | int64 | 交易所事件发生时间 |
| `bid_price` | float64 | 最优买盘价格 |
| `bid_qty` | float64 | 最优买盘挂单数量 |
| `ask_price` | float64 | 最优卖盘价格 |
| `ask_qty` | float64 | 最优卖盘挂单数量 |

数据量：约100次更新/秒 → 日均约800万行数据。

### `depth/` 目录

包含前五档订单簿数据。字段包括：`timestamp_ms`、`bid_price_0..4`、`bid_qty_0..4`、`ask_price_0..4`、`ask_qty_0..4`。快照每500毫秒更新一次 → 日均约17万行数据。

### `liquidations/` 目录

| 字段名 | 数据类型 | 描述 |
|----------------|---------|------------------------------------------------------------|
| `timestamp_ms` | int64 | 强平事件发生时间 |
| `price` | float64 | 平均成交价格 |
| `quantity` | float64 | 被强平的合约数量 |
| `side` | string | 字段为`"BUY"`时代表空头被轧空（利好行情），`"SELL"`时代表多头被强平（利空行情） |

数据分布稀疏 — 仅在市场剧烈波动时会出现批量数据。

### `mark_price/` 目录

每3秒更新一次。字段包括：`timestamp_ms`、`mark_price`、`index_price`、`funding_rate`、`next_funding_time_ms`。资金费率是加密货币期货中最具预测性的宏观信号。

### `mini_ticker/` 目录

每1秒更新一次。字段包括：`timestamp_ms`、`open_24h`、`high_24h`、`low_24h`、`close`、`volume_24h`、`quote_volume_24h`。与24小时高低价的差值可作为支撑位/阻力位信号。

## 数据加载

python
from huggingface_hub import snapshot_download
import pandas as pd
from pathlib import Path

local = snapshot_download(
"Torch-Trade/btcusdt-microbar-v2",
repo_type="dataset",
allow_patterns=["trades/2026-04-29/*.parquet"],
)

trades = pd.concat(
pd.read_parquet(f) for f in sorted(Path(local, "trades/2026-04-29").glob("*.parquet"))
)

若需在任意时间粒度下计算聚合微观结构特征，可使用[`binance-microbar`](https://github.com/TorchTrade/binance-microbar)库——`examples/build_feature_dataset.py`可直接从这些原始数据流重构出具备54个特征的机器学习可用数据集。

## 数据采集

- **数据来源**：币安USDⓈ保证金合约的公共WebSocket数据流（无需身份验证）
- **采集程序**：[`binance-microbar`](https://github.com/TorchTrade/binance-microbar)的`f88a52a`提交记录或更新版本
- **部署环境**：在树莓派5（主机名`colony1`）上持续采集，每日UTC 03:00进行数据上传
- **数据覆盖范围**：始于2026年4月29日；紧邻的前一时间段（2026年4月28日至29日）因采集程序在部署路由修复补丁时离线而缺失

## 许可证

采用MIT许可证。市场数据来源于币安公共WebSocket数据流，按“原样”提供。本数据集不构成任何投资建议，与币安无任何关联关系。