siyich/spacetools-rlpointtools

---
license: apache-2.0
task_categories:
- visual-question-answering
- image-to-text
tags:
- spatial-reasoning
- robotics
- tool-use
- reinforcement-learning
- grpo
size_categories:
- 1K<n<10K
---

# SpaceTools RL Point-Tools Dataset

RL training data for spatial reasoning with point-based tool calling (roborefer detect_one).

## Dataset Summary

- **Train**: 4,000 samples (refspatial 2k + robospatial 2k)
- **Test**: 350 samples (mixed spatial reasoning)
- **Format**: verl-compatible parquet with embedded base64 images

## Data Sources

| Split | Source | Samples |
|-------|--------|---------|
| RefSpatial - object | RefSpatial 3D | 500 |
| RefSpatial - vacant | RefSpatial 3D | 500 |
| RefSpatial - spatial | RefSpatial 3D | 500 |
| RefSpatial - ABdepth | RefSpatial 3D | 500 |
| RoboSpatial - configuration | RoboSpatial | 500 |
| RoboSpatial - compatibility | RoboSpatial | 500 |
| RoboSpatial - context | RoboSpatial | 500 |
| RoboSpatial - grounding | RoboSpatial | 500 |

## Schema

Each row contains:
- `data_source`: Dataset identifier for reward routing
- `prompt`: List of chat messages (system + user with embedded examples)
- `images`: List of base64-encoded images (data URI format)
- `ability`: Task ability tag
- `reward_model`: Ground truth and scoring style
- `extra_info`: Question text, answer, split, qa_type

## Usage with verl

```python
# Single-file usage
from datasets import load_dataset
ds = load_dataset("siyich/spacetools-rlpointtools")

# Or directly with parquet in verl config:
# data.train_files=["path/to/train.parquet"]
# data.val_files="path/to/test.parquet"
```

## Training Setup

Used with [verl](https://github.com/volcengine/verl) GRPO training:
- **Model**: Qwen2.5-VL-3B-Instruct (base)
- **Tool**: roborefer.detect_one (point detection)
- **Algorithm**: GRPO with kl_loss_coef=0.001
- **Reward**: Negative distance to centroid (point), convex hull (bbox)
- **Prompt style**: reprompt6 (2 few-shot examples with tool-call demonstrations)