libero_plus_rlds

<h1 align="center">
LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
</h1>

<p align="center">
📄 <a href="https://huggingface.co/papers/2510.13626"><strong>Paper</strong></a> |
🏗️ <a href="https://github.com/sylvestf/LIBERO-plus"><strong>Repo</strong></a> |
🌐 <a href="https://sylvestf.github.io/LIBERO-plus"><strong>Website</strong></a>
</p>


## 🔥 Overview
This repository contains the official implementation and benchmark for our paper "In-depth Robustness Analysis for Vision-Language-Action Models". We systematically expose the hidden vulnerabilities of contemporary VLA models through comprehensive robustness evaluation across seven perturbation dimensions. You can simply replace the original `libero` with a `pip install -e .` without modifying your code.

## 🚀 Key Findings
- **Significant Fragility**: VLA models exhibit extreme sensitivity to camera viewpoints and robot initial states, with performance dropping from 95% to below 30% under modest perturbations
- **Language Ignorance**: Models largely ignore language instructions, functioning more like Vision-Action models
- **Negative Compositional Generalization**: Combined perturbations reveal complex interaction effects beyond independent factors

## 📊 LIBERO-plus Benchmark

### 7 Perturbation Dimensions
We introduce **LIBERO-plus**, a comprehensive benchmark with 10,030 tasks spanning:

1. **Objects Layout** - Confounding objects and target object displacement
2. **Camera Viewpoints** - Position, orientation, and field-of-view changes
3. **Robot Initial States** - Manipulator initial pose variations
4. **Language Instructions** - LLM-based instruction rewriting
5. **Light Conditions** - Intensity, direction, color, and shadow variations
6. **Background Textures** - Scene and surface appearance changes
7. **Sensor Noise** - Photometric distortions and image degradation

### Evaluated Models
- OpenVLA and variants (OFT, OFT_w, OFT_m)
- π₀ and π₀-fast
- Nora, WorldVLA, UniVLA, RIPT-VLA

## 🛠️ Installation
Please refer to our [github repo](https://github.com/sylvestf/LIBERO-plus) for more installation details.
You can download our training dataset mentioned in our paper from this hf repo. You can also find the [assets](https://huggingface.co/datasets/Sylvest/LIBERO-plus) and the [OpenVLA-OFT weights after mix-SFT](https://huggingface.co/Sylvest/openvla-7b-oft-finetuned-libero-plus-mixdata/tree/main) on this dataset.

The extracted directory structure should look like:

```text
LIBERO-plus/
└── libero/
└── libero/
└── assets/
├── articulated_objects/
├── new_objects/
├── scenes/
├── stable_hope_objects/
├── stable_scanned_objects/
├── textures/
├── turbosquid_objects/
├── serving_region.xml
├── wall_frames.stl
└── wall.xml
```

## 🔧 Evaluation
The evaluation method is almost identical to `LIBERO`. The only required modification is adjusting `num_trials_per_task` from 50 to 1 in your configuration.

<h1 align="center">LIBERO-Plus：视觉-语言-动作模型的深度鲁棒性分析</h1>

<p align="center">
📄 <a href="https://huggingface.co/papers/2510.13626"><strong>论文</strong></a> |
🏗️ <a href="https://github.com/sylvestf/LIBERO-plus"><strong>代码仓库</strong></a> |
🌐 <a href="https://sylvestf.github.io/LIBERO-plus"><strong>项目主页</strong></a>
</p>


## 🔥 项目概述
本仓库包含我们发表的论文《视觉-语言-动作模型的深度鲁棒性分析》的官方实现与基准测试套件。我们通过覆盖七个扰动维度的全面鲁棒性评估，系统性地揭示了当前主流视觉-语言-动作（Vision-Language-Action, VLA）模型潜藏的安全漏洞。用户仅需执行`pip install -e .`即可完成安装，无需修改原有代码，直接替换原始的`libero`即可使用。

## 🚀 核心发现
- **极强的脆弱性**：视觉-语言-动作模型对相机视角与机器人初始状态极度敏感，在施加适度扰动后，模型性能可从95%骤降至30%以下
- **语言指令无视性**：模型大多忽略语言指令，其表现更接近纯视觉-动作模型
- **负向组合泛化能力**：复合扰动实验揭示了独立扰动因素之外的复杂交互效应

## 📊 LIBERO-Plus 基准测试套件

### 七大扰动维度
我们推出了**LIBERO-Plus**基准测试套件，该套件包含10030个任务，覆盖以下七类扰动：

1. **物体布局**：干扰目标物体与其他物体的位置排布
2. **相机视角**：相机位置、朝向与视场角的变化
3. **机器人初始状态**：机械臂初始位姿的偏移
4. **语言指令**：基于大语言模型（Large Language Model, LLM）的指令重写
5. **光照条件**：光照强度、方向、色彩与阴影的变化
6. **背景纹理**：场景与物体表面外观的变更
7. **传感器噪声**：光度畸变与图像降质

### 测试模型列表
- OpenVLA 及其变体（OFT、OFT_w、OFT_m）
- π₀ 与 π₀-fast
- Nora、WorldVLA、UniVLA、RIPT-VLA

## 🛠️ 安装说明
详细安装步骤请参阅我们的[GitHub代码仓库](https://github.com/sylvestf/LIBERO-plus)。
您可从该Hugging Face仓库下载论文中提及的训练数据集，也可在该数据集仓库中获取[配套资源](https://huggingface.co/datasets/Sylvest/LIBERO-plus)以及[经过混合监督微调后的OpenVLA-OFT权重](https://huggingface.co/Sylvest/openvla-7b-oft-finetuned-libero-plus-mixdata/tree/main)。

解压后的目录结构应如下所示：
text
LIBERO-plus/
└── libero/
└── libero/
└── assets/
├── articulated_objects/
├── new_objects/
├── scenes/
├── stable_hope_objects/
├── stable_scanned_objects/
├── textures/
├── turbosquid_objects/
├── serving_region.xml
├── wall_frames.stl
└── wall.xml

## 🔧 评估流程
评估流程与原始`LIBERO`基准基本一致，仅需在配置文件中将`num_trials_per_task`从50调整为1即可。