FinTabNet_OTSL

# Dataset Card for FinTabNet_OTSL

## Dataset Description

- **Homepage:** https://ds4sd.github.io
- **Paper:** https://arxiv.org/pdf/2305.03393

### Dataset Summary

This dataset is a conversion of the original [FinTabNet](https://developer.ibm.com/exchanges/data/all/fintabnet/) into the OTSL format presented in our paper "Optimized Table Tokenization for Table Structure Recognition". The dataset includes the original annotations amongst new additions.

### Dataset Structure

* cells: origunal dataset cell groundtruth (content).
* otsl: new reduced table structure token format
* html: original dataset groundtruth HTML (structure).
* html_restored: generated HTML from OTSL.
* cols: grid column length.
* rows: grid row length.
* image: PIL image

### OTSL Vocabulary:

**OTSL**: new reduced table structure token format
More information on the OTSL table structure format and its concepts can be read from our paper.
Format of this dataset extends work presented in a paper, and introduces slight modifications:

* "fcel" - cell that has content in it
* "ecel" - cell that is empty
* "lcel" - left-looking cell (to handle horizontally merged cells)
* "ucel" - up-looking cell (to handle vertically merged cells)
* "xcel" - 2d span cells, in this dataset - covers entire area of a merged cell
* "nl" - new line token

### Data Splits

The dataset provides three splits
- `train`
- `val`
- `test`

## Additional Information

### Dataset Curators

The dataset is converted by the [Deep Search team](https://ds4sd.github.io/) at IBM Research.
You can contact us at [[e-mail]](mailto:[e-mail]).

Curators:
- Maksym Lysak, [@maxmnemonic](https://github.com/maxmnemonic)
- Ahmed Nassar, [@nassarofficial](https://github.com/nassarofficial)
- Christoph Auer, [@cau-git](https://github.com/cau-git)
- Nikos Livathinos, [@nikos-livathinos](https://github.com/nikos-livathinos)
- Peter Staar, [@PeterStaar-IBM](https://github.com/PeterStaar-IBM)

### Citation Information

```bib
@misc{lysak2023optimized,
title={Optimized Table Tokenization for Table Structure Recognition},
author={Maksym Lysak and Ahmed Nassar and Nikolaos Livathinos and Christoph Auer and Peter Staar},
year={2023},
eprint={2305.03393},
archivePrefix={arXiv},
primaryClass={cs.CV}
}```

# FinTabNet_OTSL 数据集卡片

## 数据集说明
- **项目主页**：https://ds4sd.github.io
- **论文**：https://arxiv.org/pdf/2305.03393

### 数据集概述
本数据集是将原始[FinTabNet](https://developer.ibm.com/exchanges/data/all/fintabnet/)转换为我们发表于论文《面向表格结构识别的优化表格Token化》中的OTSL格式，在保留原始标注的基础上新增了补充标注内容。

### 数据集结构
* cells：原始数据集单元格真值（内容）
* otsl：全新精简版表格结构Token格式
* html：原始数据集真值HTML（结构）
* html_restored：基于OTSL生成的HTML文件
* cols：表格网格列数
* rows：表格网格行数
* image：PIL图像

### OTSL词表
**OTSL**：全新精简版表格结构Token格式。有关OTSL表格结构格式及其相关概念的更多细节可参阅我们的论文。本数据集的格式在已有研究成果的基础上进行了扩展，并引入了小幅调整：
* "fcel"：带有内容的单元格
* "ecel"：空单元格
* "lcel"：左向关联单元格（用于处理水平合并单元格）
* "ucel"：上向关联单元格（用于处理垂直合并单元格）
* "xcel"：二维跨度单元格，在本数据集中用于覆盖合并单元格的全部区域
* "nl"：换行Token

### 数据划分
本数据集包含三种数据划分方式：
- `train`：训练集
- `val`：验证集
- `test`：测试集

## 附加信息

### 数据集制作方
本数据集由IBM研究院[Deep Search团队](https://ds4sd.github.io/)转换制作。您可通过邮箱[[e-mail]](mailto:[e-mail])与我们取得联系。

制作团队成员：
- Maksym Lysak，[@maxmnemonic](https://github.com/maxmnemonic)
- Ahmed Nassar，[@nassarofficial](https://github.com/nassarofficial)
- Christoph Auer，[@cau-git](https://github.com/cau-git)
- Nikos Livathinos，[@nikos-livathinos](https://github.com/nikos-livathinos)
- Peter Staar，[@PeterStaar-IBM](https://github.com/PeterStaar-IBM)

### 引用信息
bib
@misc{lysak2023optimized,
title={Optimized Table Tokenization for Table Structure Recognition},
author={Maksym Lysak and Ahmed Nassar and Nikolaos Livathinos and Christoph Auer and Peter Staar},
year={2023},
eprint={2305.03393},
archivePrefix={arXiv},
primaryClass={cs.CV}
}

搜集汇总

背景与挑战

背景概述

FinTabNet_OTSL数据集是原始FinTabNet数据集的OTSL格式转换版本，用于表格结构识别研究，包含了原始注释和新添加的OTSL格式数据。数据集提供了三种分割（train, val, test）和详细的OTSL词汇表，由IBM Research团队维护并附有相关论文引用。