文档层级解析数据集DocHieNet

# The DocHieNet for Document Hierarchy Parsing

This repository contains the dataset used for document hierarchy parsing, including the images, and annotations of layout entities, text, reading order of entities and hierarchical structure for documents.

## Dataset Structure

The dataset is organized into two main parts: images and labels. The directory named 'hres_images' contains document images with higher resolution.

### Annotation Format

Each label file contains the annotations of one document, and the annotations are organized as :

```
{
"pages": {
"page1": {
"width": width of page 1,
"height": height of page 1
},
...

},
"contents":[
{
"box": [x1, y1, x2, y2],
"text": text of the layout entity,
"page": page number,
"label": type of the layout entity,
"linking": [
[
parent_id,
self_id
]
],
"id": self_id,
"order": reading order
},
...
]
}
```
Each item in the 'content' list contains the annotations of a layout entity, such as a paragraph, title, table, header, etc.

### Data Split
The split of training and testing subset is provided in the 'train_test_split.json' file. The split of English and Chinese subset is provided in the 'en_zh_split.json' file.

## How to Use

To set up the dataset, please follow these steps:

```sh
cat dochienet_dataset.zip.part-* > dochienet_dataset.zip
unzip dochienet_dataset.zip
```

This will reconstruct and unpack the dataset files for use.

<!-- ### Download Method:
:modelscope-code[]{type="git"} -->

# 用于文档层次结构解析的DocHieNet
本仓库包含用于文档层次结构解析的数据集，涵盖文档图像、布局实体（layout entity）与文本标注、实体阅读顺序以及文档层级结构标注。

## 数据集结构
本数据集分为两大核心模块：图像与标签。名为`hres_images`的目录存储高分辨率文档图像。

### 标注格式
每个标签文件对应一份文档的标注信息，其组织形式如下：
json
{
"pages": {
"page1": {
"width": 页面1的宽度,
"height": 页面1的高度
},
...
},
"contents":[
{
"box": [x1, y1, x2, y2],
"text": 布局实体的文本内容,
"page": 所属页码,
"label": 布局实体的类型,
"linking": [
[
父实体ID,
当前实体ID
]
],
"id": 当前实体ID,
"order": 阅读顺序
},
...
]
}

`contents`列表中的每一项均对应一个布局实体的标注信息，例如段落、标题、表格、页眉等。

### 数据划分
训练集与测试集的划分信息存储于`train_test_split.json`文件；英语与中文子集的划分信息存储于`en_zh_split.json`文件。

## 使用方法
如需部署并使用该数据集，请执行以下命令：
sh
cat dochienet_dataset.zip.part-* > dochienet_dataset.zip
unzip dochienet_dataset.zip

该命令将重组并解压数据集文件，以供后续使用。

搜集汇总

背景与挑战

背景概述

DocHieNet是一个用于文档层级解析的数据集，包含高分辨率文档图像和详细的布局实体、文本、阅读顺序及层次结构注释。数据集提供了训练/测试子集和英文/中文子集的划分，适用于文档结构分析和理解任务。