GIZ/sample_datatset

该数据集包含三个部分：1. 5个原始PDF文件；2. 一个parquet文件，其中包含来自上述5个PDF以及其他30个PDF的文本块，并提供了嵌入（使用bge-m3模型）、元数据、文本块和ID列；3. 一个testdata JSON文件，描述了第一版查询重写范围的所有场景，包括相关问题、对应的文本块（通过ID标识）以及查询重写的预期行为。数据集主要用于支持查询重写任务，涵盖多种场景和文本块处理。

This dataset comprises three components: 1. Five raw PDF files; 2. A Parquet file containing text chunks extracted from the aforementioned 5 PDFs plus another 30 PDFs, with columns including embeddings (generated using the bge-m3 model), metadata, text chunks, and IDs; 3. A testdata JSON file that describes all scenarios within the scope of the first version of query rewriting, including relevant questions, corresponding text chunks identified by their IDs, and the expected behavior of query rewriting. The dataset is primarily designed to support query rewriting tasks, covering multiple scenarios and text chunk processing.