SSHAFER/agency-personalities-trails

--- pretty_name: Personalities-Trails language: - en - zh license: other license_name: research-only-custom license_link: https://huggingface.co/datasets/SSHAFER/agency-personalities-trails/blob/main/LICENSE task_categories: - text-generation - feature-extraction tags: - literature - character-analysis - roleplay - rag - bilingual - research size_categories: - 10K<n<100K configs: - config_name: en-traits data_files: - split: train path: en-traits/*.json - config_name: en-retrieval data_files: - split: train path: en-retrieval/*.json - config_name: en-retrieval+en-traits data_files: - split: train path: en-retrieval+en-traits/*.json - config_name: ch-traits data_files: - split: train path: ch-traits/*.json - config_name: ch-retrieval data_files: - split: train path: ch-retrieval/*.json - config_name: ch-retrieval+ch-traits data_files: - split: train path: ch-retrieval+ch-traits/*.json --- # Personalities-Trails ## Overview Personalities-Trails is a bilingual literary analysis dataset for research on artificial agency, literary character modeling, retrieval-augmented generation, and role-playing evaluation. The dataset is built from selected literary works and organized into multiple subsets for detailed trait analysis, retrieval-oriented structured summaries, and merged settings that combine both views. This repository contains processed research data only, including structured annotations, metadata, and limited text excerpts. It does not provide complete literary works and should not be treated as a substitute for the original publications. ## Dataset Summary - Root directory: `resource/` - Total JSON files: `412` - Approximate size: `~5.3 GB` - Estimated total records: `50,000+` - Languages: Chinese and English ### Subsets | Subset | Files | Size | Language | Description | |------|------:|------:|------|------| | `en-traits/` | 90 | 1.3 GB | English | Full English literary analysis | | `en-retrieval/` | 37 | 869 MB | English | English analysis with retrieval-oriented short summaries | | `en-retrieval+en-traits/` | 90 | 1.7 GB | English | Merged English subset | | `ch-traits/` | 90 | 664 MB | Chinese | Full Chinese literary analysis | | `ch-retrieval/` | 15 | 100 MB | Chinese | Chinese analysis with retrieval-oriented short summaries | | `ch-retrieval+ch-traits/` | 90 | 694 MB | Chinese | Merged Chinese subset | Other files currently present in the directory include `dataset_comparison.xlsx`, `fig.pptx`, and `find_en.py`. ## Data Structure All JSON files use a top-level array. Each element is a sample containing instructions, source text, outputs, and metadata. ### Common Fields ```json { "instruction": { "intro": "Prompt for intro analysis", "personalities_trails": "Prompt for character trait analysis", "self_awareness": "Prompt for self-awareness analysis", "scene": "Prompt for scene analysis" }, "text": "Source literary excerpt", "input": "", "output": { "intro": "Structured analysis of character/location/background/event", "personalities_trails": "Detailed character profile", "self_awareness": "Self-awareness analysis", "scene": "Scene analysis" }, "metadata": { "element_id": "Unique identifier", "filename": "Source EPUB filename", "languages": "eng / zho" } } ``` ### Retrieval-Specific Field Some retrieval-related subsets contain an additional `output-short` field for structured summaries. ```json { "output-short": { "scenario": { "place": "Location", "background": "Background", "event": "Event" }, "people": [ { "character-profile": { "name": "Character name", "sketch": "Character sketch" }, "literary-characterization": { "appearance": "Appearance", "language": "Language style", "action": "Behavior", "psychology": "Psychology", "demeanor": "Demeanor" }, "psychological-analysis": { "perspective-on-life": "View of life" } } ] } } ``` Note: the `scenario` field appears in English retrieval files; Chinese retrieval files may only contain the `people` field under `output-short`. ## Subset Types | Type | Example Directories | Fields | Purpose | |------|------|------|------| | `traits` | `en-traits/`, `ch-traits/` | Common fields | Detailed literary analysis | | `retrieval` | `en-retrieval/`, `ch-retrieval/` | Common fields + `output-short` | Analysis plus compact structured summaries | | `retrieval+traits` | `en-retrieval+en-traits/`, `ch-retrieval+ch-traits/` | Combined content | Merged subsets for broader use | ## Example Statistics - Example file: `traits_1984.epub.json` contains `926` records - Naming pattern: `traits_[book-title].epub.json` - Main analysis dimensions: `intro`, `personalities_trails`, `self_awareness`, `scene` ## Source and Construction The dataset is derived from EPUB-format books spanning Chinese and English literary works, including both classic and modern titles. Literary passages are processed into structured annotations with detailed outputs (`output`) and, in some subsets, short summaries (`output-short`). ## Intended Use This dataset is intended for non-commercial research use, including: - literary character modeling - artificial agency research - retrieval-augmented generation experiments - role-playing and character simulation evaluation - analysis of trait representation and self-perception ## Prohibited Use This dataset must not be used for: - commercial use of any kind - commercial training or fine-tuning - reconstructing or substituting the original books - unlawful redistribution of excerpted text - any use that infringes the rights of authors, translators, publishers, or other rights holders ## Copyright and License Notice This dataset contains limited excerpts derived from copyrighted literary works. Rights in the original texts remain with their respective rights holders. Please review the full license terms in [`LICENSE`](./LICENSE). If you are a rights holder and believe any content should be revised or removed, please contact the maintainer. ## Usage ### Download the full dataset ```bash git lfs install git clone https://huggingface.co/datasets/your-username/personalities-trails ``` ### Download selected subsets ```python from huggingface_hub import snapshot_download local_dir = snapshot_download( repo_id="your-username/personalities-trails", repo_type="dataset", allow_patterns=[ "en-traits/*", "README.md", "LICENSE", ], ) print(local_dir) ``` ### Load a JSON file directly ```python import json from pathlib import Path path = Path("en-traits/traits_1984.epub.json") with path.open("r", encoding="utf-8") as f: data = json.load(f) print(len(data)) print(data[0]["metadata"]) ``` ### Load with `datasets` ```python from datasets import load_dataset dataset = load_dataset( "json", data_files="en-traits/traits_1984.epub.json", split="train", ) print(dataset[0]["text"]) ``` ## Limitations - The dataset includes only limited excerpts rather than complete literary works. - Redistribution constraints may apply because the data is derived from copyrighted books. - Coverage depends on the selected books and processing pipeline, and does not represent all literary traditions or styles. ## Citation If you use this dataset in research, please cite the repository or the associated paper/project page when available.--- 数据集名称：人格特质集（Personalities-Trails） 语言：英语、汉语 许可类型：其他 许可名称：仅用于研究的自定义许可 许可链接：https://huggingface.co/datasets/SSHAFER/agency-personalities-trails/blob/main/LICENSE 任务类别：文本生成、特征提取 标签：文学、角色分析、角色扮演（roleplay）、检索增强生成（Retrieval-Augmented Generation, RAG）、双语研究 样本规模：10K < 样本数 < 100K 配置项： - 配置名称：en-traits 数据文件： - 拆分：训练集 路径：en-traits/*.json - 配置名称：en-retrieval 数据文件： - 拆分：训练集 路径：en-retrieval/*.json - 配置名称：en-retrieval+en-traits 数据文件： - 拆分：训练集 路径：en-retrieval+en-traits/*.json - 配置名称：ch-traits 数据文件： - 拆分：训练集 路径：ch-traits/*.json - 配置名称：ch-retrieval 数据文件： - 拆分：训练集 路径：ch-retrieval/*.json - 配置名称：ch-retrieval+ch-traits 数据文件： - 拆分：训练集 路径：ch-retrieval+ch-traits/*.json --- # 人格特质集（Personalities-Trails） ## 概览 人格特质集（Personalities-Trails）是一个双语文学分析数据集，适用于人工智能主体性（artificial agency）、文学角色建模、检索增强生成（Retrieval-Augmented Generation, RAG）以及角色扮演评估相关研究。 该数据集源自精选文学作品，被划分为多个子集，分别用于精细化特质分析、面向检索的结构化摘要生成，以及融合两种视角的组合配置。 本仓库仅包含经过处理的研究数据，包括结构化标注、元数据与有限文本节选，不提供完整文学作品，不得替代原版出版物。 ## 数据集摘要 - 根目录：`resource/` - JSON文件总数：`412` - 近似总大小：`~5.3 GB` - 预估总样本数：`50,000+` - 支持语言：汉语与英语 ### 子集详情 | 子集名称 | 文件数量 | 占用大小 | 语言 | 描述 | |------|------:|------:|------|------| | `en-traits/` | 90 | 1.3 GB | 英语 | 完整英语文学分析数据集 | | `en-retrieval/` | 37 | 869 MB | 英语 | 附带检索导向型短摘要的英语文学分析数据集 | | `en-retrieval+en-traits/` | 90 | 1.7 GB | 英语 | 融合型英语子集 | | `ch-traits/` | 90 | 664 MB | 汉语 | 完整汉语文学分析数据集 | | `ch-retrieval/` | 15 | 100 MB | 汉语 | 附带检索导向型短摘要的汉语文学分析数据集 | | `ch-retrieval+ch-traits/` | 90 | 694 MB | 汉语 | 融合型汉语子集 | 目录中其他现有文件包括`dataset_comparison.xlsx`、`fig.pptx`与`find_en.py`。 ## 数据结构 所有JSON文件均采用顶级数组格式，每个元素为一条样本，包含指令、源文本、输出与元数据字段。 ### 通用字段 json { "instruction": { "intro": "引言分析提示词", "personalities_trails": "角色特质分析提示词", "self_awareness": "自我意识分析提示词", "scene": "场景分析提示词" }, "text": "文学节选源文本", "input": "", "output": { "intro": "角色/场景/背景/事件结构化分析", "personalities_trails": "精细化角色档案", "self_awareness": "自我意识分析", "scene": "场景分析" }, "metadata": { "element_id": "唯一标识符", "filename": "源EPUB文件名", "languages": "eng / zho" } } ### 检索专属字段 部分与检索相关的子集额外包含`output-short`字段，用于存储结构化摘要。 json { "output-short": { "scenario": { "place": "场景地点", "background": "背景信息", "event": "事件详情" }, "people": [ { "character-profile": { "name": "角色姓名", "sketch": "角色梗概" }, "literary-characterization": { "appearance": "外貌特征", "language": "语言风格", "action": "行为表现", "psychology": "心理活动", "demeanor": "言谈举止" }, "psychological-analysis": { "perspective-on-life": "人生观念" } } ] } } 注意：英语检索文件包含`scenario`字段；汉语检索文件的`output-short`下可能仅包含`people`字段。 ## 子集类型 | 类型 | 示例目录 | 包含字段 | 用途 | |------|------|------|------| | `traits` | `en-traits/`、`ch-traits/` | 通用字段 | 精细化文学分析 | | `retrieval` | `en-retrieval/`、`ch-retrieval/` | 通用字段 + `output-short` | 分析内容附带紧凑结构化摘要 | | `retrieval+traits` | `en-retrieval+en-traits/`、`ch-retrieval+ch-traits/` | 融合内容 | 面向多元应用的融合子集 | ## 示例统计 - 示例文件：`traits_1984.epub.json`包含`926`条样本 - 命名规则：`traits_[书籍标题].epub.json` - 核心分析维度：`intro`、`personalities_trails`、`self_awareness`、`scene` ## 来源与构建 本数据集源自EPUB格式的中英文学作品，涵盖经典与现代出版物。文学节选被处理为结构化标注，包含详细输出（`output`）字段，部分子集额外提供短摘要（`output-short`）字段。 ## 预期用途 本数据集仅用于非商业研究，包括： - 文学角色建模 - 人工智能主体性（artificial agency）研究 - 检索增强生成（Retrieval-Augmented Generation, RAG）实验 - 角色扮演与角色模拟评估 - 特质表征与自我认知分析 ## 禁止用途 本数据集不得用于： - 任何形式的商业用途 - 商业训练或微调 - 复刻或替代原版书籍 - 非法分发节选文本 - 任何侵犯作者、译者、出版商或其他权利持有人权益的行为 ## 版权与许可声明 本数据集包含源自受版权保护的文学作品的有限节选，原文本的所有权利仍归属于对应的权利持有人。 请查阅[`LICENSE`](./LICENSE)中的完整许可条款。若您为权利持有人并认为任何内容需要修改或移除，请联系数据集维护者。 ## 使用方法 ### 下载完整数据集 bash git lfs install git clone https://huggingface.co/datasets/your-username/personalities-trails ### 下载指定子集 python from huggingface_hub import snapshot_download local_dir = snapshot_download( repo_id="your-username/personalities-trails", repo_type="dataset", allow_patterns=[ "en-traits/*", "README.md", "LICENSE", ], ) print(local_dir) ### 直接加载JSON文件 python import json from pathlib import Path path = Path("en-traits/traits_1984.epub.json") with path.open("r", encoding="utf-8") as f: data = json.load(f) print(len(data)) print(data[0]["metadata"]) ### 使用`datasets`库加载 python from datasets import load_dataset dataset = load_dataset( "json", data_files="en-traits/traits_1984.epub.json", split="train", ) print(dataset[0]["text"]) ## 局限性 - 本数据集仅包含有限文本节选，而非完整文学作品。 - 由于数据源自受版权保护的书籍，可能存在分发限制。 - 数据集覆盖范围取决于所选书籍与处理流程，无法代表所有文学传统与风格。 ## 引用 若您在研究中使用本数据集，请在发表成果时引用本仓库或相关论文/项目页面（若可获取）。