CAS-SIAT-XinHai/CPsyExam
收藏资源简介：
--- language: zh license: cc-by-4.0 datasets: - CPsyExam --- # CPsyExam Dataset CPsyExam is a comprehensive dataset designed for evaluating Chinese psychological examination capabilities in large language models. ## Dataset Description The CPsyExam dataset is structured to assess psychological examination competencies in a Chinese context. It provides a standardized way to evaluate how well language models can handle psychological assessment tasks. The dataset contains multiple-choice questions from various psychological domains, including clinical and counseling psychology. ## News 🔥 - **December 9, 2024**: Official website launched at [https://cpsyexam.xinhai.co](https://cpsyexam.xinhai.co) - **November 30, 2024**: Our paper "CPsyExam: Chinese Benchmark for Evaluating Psychology using Examinations" has been accepted by COLING 2025! 🎉 ## Data Format The dataset is organized in JSON format with the following splits: - Training set - Validation set - Test set Each example in the dataset follows this structure: ```json { "subject_name": "心理学科目名称", "question_type": "multi/single", "kind": "knowledge", "question": "题目内容", "options": { "A": "选项A内容", "B": "选项B内容", "C": "选项C内容", "D": "选项D内容", "E": "选项E内容" }, "explanation": "题目解析说明", "id": "唯一标识ID" } ``` ### Field Descriptions: - `subject_name`: The specific subject area in psychology - `question_type`: Type of question (e.g., "multi" for multiple choice) - `kind`: Category of the question (e.g., "knowledge") - `question`: The actual question text - `options`: A dictionary containing the multiple choice options (A through E) - `explanation`: Detailed explanation of the correct answer - `id`: Unique identifier for the question ## Usage ### Loading the Dataset ```python from datasets import load_dataset dataset = load_dataset("CAS-SIAT-XinHai/CPsyExam") ``` ### Data Preparation for Fine-tuning For those interested in using the dataset for fine-tuning, you can prepare the data according to the [SFT](https://github.com/CAS-SIAT-XinHai/CPsyExam/blob/main/README.md#sft) section. ## Leaderboard Track model performance and submit your results at our [leaderboard](http://cpsyexam.xinhai.co/leaderboard). ## Citation If you use this dataset in your research, please cite: ```bibtex @misc{CPsyExam benchmark, title={CPsyExam: Chinese Benchmark for Evaluating Psychology using Examinations}, author={Jiahao Zhao, Jingwei Zhu, Minghuan Tan, Min Yang, Di Yang, Chenhao Zhang, Guancheng Ye, Chengming Li, Xiping Hu}, year = {2024}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/CAS-SIAT-XinHai/CPsyExam}} } ``` ## License This dataset is licensed under CC-BY-4.0. ## Contact For more information, visit our [official website](http://cpsyexam.xinhai.co).
CPsyExam is a comprehensive dataset designed for evaluating Chinese psychological examination capabilities in large language models. It contains multiple-choice questions from various psychological domains, including clinical and counseling psychology, aiming to standardize the evaluation of language models performance in handling psychological assessment tasks.
