jamesding0302/memgen-annotations

--- task_categories: - other configs: - config_name: AmazonReviews2014-Beauty data_files: - split: val path: AmazonReviews2014-Beauty/val.jsonl - split: test path: AmazonReviews2014-Beauty/test.jsonl - config_name: AmazonReviews2014-Sports_and_Outdoors data_files: - split: val path: AmazonReviews2014-Sports_and_Outdoors/val.jsonl - split: test path: AmazonReviews2014-Sports_and_Outdoors/test.jsonl - config_name: AmazonReviews2023-Industrial_and_Scientific data_files: - split: val path: AmazonReviews2023-Industrial_and_Scientific/val.jsonl - split: test path: AmazonReviews2023-Industrial_and_Scientific/test.jsonl - config_name: AmazonReviews2023-Musical_Instruments data_files: - split: val path: AmazonReviews2023-Musical_Instruments/val.jsonl - split: test path: AmazonReviews2023-Musical_Instruments/test.jsonl - config_name: AmazonReviews2023-Office_Products data_files: - split: val path: AmazonReviews2023-Office_Products/val.jsonl - split: test path: AmazonReviews2023-Office_Products/test.jsonl - config_name: Steam data_files: - split: val path: Steam/val.jsonl - split: test path: Steam/test.jsonl - config_name: Yelp-Yelp_2020 data_files: - split: val path: Yelp-Yelp_2020/val.jsonl - split: test path: Yelp-Yelp_2020/test.jsonl --- # MemGen Annotations This is the annotation dataset for the paper **[How Well Does Generative Recommendation Generalize?](https://huggingface.co/papers/2603.19809)**. <a href="https://huggingface.co/papers/2603.19809"><img src="https://img.shields.io/badge/Paper-ArXiv-red"></a> <a href="https://github.com/Jamesding000/MemGen-GR"><img src="https://img.shields.io/badge/Code-GitHub-green"></a> <a href="https://huggingface.co/jamesding0302/memgen-checkpoints"><img src="https://img.shields.io/badge/Models-Hugging%20Face-blue"></a> The annotations categorize evaluation instances under the leave-one-out protocol: - **test** split uses the **last** item in the user history sequence as target, - **val** split uses the **second-to-last** item as target. ## Columns - `sample_id`: row index within the split in the original dataset. - `user_id`: raw user identifier (join key). - `master`: one of `memorization`, `generalization`, `uncategorized`. - `subcategories`: list of `{rule, hop}` for fine-grained generalization types. - `all_labels`: all string labels (e.g., `["generalization", "symmetry_3"]`). ## Load in M&G annotations ```python from datasets import load_dataset labels = load_dataset( "jamesding0302/memgen-annotations", "AmazonReviews2014-Beauty", split="test", ) print(labels[0]) ``` ## Merge with processed dataset ```python # 1) Load your processed dataset split (must be aligned with labels by row order) ds = pipeline.split_datasets["test"] # 2) Append label columns to the original dataset ds = (ds .add_column("master", labels["master"]) .add_column("subcategories", labels["subcategories"]) .add_column("all_labels", labels["all_labels"])) ```--- task_categories: - 其他 configs: - config_name: AmazonReviews2014-Beauty data_files: - split: val path: AmazonReviews2014-Beauty/val.jsonl - split: test path: AmazonReviews2014-Beauty/test.jsonl - config_name: AmazonReviews2014-Sports_and_Outdoors data_files: - split: val path: AmazonReviews2014-Sports_and_Outdoors/val.jsonl - split: test path: AmazonReviews2014-Sports_and_Outdoors/test.jsonl - config_name: AmazonReviews2023-Industrial_and_Scientific data_files: - split: val path: AmazonReviews2023-Industrial_and_Scientific/val.jsonl - split: test path: AmazonReviews2023-Industrial_and_Scientific/test.jsonl - config_name: AmazonReviews2023-Musical_Instruments data_files: - split: val path: AmazonReviews2023-Musical_Instruments/val.jsonl - split: test path: AmazonReviews2023-Musical_Instruments/test.jsonl - config_name: AmazonReviews2023-Office_Products data_files: - split: val path: AmazonReviews2023-Office_Products/val.jsonl - split: test path: AmazonReviews2023-Office_Products/test.jsonl - config_name: Steam data_files: - split: val path: Steam/val.jsonl - split: test path: Steam/test.jsonl - config_name: Yelp-Yelp_2020 data_files: - split: val path: Yelp-Yelp_2020/val.jsonl - split: test path: Yelp-Yelp_2020/test.jsonl --- # MemGen 标注数据集 本数据集为论文**《生成式推荐的泛化性能如何？》**（https://huggingface.co/papers/2603.19809）的配套标注数据集。 <a href="https://huggingface.co/papers/2603.19809"><img src="https://img.shields.io/badge/论文-ArXiv-红色"></a> <a href="https://github.com/Jamesding000/MemGen-GR"><img src="https://img.shields.io/badge/代码-GitHub-绿色"></a> <a href="https://huggingface.co/jamesding0302/memgen-checkpoints"><img src="https://img.shields.io/badge/模型-Hugging%20Face-蓝色"></a> 本标注基于留一法（leave-one-out）协议对评估样本进行分类： - **测试划分（test split）** 以用户历史交互序列的**最后一项**作为预测目标 - **验证划分（val split）** 以用户历史交互序列的**倒数第二项**作为预测目标 ## 列定义 - `sample_id`：原始数据集中对应划分内的行索引 - `user_id`：原始用户标识符（关联键） - `master`：标注主类别，可选值为`记忆化（memorization）`、`泛化（generalization）`与`未分类（uncategorized）` - `subcategories`：细粒度泛化类型的`{rule, hop}`列表 - `all_labels`：所有字符串形式的标注标签（例如`["generalization", "symmetry_3"]`） ## 加载MemGen标注数据集 python from datasets import load_dataset labels = load_dataset( "jamesding0302/memgen-annotations", "AmazonReviews2014-Beauty", split="test", ) print(labels[0]) ## 与处理后数据集合并 python # 1) 加载处理后的数据集划分（需与标注按行序对齐） ds = pipeline.split_datasets["test"] # 2) 将标注列追加至原始数据集 ds = (ds .add_column("master", labels["master"]) .add_column("subcategories", labels["subcategories"]) .add_column("all_labels", labels["all_labels"]))