YASO

YASO是一个针对开放领域评论的目标情感分析评估数据集，由IBM研究院创建。该数据集包含2215条英文句子，来自数十个评论领域，每条句子都标注了目标词及其情感。YASO的创建过程涉及从多个来源收集用户评论，并通过两阶段标注方案进行标注，确保了标注的可靠性和一致性。该数据集主要用于跨领域的目标情感分析评估，旨在解决现有数据集在跨领域评估中的局限性，为情感分析技术的发展提供更广泛的数据支持。

YASO is an open-domain review-oriented target sentiment analysis evaluation dataset created by IBM Research. It comprises 2215 English sentences sourced from dozens of review domains, with each sentence annotated with its target terms and corresponding sentiment polarities. The development of YASO entails collecting user reviews from multiple sources and adopting a two-stage annotation scheme to ensure the reliability and consistency of annotations. This dataset is primarily intended for cross-domain target sentiment analysis evaluation, aiming to address the limitations of existing datasets in cross-domain assessment and provide broader data support for the advancement of sentiment analysis technologies.

原始信息汇总

YASO: A Targeted Sentiment Analysis Evaluation Dataset for Open-Domain Reviews

数据集内容

• YASO评估数据集：用于目标情感分析（TSA）的评估数据集。
• TSA-MD数据集：新增数据集，详细描述见TSA-MD/README.md。
• 领域标签：新增领域标签，详细信息见changelog。

使用说明

• 数据获取：部分句子来自不可重新分发的数据集，获取原始文本的说明见yaso_tsa/data/README.md。

• 评估代码安装：

  • 使用pip安装： bash pip install git+ssh://[e-mail]/IBM/yaso-tsa.git#egg=yaso-tsa

  • 或克隆代码并安装依赖： bash git clone [e-mail]:IBM/yaso-tsa.git cd yaso-tsa/yaso_tsa pip install -r requirements.txt

• 运行评估：

  • 使用yaso_tsa.evaluate_tsa模块，例如： bash python -m yaso_tsa.evaluate_tsa --predictions_path tests/data/test_data.json --labels_path tests/data/test_labels.json

  • 预期输出示例： text [MainThread] 2021-09-13:16:37:15,137 INFO [evaluate_tsa.py:34] Loaded labeled data: <TsaLabels labeled: 4, sentences: 3> [MainThread] 2021-09-13:16:37:15,190 INFO [evaluate_tsa.py:44] precision=0.6666666666666666 [MainThread] 2021-09-13:16:37:15,190 INFO [evaluate_tsa.py:44] recall=0.6666666666666666 [MainThread] 2021-09-13:16:37:15,190 INFO [evaluate_tsa.py:44] F1=0.6666666666666666

引用

• 如在出版物中使用YASO，请引用以下论文： text Matan Orbach, Orith Toledo-Ronen, Artem Spector, Ranit Aharonov, Yoav Katz and Noam Slonim. 2021. YASO: A Targeted Sentiment Analysis Evaluation Dataset for Open-Domain Reviews. EMNLP.

作者

• 数据集收集：Matan Orbach, Orith Toledo-Ronen, Artem Spector, Ranit Aharonov, Yoav Katz, Noam Slonim。
• 评估代码编写：Matan Orbach 和 Artem Spector。