Sample of ARABIC OPINION SUMMARY EVALUATION DATASET (AOSED)

This dataset contains a sample of our Arabic opinion summary evaluation framework. It includes 5 restaurant reviews from our complete collection of 100 restaurants, with corresponding reference summaries. The data showcases human evaluation metrics for three LLMs (GPT-4o, Claude, and JAIS) across relevance, faithfulness, and sentiment preservation dimensions. The dataset also includes automated evaluation results using our proposed ArabicOpinion framework, with Matryoshka embeddings at the opinion level showing the best performance for relevance and faithfulness assessment. For sentiment analysis, we utilize Arabert, which is fine-tuned specifically on restaurant reviews. This sample demonstrates our methodology for evaluating Arabic opinion summarization systems.