"Beyond LLM-as-a-Judge: A Multi-LLM Framework for Reliable Chatbot Evaluation"

"This dataset is a structured CSV file generated through an automated chatbot evaluation pipeline. It contains evaluation logs of chatbot responses assessed using both single LLM and multi-LLM (ensemble-based) evaluation approaches.The dataset includes key fields such as the input question, corresponding chatbot response, evaluator type (single LLM or multi-LLM), evaluation dimension (e.g., conversational quality, reasoning and factuality, safety alignment, robustness and consistency), run identifier, and the numerical score assigned by each evaluator.The primary purpose of this dataset is to analyze and compare the reliability, consistency, and behavior of different LLM-based evaluation strategies. It is specifically designed to highlight the limitations of single LLM evaluation and demonstrate the advantages of consensus-based multi-LLM evaluation.This dataset can be used for statistical analysis, benchmarking evaluation frameworks, studying inter-evaluator variability, and developing more robust AI evaluation systems. It is also suitable for research in trustworthy AI, ensemble evaluation methods, and automated quality assessment of conversational agents."