TopoChat Evaluation Datasets and Results for Materials Science Question-Answering System

This repository contains the evaluation datasets and experimental results for TopoChat, a knowledge-enhanced question-answering framework for materials science. The data includes:

📊 Evaluation Datasets:

MaScQA benchmark dataset with 650 questions and ground truth answers (download from MaScQA)

TopoQA dataset with 200 specialized topological materials questions and answers (created on our own and we will continuously update)

📈 Experimental Results:

Performance metrics on MaScQA (68.62% accuracy)

Performance metrics on TopoQA (65% accuracy)

Comparison results with baseline models (GPT4, LLama3.1 70B)

Ablation study results demonstrating the contribution of each component

This dataset supports the research paper "TopoChat: Enhancing Topological Materials Retrieval With Large Language Model and Multi-Source Knowledge" and can be used to reproduce our experimental results or as benchmarks for future research in materials science question-answering systems. We welcome community contributions and feedback to improve the dataset quality.