CR-TKGQA

CR-TKGQA A Temporal Knowledge Graph Question Answering Dataset Involving Complex Reasoning Dataset Split of CR-TKGQA: train dev test test_sample1000_seed1: benchmark for methods in our paper, use "random.seed(1) random.sample(test)" to generate Domain of CR-TKGQA: id question question_tagged: Question with entities and literals marked answer answer_type: Type of answer, one or more in [Entity, Time, Number, Boolean] topic_entity_label_map: Map of topic entities in question, in the form of {QID : mention} gold_entity_label_map: Map of gold entities in sparql, in the form of {QID : label} gold_relation_label_map: Map of gold properties in sparql, in the form of {PID : label} sparql question_creation_date origin: Process of construction of the question, one in [Seed, Generation, Static Entity Augmentation, Temporal Entity Augmentation, Event Time Augmentation] Extra domain of test: comp_level: Compositional level of the question, one in [iid, compositional, zero-shot] answer_entity_labels: labels and alians of gold answer entities, used for evaluation of DirectQA & RTQA linking_entity_label_map: Map of linking entities, employing GPT-4o-mini to extract entity mentions from the question, retrieving the top-5 candidate entities via the Wikidata API (https://www.wikidata.org/w/api.php) for each mention, and then using GPT-4o-mini to select the most plausible entity for each mention For Analysis Please turn to analysis/sorted_dataset_analysis.py. The environment needed for this script is simple, you only need to install tqdm and networkx==3.4.2. Results are in analysis_results. run analysis/sorted_dataset_analysis.py to get the results in Tab.3 (# calculate_splits_statics), Tab.4 (# analysis_temporal_taxonomy & # analysis_split_complexity) and Tab.5 (# result_analysis) run analysis/statistic.py to get the results in Tab.6