Supplementary Material for Paper "Distilling Event Sequence Knowledge From Large Language Models"

Supplementary Material for Paper:Distilling Event Sequence Knowledge From Large Language ModelsSomin Wadhwa, Oktie Hassanzadeh, Debarun Bhattacharjya, Ken Barker, and Jian Ni

Appendix:- Appendix.pdf: contains our prompts and a description of our human evaluation details.

Data:- base_kg_v7.jsonl: Our Wikidata-based Event Causal Knowledge Graph.

Outputs:- sample_new_patterns_discovered.txt: examples of observed new patters through application of sequential pattern mining algorithms, described in section 3.- precision_eval_sample.txt: examples of output evaluated with a precision-evaluator model. - bsumm_output.txt: sample outputs of identified influencing events through the application of binary summary markov model, described in section 5.2.

Code:- src/generator.py: ingests ICL prompts and generates requisite event sequences.- src/benchmarking.py: ingests a _trained_ Flan-style seq2seq model to evaluate precision, and recall from the base KG.- src/utils.py: utilities for generator and benchmarking.

To cite:

@inproceedings{WadhwaHBBN24,
author = {Somin Wadhwa and
Oktie Hassanzadeh and
               Debarun Bhattacharjya and
               Ken Barker and
               Jian Ni},
title = {Distilling Event Sequence Knowledge From Large Language Models},
booktitle = {The Semantic Web - 23rd International Conference, {ISWC} 2024},
series = {Lecture Notes in Computer Science},
publisher = {Springer},
year = {2024},
}