Replication Package for the paper "Conversing with business process-aware Large Language Models: the BPLLM framework"

Replication Package for the research paper "Conversing with business process-aware Large Language Models: the BPLLM framework".

The package includes the process models, the questions (and expected answers), the results of the qualitative evaluation, and the Hugging Face links to the fine-tuned versions of Llama 3.1 8B employed in the quantitative evaluation of the framework.

In particular, the process models are:

The natural language Directly-follows graph (DFG) of the Food Delivery process: food_delivery_activities.txt for the definition of the activities and food_delivery_flow.txt for the sequence flow.

The BPMN model of the Food Delivery, E-commerce, and Reimbursement processes: ecommerce.bpmn, food_delivery.bpmn, and reimbursement.bpmn.

The datasets with the questions and the expected answers are:

1_questions_answers_not_refined_for_DFG.csv ;

1.1_questions_answers_refined_for_DFG.csv ;

2_questions_answers_not_refined.csv ;

3_questions_answers_refined.csv ;

4_questions_answers_different_processes.csv ;

5_questions_answers_similar_processes.csv ;

6_questions_answers_refined_ft.csv .

The complete results of the qualitative evaluation are contained in the file qualitative_experiments_results.pdf.

The Hugging Face links to the fine-tuned versions of Llama 3.1 8B are reported in hf_links_finetuned_models.pdf.

本数据集为研究论文《面向业务流程感知大语言模型的对话：BPLLM框架》（Conversing with business process-aware Large Language Models: the BPLLM framework）的复现包。本复现包包含业务流程模型、问题（含预期答案）、定性评估结果，以及用于该框架定量评估的Llama 3.1 8B微调版本的Hugging Face链接。具体而言，本复现包所涵盖的业务流程模型如下：
1. 餐饮配送流程的自然语言直接跟随图（Directly-follows graph, DFG）：活动定义详见food_delivery_activities.txt，序列流程定义详见food_delivery_flow.txt；
2. 餐饮配送、电子商务与报销流程的业务流程模型与符号（Business Process Model and Notation, BPMN）模型，对应文件分别为ecommerce.bpmn、food_delivery.bpmn及reimbursement.bpmn。

包含问题与预期答案的数据集如下：
1_questions_answers_not_refined_for_DFG.csv、1.1_questions_answers_refined_for_DFG.csv、2_questions_answers_not_refined.csv、3_questions_answers_refined.csv、4_questions_answers_different_processes.csv、5_questions_answers_similar_processes.csv、6_questions_answers_refined_ft.csv。

定性评估的完整结果收录于qualitative_experiments_results.pdf文件中。用于该框架定量评估的Llama 3.1 8B微调版本的Hugging Face链接，详见hf_links_finetuned_models.pdf文件。