BPC: A Benchmark Dataset for Causal Business Process Reasoning

Large Language Models (LLMs) are increasingly used for boosting organizational efficiency and automating tasks.

While not originally designed for complex cognitive processes, recent efforts have further extended to employ LLMs in activities such as reasoning, planning,

and decision-making. In business processes, such abilities could be invaluable for leveraging the massive corpora LLMs have been trained on for gaining a deep understanding of such processes. In adherence to this goal, we attach here the BP^C dataset, a newly developed set of process-aware Q&A that can be used to assess the ability of LLMs to reason about causal and process perspectives of business operations.

We refer to this view as Causally-augmented Business Processes (BP^C). The benchmark comprises a set of domain-specific BP^C related situations,

a set of questions about these situations, and a set of ground truth answers to these questions.

Reasoning on BP^C is of crucial importance for process interventions and process improvement.

The benchmark could be used in one of two possible modalities: testing the performance of any target LLM and training an LLM to advance its capability to reason about BP^C.