GenAI experiments: Extracting knowledge from educational materials

This collection contains the results of four ClimEmpower / MAIA GenAI experiments. These experiments aim to asess how and to what extent the Generative AI models can help knowledge curators extract knowledge from documents they need to analyse. 

Concrete high-level research questions these experiments aim to resolve are:

RQ1: To what extent can the AI answers be used to formulate the final answers, without reading the whole document?

RQ2: Which types of questions are easier or more difficult for GenAI models to answer?

RQ3: How, and to what extent, can the answers be improved through prompt engineering?

RQ4: To what extent do the GenAI models follow instructions to base the answers (only) on the content provided in the document?

RQ5: How does the choice of GenAI model reflect in experiment results?

In addition, we were also interested in finding out the ways to further improve the SumQA, a Generative AI service that was developed in the MAIA project and supports batch-processing of documents.

本数据集收录了四项ClimEmpower/MAIA生成式AI（Generative AI）实验的成果。这些实验旨在评估生成式AI模型能够以何种方式、在多大程度上辅助知识管理员从待分析文档中提取知识。
本系列实验旨在解决的具体高层级研究问题如下：
RQ1：在无需通读完整文档的前提下，AI生成的答复可在多大程度上用于构建最终答案？
RQ2：哪些类型的问题对于生成式AI模型而言更易或更难作答？
RQ3：通过提示工程能够以何种方式、在多大程度上优化AI生成的答复？
RQ4：生成式AI模型在多大程度上会遵循指令，将答复仅基于给定文档中的内容生成？
RQ5：生成式AI模型的选型如何体现在实验结果中？

此外，本研究还希望探索进一步优化SumQA的路径——SumQA是MAIA项目中开发的一款支持文档批量处理的生成式AI服务。