Explanation Quality Survey - the Tax Refund Case

This entry holds all files related to the survey for elicitation of perceived explanation quality (fidelity and interoperability) that was adapted for the process of tax-return claims.

The following files are disclosed:

data-survey.csv - Survey dataset in CSV format, including reponses from 128 users, each with ratings for 2 inquiry cases.

survey-forms.eus - Survey dataset in .eus format matching the ECSurvey system.

data-survey.xlsx – Survey dataset in excel format.

data-survey-spss.sav – Survey dataset in SPSS format.

survey-forms-g1.pdf - survey-forms-g6.pdf – All survey forms.

Prompts.docx – All LLM prompts corresponding to the 3 inquiry cases.

groups assignments.docx – desciption of all 6 experimental combinations of query number and LLM type.

Generated-explanations.docx – all versions of explanations generated by the LLMs.

LLM-as-a-judge-prompt.docx - a prompt that was employed to get LLM ratings for the explanations.

gpt4o_llama_final_responses_stats.xlsx - responses with LLM and human ratings.