Robustness of large language models in moral judgments

Large language models (LLMs) are used for an increasing variety of tasks, some of which may even have effects on decision making. Therefore, there has been an increasing interest in understanding how societal norms and moral judgments may be reflected in the output of LLMs. Recent work has therefore tested LLMs on various moral judgment tasks and drawn conclusions regarding the similarities between LLMs and humans. The present contribution critically assesses the validity of the method and results employed in previous work for eliciting moral judgments from LLMs. We find that previous results are confounded by biases in the presentation of the options in moral judgment tasks, and that LLM responses are highly sensitive to prompt formulation variants as simple as changing \"Case 1\" and \"Case 2\" to \"(A)\" and \"(B)\". Our results hence indicate that previous conclusions on moral judgments of LLMs cannot be upheld. We make recommendations for more sound methodological setups for future studies..., Adapt the automated data generation process of the Takemoto (2024) paper., , # Robustness of large language models in moral judgments

This repository has three experiments.

1. Revised the data generation code in a balanced way.
2. Prompt variations for evaluating the robustness of the LLMs in dilemma situations.
3. Prompt variations for evaluating the robustness of the LLMs in non-dilemma situations.

Details are as belows.

We revised the data generation code in a balanced way and added prompt variations for evaluating the robustness of the LLMs. Moreover, we conducted further prompt variations for evaluating robustness of the large language models on value-laden tasks. We found limitations of LLMs in performing complex moral reasoning, particularly when required to simultaneously process multiple moral values (e.g., young (versus old) AND female (versus male) AND fit (versus large), etc.).

Logically, there could be different reasons for the inconsistency in model responses. It could be the case that the models are simply not able to properly follow the task...