Robustness of large language models in moral judgments

Large language models (LLMs) are used for an increasing variety of tasks, some of which may even have effects on decision making. Therefore, there has been an increasing interest in understanding how societal norms and moral judgments may be reflected in the output of LLMs. Recent work has therefore tested LLMs on various moral judgment tasks and drawn conclusions regarding the similarities between LLMs and humans. The present contribution critically assesses the validity of the method and results employed in previous work for eliciting moral judgments from LLMs. We find that previous results are confounded by biases in the presentation of the options in moral judgment tasks, and that LLM responses are highly sensitive to prompt formulation variants as simple as changing "Case 1" and "Case 2" to "(A)" and "(B)". Our results hence indicate that previous conclusions on moral judgments of LLMs cannot be upheld. We make recommendations for more sound methodological setups for future studies. Methods Adapt the automated data generation process of the Takemoto (2024) paper.