Limits of ChatGPT's Conversational Pragmatics in a Turing Test About Ethics, Commonsense, and Cultural Sensitivity

Does ChatGPT deliver its explicit claim to be culturally sensitive and its implicit claim to be a friendly digital person when conversing with human users? These claims are investigated from the perspective of linguistic pragmatics, particularly Grice's cooperative principle in communication. Following the pattern of real-life communication, turn-taking conversations reveal limitations in the LLM's grasp of the entire contextual setting described in the prompt. The prompts included ethical issues, a hiking adventure, geographical orientation and bodily movement. For cultural sensitivity the prompts came from a Pakistani Muslim in English language, from a Hindu in English, and from a Chinese in Chinese language. The issues were deeply cultural issues involving feelings and affects. Qualitative analysis of the conversation pragmatics showed that ChatGPT is often unable to conduct conversations according to the pragmatic principles of quantity, reliable quality, remaining in focus, and being clear in expression. We conclude that ChatGPT should not be presented as a global LLM but be subdivided into several culture-specific modules.ChatGPT在与人类用户对话时，能否兑现其"具备文化敏感性"的显性宣称，以及"作为友善数字人"的隐性宣称？本研究从语言语用学（linguistic pragmatics）视角，尤其是格赖斯交际合作原则（Grice's cooperative principle in communication）出发，对上述两项宣称展开系统性探究。遵循现实交际的话轮交替模式，对话轮次能够揭示大语言模型（Large Language Model）对提示词所描述的完整语境设定的把握存在显著局限。本次研究所采用的提示词涵盖伦理议题、徒步探险、地理定位与肢体动作四类场景。针对文化敏感性验证环节，提示词分别来自以英语表达的巴基斯坦穆斯林、以英语表达的印度教徒，以及以汉语表达的华人，所涉议题均为牵涉情感与情绪的深层文化议题。通过对对话语用特征的质性分析可知，ChatGPT时常无法恪守交际合作原则下的四大语用准则：数量准则、质量准则、话题关联准则与表达明晰准则。本研究最终得出结论：不应将ChatGPT定位为通用型大语言模型，而应将其拆分为多个适配特定文化的专属模块。