Do We Like AI to Show “Thinking”? Revealing the Effect of Chain-of-Thought on Human Perceptions

This research investigates how the display of AI’s “thinking” process, via Chain-of-Thought (CoT), impacts human perceptions while exploring the moderating roles of user motivation and task difficulty. While CoT was originally designed to enhance AI’s reasoning ability, its effect on human-AI interaction remains underexplored. Through a 2 (AI “thinking”: show vs. not show) × 2 (motivation: productivity vs. non-productivity) × 2 (difficulty: easy vs. difficult) between-subjects experiment and a complementary experiment, it was found that displaying “thinking” significantly increased perceived anthropomorphism, indirectly improving user attitudes and continuation intentions. However, contextual factors critically moderated outcomes: showing “thinking” enhanced perceived intelligence, attitudes, and continuation intentions in non-productivity contexts but these effects diminished in productivity-oriented tasks. These results challenge the universal applicability of the Media Equation paradigm, highlighting the context-aware Media Evocation paradigm for human-AI interaction.