Existence proof and improvement study of nontrivial upper bounds on task recognition of large language models

Recent advancements in large language models (LLMs) have significantly enhanced their ability to solve tasks traditionally performed by humans, thereby narrowing the gap between human and artificial intelligence. However, the LLM performance remains highly sensitive to minor variations in prompts. One of the phenomena highlighted in this paper is that providing multiple examples of question-answer pairs can substantially improve LLM performance, even when the answers are randomly assigned. In the domain of in-context learning, LLMs leverage these examples through two mechanisms: task recognition and task learning, with the performance boost from randomly labeled examples attributed primarily to task recognition. This paper posits that the continued reliance on such examples underscores the need for improvement in task recognition. Building on this insight, we propose that this deficiency arises from the long-tailed characteristics of natural language data distribution and the inherent similarities between tasks. A series of experiments were conducted to validate the theoretical assumptions and conclusions. In addition, the impact of several factors on LLMs' task recognition ability was empirically analyzed. Based on the theoretical discussion, we also explore possible directions for improving LLMs' task recognition.