LLMs and Stack Overflow Discussions: Reliability, Impact, and Challenges

Since the release of ChatGPT in November 2022, the landscape of developer Q&A platforms, particularly Stack Overflow, has undergone significant changes. The ability of large language models (LLMs) to generate immediate, human-like responses to technical questions has started discussions on their potential to replace traditional Q&A platforms. This dataset was collected as part of an empirical study analyzing Stack Overflow questions and evaluating responses generated by ChatGPT and LLaMA.

The dataset supports research aimed at:

Assessing the reliability of LLM-generated answers and their potential long-term impact on platforms like Stack Overflow.

Tracking the evolution of user engagement with Stack Overflow post-ChatGPT’s release.

Comparing the performance of ChatGPT and LLaMA across different topics.