SnipGen: A Mining Repository Framework for Evaluating LLMs for Code

This a repostiroy that contains datasets for SnipGEN replication.  Each testbed is contained on a JSON file. There are three JSON file with curated data and tuned for a SE task. The tar.gz contains the raw data collected after mining github repostiries.

The first testbed is the  summarization taks curated. This file contains the name of the repostory the snippet comes from, the file name, the commit message, the snippet and the linked documentation. This testbed is used for completing code from code description and the combination of docstring and code.

The code completion file, contains the used prompts for control, Treatment1 (T1) and Treatment(T2)  and the predicted outcome from chatGPT.