On Inter-dataset Code Duplication and Data Leakage in Large Language Models

This dataset encompasses the sparse graph referenced in the publication titled "On Inter-dataset Code Duplication and Data Leakage in Large Language Models."

This resource is a snapshot of the original repository, and the graph is preserved in the interduplication.db database. The schema of this database is easily understandable and is available in the original repository. Each code snippet is identified by a unique identifier (id_within_dataset) that corresponds to its identification within the dataset from which it was extracted. The complete datasets are stored in .jsonl files within their respective folders (e.g., python-150/data.jsonl, codetrans/data.jsonl, etc.).