遇见数据集

SimilCode Benchmark: A Dual-Language Source-Code Similarity Dataset for the Evaluation of AI-Based Detectors

收藏
Zenodo2026-11-04 更新2026-11-05 收录
官方服务:

资源简介:

SimilCode Benchmark is a controlled dataset of 120 source-code pairs in C# and Java (15 reference programs per language, four similarity categories: identical, functional, structural and different) for evaluating source-code similarity detectors in programming education. It includes the raw results of a replicated evaluation of four commercial large language models (GPT-5.5, DeepSeek-V4-pro, Gemini 3.1 Pro and Claude Opus 5; 1,440 evaluations, three replicas per pair and model) and the outputs of Moss, JPlag and Dolos on the same pairs; a classification of the pairs by code-clone type; a second run with 84 copies of the reference programs rewritten with generative AI (by a local open-weight model and by commercial models), verified by execution and scored together with independent solutions to the same exercises (1,368 LLM evaluations and the three detectors), including the refusals of the generators; an analysis of how well each dimensional score of the language models, their weighted global score and a derivation score that excludes functional similarity separate copies from independent solutions; the validation corpus and predictions of the Big O estimator of SimilCode; and scripts that reproduce the reported numbers without calling any API. Contents, protocol and reproduction instructions: see README.md, CHANGELOG.md and copias_ia/README_COPIAS_IA.md in the archived repository.

提供机构:
Zenodo
创建时间:
2026-11-04
二维码
社区交流群
二维码
科研交流群
商业服务