Chaff from the Wheat: Characterizing and Detecting Risky Pre-trained Models

Introduction This is the replication package for the paper "Chaff from the Wheat: Characterizing and Detecting Risky Pre-trained Models". Task Definition Given the information of pre-trained models (PTMs), the task is to identify whether it is a risky PTM. We treat the task as binary classification (0/1), where 1 stands for risky PTMs and 0 for benign PTMs. File Information Description dataset.zip: This folder contains the dataset, which includes the collected dataset, real-world dataset, and sample dataset. evaluation.zip: This folder includes the performance evaluation scripts. models.zip: This folder includes the weight of codebert and codet5, used to train the RPTMHunter. result.zip: This folder includes the experimental results. script.zip: This folder includes the scripts of proposed automated risky PTM detection framework (RPTMHunter) and the dataset collection script. interviews.zip: This folder contains the content of interviews. readme.md: This is a readme file, including the detailed information of dataset and script steps to conduct the experiment.