Chaff from the Wheat: Characterizing and Detecting Risky Pre-trained Models

Introduction

This is the replication package for the paper "Chaff from the Wheat: Characterizing and Detecting Risky Pre-trained Models".

Task Definition

Given the information of pre-trained models (PTMs), the task is to identify whether it is a risky PTM. We treat the task as binary classification (0/1), where 1 stands for risky PTMs and 0 for benign PTMs.

File Information Description

dataset.zip: This folder contains the dataset, which includes the collected dataset, real-world dataset, and sample dataset.

evaluation.zip: This folder includes the performance evaluation scripts.

models.zip: This folder includes the weight of codebert and codet5, used to train the RPTMHunter.

result.zip: This folder includes the experimental results.

script.zip: This folder includes the scripts of proposed automated risky PTM detection framework (RPTMHunter) and the dataset collection script.

interviews.zip: This folder contains the content of interviews.

readme.md: This is a readme file, including the detailed information of dataset and script steps to conduct the experiment.