Replication Package of "Battling Phish"

This replication package contains all datasets and source code used in our study on phishing URL detection. The study investigates the effectiveness of traditional machine learning (ML), deep learning (DL), and large language model (LLM)-based methods using a consistent set of 32 URL-based features.<br><br>Directory Structure├── Datasets/│ ├── Dataset-1.csv│ ├── Dataset-2.csv│ ├── Dataset-3.csv│ ├── Dataset-4.csv│ ├── Dataset-5.csv│ ├── Phishing_Site_URLs_32_Features_Extracted_Data.csv│ └── Legit_Phish_32_Features_Extracted_Data.csv│└── Source_Codes/ ├── Feature_extraction_source_code.py ├── Feature_importance_analysis_source_code.py ├── ML/ │ ├── Seven_ML_Models_trained_on_LP.py │ ├── Seven_ML_Models_trained_on_PSU.py │ ├── SoftVoting_trained_on_LP.py │ ├── SoftVoting_trained_on_PSU.py │ ├── HardVoting_trained_on_LP.py │ └── HardVoting_trained_on_PSU.py │ ├── DL/ │ ├── [DLModel1]_trained_on_LP.py │ ├── [DLModel1]_trained_on_PSU.py │ └── ... (total 16 files for 8 DL algorithms) │ └── LLM/ ├── BERT_Fine_Tuned_on_LP.py ├── BERT_Fine_Tuned_on_PSU.py ├── DistilBERT_Fine_Tuned_on_LP.py ├── DistilBERT_Fine_Tuned_on_PSU.py ├── PhishBERT_Evaluation.py └── URLBERT_Evaluation.pyDatasets:<code>Dataset-1.csv</code> to <code>Dataset-5.csv</code>:<br>Used for feature importance analysis.<code>Phishing_Site_URLs_32_Features_Extracted_Data.csv</code> (PSU dataset):<br>Includes phishing and legitimate URLs with 32 extracted lexical features.<code>Legit_Phish_32_Features_Extracted_Data.csv</code> (LP dataset):<br>Another benchmark dataset with the same 32 features, used for comparative evaluation.<b>Note</b>: PSU and LP datasets are used for both training and evaluating ML, DL, and LLM-based models.Source Code:<code><strong>Feature_extraction_source_code.py</strong></code><br>Extracts 32 handcrafted lexical features from raw URL data.<code><strong>Feature_importance_analysis_source_code.py</strong></code><br>Performs feature selection using seven statistical and model-based ranking methods.Machine Learning (ML)Implements ML classifiers individually trained on LP and PSU datasets:Logistic Regression, Decision Tree, Random Forest, Extra Trees, Gradient Boosting, AdaBoost, and XGBoost.Soft Voting and Hard Voting ensembles are also implemented.Scripts:<code>Seven_ML_Models_trained_on_LP.py</code><code>Seven_ML_Models_trained_on_PSU.py</code><code>SoftVoting_trained_on_LP.py</code>, <code>SoftVoting_trained_on_PSU.py</code><code>HardVoting_trained_on_LP.py</code>, <code>HardVoting_trained_on_PSU.py</code>Deep Learning (DL)Implements eight deep learning architectures (each trained separately on LP and PSU):Total of 16 scripts — 2 per DL model (1 for LP, 1 for PSU).Large Language Models (LLMs)<b>Fine-tuned</b>:<code>BERT_Fine_Tuned_on_LP.py</code>, <code>BERT_Fine_Tuned_on_PSU.py</code><code>DistilBERT_Fine_Tuned_on_LP.py</code>, <code>DistilBERT_Fine_Tuned_on_PSU.py</code><b>Pre-trained, zero-shot or direct evaluation</b>:<code>PhishBERT_Evaluation.py</code><code>URLBERT_Evaluation.py</code>