Ways to teach a little-brain language model to recommend new and attractive domain names to register

A sample dataset is provided in archive dataset.zip. Full dataset cannot be released due to policy limitations. Archive password will be provided after publication of the associated research paper. Archive contents:

domain_list_pl.txt - a list of crawled domains.

pl/001/*.json - sample webpage excerpts. Each file contains a JSON table with the following fields: domain name, webpage title, declared language, metadata description and metadata keyword.

freq_dict_pl.json - frequency dictionary for Polish words.

contrastive.arrow - human-evaluated domain recommendations in form of contrastive pairs, a table with columns named: `prompt', 'chosen' and `rejected', of a respective meaning.

File decompose_dnames0.py contains code that sanitizes webpage excerpts and splits a domain name into component words (further used as the desired model output).

The base BART model was trained in SFT phase with the standard transformers/examples/pytorch/summarization/run_summarization.py script provided in GitHub's HuggingFace repository.

DPO training was carried out using standard scripts provided in TRL (huggingface.co/docs/trl) library, with the following settings:

Learning rate: 1e-5

Number of epochs: 3

Warmup steps: 30

Batch size: 32

Loss type: sigmoid

Beta: 0.1