Curlie Domain-Level Multi-Label Content Classification Benchmark

A multi-label content-classification benchmark for Internet apex domain names, derived from the Curlie web directory (https://curlie.org/, successor to the Open Directory Project / DMOZ), via the processed release of Lugeon et al. (2022, https://doi.org/10.6084/m9.figshare.19406693.v5). Each example pairs an apex domain string with a 14-dimensional binary indicator vector over Curlie's top-level content categories. Contents (JSON Lines) File Examples train.jsonl 566,826 val.jsonl 161,950 test.jsonl 80,976 metadata.json full schema, statistics, preprocessing parameters Each line: {"text": "0-360.com", "class_vector": [0,1,0,0,0,0,0,0,0,0,0,1,0,0]} Label space (14 top-level Curlie categories) Arts, Business, Computers, Games, Health, Home, Kids_and_Teens, News, Recreation, Reference, Science, Shopping, Society, Sports. Preprocessing pipeline Start from the 2,275,150 source rows of the homepage2vec Curlie release; 1,986,496 unique URIs; 885,582 with class vectors. Drop entries without a class vector (1,100,914), IP addresses (6), PSL-private suffixes (11,055). Collapse subdomains to apex domains using the PSL (60,597 collapses). Aggregate labels per apex domain by union; drop 4,172 domains whose subdomain-level labels are mutually contradictory (13,470 conflicting records). Apply Unicode NFC normalization and IDNA 2008 validation. Random 70 / 20 / 10 train/val/test split, seed 42. Statistics 820,807 unique apex domains → 809,752 final examples. 15,200 domains (1.88 %) carry two or more labels. Class imbalance ≈ 29× (Business 246,654 vs. Kids_and_Teens 8,496). 490 unique TLDs observed; per-TLD content profiles available in the companion paper (Appendix). Intended use Training and evaluation of multi-label content classification on domain names; tokenizer/architecture probes (see companion paper). Limitations Curlie is a curated, human-edited directory; coverage is biased toward older, manually submitted sites. The taxonomy was designed for general web content and may not be optimal for short domain-name strings (we discuss this in the paper's Future Work). Attribution Built on top of the Curlie web directory (CC BY-SA 3.0) and the homepage2vec processed release (Lugeon, Piccardi & West, ICWSM 2022). Please cite both, in addition to this dataset.