A Dataset of Information (DNS, IP, WHOIS/RDAP, TLS, GeoIP) for a Large Corpus of Benign, Phishing, and Malware Domain Names 2024

The dataset contains DNS records, IP-related features, WHOIS/RDAP information, information from TLS handshakes and certificates, and GeoIP information for 368,956 benign domains from Cisco Umbrella, 461,338 benign domains from the actual CESNET network traffic, 164,425 phishing domains from PhishTank and OpenPhish services, and 100,809 malware domains from various sources like ThreatFox, The Firebog, MISP threat intelligence platform, and other sources. The ground truth for the phishing dataset was double-check with the VirusTotal (VT) service. Domain names not considered malicious by VT have been removed from phishing and malware datasets. Similarly, benign domain names that were considered risky by VT have been removed from the benign datasets. The data was collected between March 2023 and July 2024. The final assessment of the data was conducted in August 2024. The dataset is useful for cybersecurity research, e.g. statistical analysis of domain data or feature extraction for training machine learning-based classifiers, e.g. for phishing and malware website detection. Data Files The data is located in the following individual files: benign_umbrella.json - data for 368,956 benign domains from Cisco Umbrella, benign_cesnet.json - data for 461,338 benign domains from the CESNET network, phishing.json - data for 164,425 phishing domains, and malware.json - data for 100,809 malware domains. Data Structure Both files contain a JSON array of records generated using mongoexport. The following table documents the structure of a record. Please note that: some fields may be missing (they should be interpreted as nulls), extra fields may be present (they should be ignored). Field name Field type Nullable Description domain_name String No The evaluated domain name url String No The source URL for the domain name evaluated_on Date No Date of last collection attempt source String No An identifier of the source sourced_on Date No Date of ingestion of the domain name dns Object Yes Data from DNS scan rdap Object Yes Data from RDAP or WHOIS tls Object Yes Data from TLS handshake ip_data Array of Objects Yes Array of data objects capturing the IP addresses related to the domain name DNS data (dns field) A Array of Strings No Array of IPv4 addresses AAAA Array of Strings No Array of IPv6 addresses TXT Array of Strings No Array of raw TXT values CNAME Object No The CNAME target and related IPs MX Array of Objects No Array of objects with the MX target hostname, priority and related IPs NS Array of Objects No Array of objects with the NS target hostname and related IPs SOA Object No All the SOA fields, present if found at the target domain name zone_SOA Object No The SOA fields of the target’s zone (closest point of delegation), present if found and not a record in the target domain directly dnssec Object No Flags describing the DNSSEC validation result for each record type ttls Object No The TTL values for each record type remarks Object No The zone domain name and DNSSEC flags RDAP data (rdap field) copyright_notice String No RDAP/WHOIS data usage copyright notice dnssec Bool No DNSSEC presence flag entitites Object No An object with various arrays representing the found related entity types (e.g. abuse, admin, registrant). The arrays contain objects describing the individual entities. expiration_date Date Yes The current date of expiration handle String No RDAP handle last_changed_date Date Yes The date when the domain was last changed name String No The target domain name for which the data in this object are stored nameservers Array of Strings No Nameserver hostnames provided by RDAP or WHOIS registration_date Date Yes First registration date status Array of Strings No The state of the registered object (see RFC 7483, section 10.2.2) terms_of_service_url String No URL of the RDAP usage ToS url String No URL of the RDAP entity whois_server String No WHOIS server address TLS data (tls field) cipher String No TLS cipher suite description according to IANA protocol String No One of “TLS”, ”TLSv1.2”, ”TLSv1.3” certificates Array of Objects No Array of objects representing the certificate chain, the first element is the root certificate IP data (elements in the ip_data array) ip String No The IP address from_record String No The type of the DNS record the address was captured from remarks Object No Ping round-trip time, “is alive” flag and rdap/geo/asn evaluation dates rdap Object Yes RDAP data, similar to DNS RDAP, see the JSON Schema for details geo Object Yes Geolocation data from the GeoLite2 City database (e.g. latitude, longitude, city, country, etc.) asn Object Yes Autonomous system data from the GeoLite2 ASN database (ASN, organization, network) Acknowledgements We would like to thank the OpenPhish Team for grating permission to use and publish their dataset. We also thank VirusTotal for providing us access to the API for research purposes. This dataset includes GeoLite2 data created by MaxMind, available from https://www.maxmind.com. The research has been supported by the Flow-based Encrypted Traffic Analysis project, no. VJ02010024, granted by the Ministry of the Interior of the Czech Republic and the Smart Information Technology for a Resilient Society project, no. FIT-S-23-8209, granted by Brno University of Technology.