kknana11/LightNovel5000

该数据集名为脚盆轻小说汉化 - 从未禁止爬虫的公共网站爬取，是一个包含约1500部轻小说的集合，数据来源于未禁止爬虫的公共网站。数据集提供带插图的PDF文件和txt文本文件，可用于训练风格化的大语言模型（LLM）。作者已对文本进行部分清理，但不保证完全清理。文件结构包括加密的PDF和txt文件、合并文本、索引文件等，方便用户访问和处理。数据集规模在100亿到1000亿之间，支持中文和英文，专注于文本生成和翻译任务。This dataset, titled Light novels translated in Chinese - crawled from public websites that do not prohibit crawlers, contains approximately 1500 light novels sourced from public websites that do not prohibit crawlers. It includes PDF files with illustrations and txt text files, which can serve as a good data source for training stylish large language models (LLMs). The author has partially cleaned the text but does not guarantee it is fully cleaned. The file structure comprises encrypted PDF and txt files, a concatenated text file, index files, and more, facilitating access and processing. The dataset size ranges between 10 billion and 100 billion, supports Chinese and English languages, and focuses on text-generation and translation tasks.