有没有相关的论文或文献参考？
这个数据集是基于什么背景创建的？
数据集的作者是谁？
能帮我联系到这个数据集的作者吗？
这个数据集如何下载？
defunct-datasets/bookcorpusopen|自然语言处理数据集|文本生成数据集
收藏hugging_face2023-11-24 更新2024-06-15 收录
下载链接：
https://hf-mirror.com/datasets/defunct-datasets/bookcorpusopen
下载链接
链接失效反馈资源简介：
BookCorpusOpen是一个由书籍组成的语料库，每个条目包含书名和未经处理的书籍文本。该数据集已失效，无法访问，原因是源数据不可用。该数据集最初由Shawn Presser准备，并由非营利平台The-Eye托管，用于数据保存。数据集主要为英文，适用于文本生成和填充掩码等任务。数据集包含17868个项目，每个项目有两个字段：title（书名）和text（文本内容）。书籍文本内容丰富，包含从细粒度信息到高级语义的广泛内容。
提供机构：
defunct-datasets
原始信息汇总
数据集概述
数据集基本信息
- 数据集名称: BookCorpusOpen
- 语言: 英语
- 许可证: 未知
- 多语言性: 单语种
- 大小分类: 10K<n<100K
- 源数据: 原始数据
- 任务类别: 文本生成、填充掩码
- 任务ID: 语言建模、掩码语言建模
- 数据集ID: bookcorpus
数据集结构
数据实例
- 下载数据大小: 2.40 GB
- 生成数据大小: 6.64 GB
- 总磁盘使用量: 9.05 GB
数据字段
- 标题: 字符串类型
- 文本: 字符串类型
数据分割
- 训练集: 17868条数据
数据集创建
数据来源
- 数据准备者: Shawn Presser
- 数据托管: The-Eye
许可证信息
- 数据来源: smashwords.com
- 条款服务: smashwords.com 条款服务
引用信息
@InProceedings{Zhu_2015_ICCV, title = {Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books}, author = {Zhu, Yukun and Kiros, Ryan and Zemel, Rich and Salakhutdinov, Ruslan and Urtasun, Raquel and Torralba, Antonio and Fidler, Sanja}, booktitle = {The IEEE International Conference on Computer Vision (ICCV)}, month = {December}, year = {2015} }
贡献者
- 贡献者: @vblagoje
AI搜集汇总
数据集介绍
构建方式
BookCorpusOpen数据集是由Shawn Presser构建，并托管于The-Eye平台。该数据集包含了17868本图书的标题和未经处理的文本内容。这些图书主要来源于互联网的公开资源，特别是Smashwords网站，旨在为文本生成、语言模型训练等任务提供丰富的文本数据。
特点
BookCorpusOpen数据集的特点在于其包含的文本内容未经处理，保留了书籍原始的语言风格和结构。该数据集为单语种英语数据集，适用于文本生成和填充遮蔽等任务。此外，数据集的构建注重版权开放，但具体版权信息不详。数据集规模适中，便于在多种计算资源上进行操作。
使用方法
使用BookCorpusOpen数据集时，用户可以直接从The-Eye平台下载，然后根据具体任务需求对文本数据进行预处理。数据集以plain_text格式存储，包含标题和文本两个字段，用户可以根据字段信息进行数据解析和利用。需要注意的是，数据集的使用应遵循相关的版权规定，尊重原作者的知识产权。
背景与挑战
背景概述
BookCorpusOpen数据集，由Shawn Presser策划，并得到The-Eye平台的慷慨托管，是一个包含17868本图书的文本数据集。这些图书内容不仅包含细腻的信息，如角色、物体或场景的外观描述，还涵盖高层次语义，如人物的思想、情感及其在故事中的变化。该数据集创建于2015年，旨在为自然语言处理任务如文本生成、填空等提供丰富的文本资源，对于理解细粒度文本内容和高层语义具有显著的研究价值，对相关领域产生了深远的影响。
当前挑战
尽管BookCorpusOpen数据集对研究领域贡献良多，但也面临一些挑战。首先，数据集已经不再可用，源数据不可访问，这限制了其长期的研究价值。其次，构建过程中遇到了数据集版权和许可问题，书籍内容是从smashwords.com抓取的，其使用条款限制了数据的进一步应用和分享。此外，数据集中可能存在的个人和敏感信息、潜在的偏见以及其他未知局限性，都是使用该数据集时需要考虑的重要因素。
常用场景
经典使用场景
在自然语言处理领域，BookCorpusOpen数据集以其丰富的文本内容，成为训练语言模型的重要资源。该数据集常被用于文本生成和填充任务，为模型提供了大量书籍文本，以学习语言的结构和语义。通过这些文本，模型能够掌握叙述的连贯性、角色性格的描述以及场景的渲染。
实际应用
在实际应用中，BookCorpusOpen数据集可用于构建更智能的推荐系统，通过对书籍内容的深入理解，为读者提供个性化的阅读建议。同时，它也能够支持开发更为精准的文本分析工具，用于情感分析、主题分类等，进而提升内容审核和情报挖掘的效率。
衍生相关工作
基于BookCorpusOpen数据集，研究者们衍生出了一系列相关工作，如书籍与电影的对应关系研究、故事性视觉解释的生成等。这些工作不仅推动了跨模态学习的进展，也为电影与文学作品的结合提供了新的视角和思路。
以上内容由AI搜集并总结生成
数据主题
5,000+
优质数据集
54 个
任务类型
进入经典数据集
热门数据集
Billboard-Hot-100
该数据集包含了自1958年以来所有Billboard Hot 100榜单的历史数据，详细记录了每首歌曲的排名、日期、表演者等信息。
github 收录
Canadian Census
**Overview** The data package provides demographics for Canadian population groups according to multiple location categories: Forward Sortation Areas (FSAs), Census Metropolitan Areas (CMAs) and Census Agglomerations (CAs), Federal Electoral Districts (FEDs), Health Regions (HRs) and provinces. **Description** The data are available through the Canadian Census and the National Household Survey (NHS), separated or combined. The main demographic indicators provided for the population groups, stratified not only by location but also for the majority by demographical and socioeconomic characteristics, are population number, females and males, usual residents and private dwellings. The primary use of the data at the Health Region level is for health surveillance and population health research. Federal and provincial departments of health and human resources, social service agencies, and other types of government agencies use the information to monitor, plan, implement and evaluate programs to improve the health of Canadians and the efficiency of health services. Researchers from various fields use the information to conduct research to improve health. Non-profit health organizations and the media use the health region data to raise awareness about health, an issue of concern to all Canadians. The Census population counts for a particular geographic area representing the number of Canadians whose usual place of residence is in that area, regardless of where they happened to be on Census Day. Also included are any Canadians who were staying in that area on Census Day and who had no usual place of residence elsewhere in Canada, as well as those considered to be 'non-permanent residents'. National Household Survey (NHS) provides demographic data for various levels of geography, including provinces and territories, census metropolitan areas/census agglomerations, census divisions, census subdivisions, census tracts, federal electoral districts and health regions. In order to provide a comprehensive overview of an area, this product presents data from both the NHS and the Census. NHS data topics include immigration and ethnocultural diversity; aboriginal peoples; education and labor; mobility and migration; language of work; income and housing. 2011 Census data topics include population and dwelling counts; age and sex; families, households and marital status; structural type of dwelling and collectives; and language. The data are collected for private dwellings occupied by usual residents. A private dwelling is a dwelling in which a person or a group of persons permanently reside. Information for the National Household Survey does not include information for collective dwellings. Collective dwellings are dwellings used for commercial, institutional or communal purposes, such as a hotel, a hospital or a work camp. **Benefits** - Useful for canada public health stakeholders, for public health specialist or specialized public and other interested parties. for health surveillance and population health research. for monitoring, planning, implementation and evaluation of health-related programs. media agencies may use the health regions data to raise awareness about health, an issue of concern to all canadians. giving the addition of longitude and latitude in some of the datasets the data can be useful to transpose the values into geographical representations. the fields descriptions along with the dataset description are useful for the user to quickly understand the data and the dataset. **License Information** The use of John Snow Labs datasets is free for personal and research purposes. For commercial use please subscribe to the [Data Library](https://www.johnsnowlabs.com/marketplace/) on John Snow Labs website. The subscription will allow you to use all John Snow Labs datasets and data packages for commercial purposes. **Included Datasets** - [Canadian Population and Dwelling by FSA 2011](https://www.johnsnowlabs.com/marketplace/canadian-population-and-dwelling-by-fsa-2011) - This Canadian Census dataset covers data on population, total private dwellings and private dwellings occupied by usual residents by forward sortation area (FSA). It is enriched with the percentage of the population or dwellings versus the total amount as well as the geographical area, province, and latitude and longitude. The whole Canada's population is marked as 100, referring to 100% for the percentages. - [Detailed Canadian Population Statistics by CMAs and CAs 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-cmas-and-cas-2011) - This dataset covers the population statistics of Canada by Census Metropolitan Areas (CMAs) and Census Agglomerations (CAs). It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by FED 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-fed-2011) - This dataset covers the population statistics of Canada from 2011 by Federal Electoral District of 2013 Representation Order. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by Health Region 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-health-region-2011) - This dataset covers the population statistics of Canada by health region. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by Province 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-province-2011) - This dataset covers the population statistics of Canada by provinces and territories. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. **Data Engineering Overview** **We deliver high-quality data** - Each dataset goes through 3 levels of quality review - 2 Manual reviews are done by domain experts - Then, an automated set of 60+ validations enforces every datum matches metadata & defined constraints - Data is normalized into one unified type system - All dates, unites, codes, currencies look the same - All null values are normalized to the same value - All dataset and field names are SQL and Hive compliant - Data and Metadata - Data is available in both CSV and Apache Parquet format, optimized for high read performance on distributed Hadoop, Spark & MPP clusters - Metadata is provided in the open Frictionless Data standard, and its every field is normalized & validated - Data Updates - Data updates support replace-on-update: outdated foreign keys are deprecated, not deleted **Our data is curated and enriched by domain experts** Each dataset is manually curated by our team of doctors, pharmacists, public health & medical billing experts: - Field names, descriptions, and normalized values are chosen by people who actually understand their meaning - Healthcare & life science experts add categories, search keywords, descriptions and more to each dataset - Both manual and automated data enrichment supported for clinical codes, providers, drugs, and geo-locations - The data is always kept up to date – even when the source requires manual effort to get updates - Support for data subscribers is provided directly by the domain experts who curated the data sets - Every data source’s license is manually verified to allow for royalty-free commercial use and redistribution. **Need Help?** If you have questions about our products, contact us at [info@johnsnowlabs.com](mailto:info@johnsnowlabs.com).
Databricks 收录
Materials Project 在线材料数据库
Materials Project 是一个由伯克利加州大学和劳伦斯伯克利国家实验室于 2011 年共同发起的大型开放式在线材料数据库。这个项目的目标是利用高通量第一性原理计算，为超过百万种无机材料提供全面的性能数据、结构信息和计算模拟结果，以此加速新材料的发现和创新过程。数据库中的数据不仅包括晶体结构和能量特性，还涵盖了电子结构和热力学性质等详尽信息，为研究人员提供了丰富的材料数据资源。相关论文成果为「Commentary: The Materials Project: A materials genome approach to accelerating materials innovation」。
超神经 收录
VoxBox
VoxBox是一个大规模语音语料库，由多样化的开源数据集构建而成，用于训练文本到语音（TTS）系统。
github 收录
LFW
人脸数据集;LFW数据集共有13233张人脸图像，每张图像均给出对应的人名，共有5749人，且绝大部分人仅有一张图片。每张图片的尺寸为250X250，绝大部分为彩色图像，但也存在少许黑白人脸图片。 URL: http://vis-www.cs.umass.edu/lfw/index.html#download
AI_Studio 收录