多个数据集

github2019-06-10 更新2024-05-31 收录

下载链接：

https://github.com/MVPTylerE/awesome-public-datasets

下载链接

链接失效反馈

官方服务：

资源简介：

这是一个主题中心的高质量开放数据集列表，收集并整理了来自博客、答案和用户响应的数据源。

This is a high-quality open dataset list centered around specific themes, which collects and organizes data sources from blogs, answers, and user responses.

创建时间：

2019-06-10

原始信息汇总

数据集概述

农业

Hyperspectral benchmark dataset on soil moisture: 数据集链接: Hyperspectral benchmark dataset on soil moisture
U.S. Department of Agricultures Nutrient Database: 数据集链接: U.S. Department of Agricultures Nutrient Database
U.S. Department of Agricultures PLANTS Database: 数据集链接: U.S. Department of Agricultures PLANTS Database

生物学

1000 Genomes: 数据集链接: 1000 Genomes
American Gut (Microbiome Project): 数据集链接: American Gut (Microbiome Project)
Broad Bioimage Benchmark Collection (BBBC): 数据集链接: Broad Bioimage Benchmark Collection (BBBC)
Broad Cancer Cell Line Encyclopedia (CCLE): 数据集链接: Broad Cancer Cell Line Encyclopedia (CCLE)
Cell Image Library: 数据集链接: Cell Image Library
Complete Genomics Public Data: 数据集链接: Complete Genomics Public Data
EBI ArrayExpress: 数据集链接: EBI ArrayExpress
EBI Protein Data Bank in Europe: 数据集链接: EBI Protein Data Bank in Europe
ENCODE project: 数据集链接: ENCODE project
Electron Microscopy Pilot Image Archive (EMPIAR): 数据集链接: Electron Microscopy Pilot Image Archive (EMPIAR)
Ensembl Genomes: 数据集链接: Ensembl Genomes
Gene Expression Omnibus (GEO): 数据集链接: Gene Expression Omnibus (GEO)
Gene Ontology (GO) - GO annotation files: 数据集链接: Gene Ontology (GO) - GO annotation files
Global Biotic Interactions (GloBI): 数据集链接: Global Biotic Interactions (GloBI)
Harvard Medical School (HMS) LINCS Project: 数据集链接: Harvard Medical School (HMS) LINCS Project
Human Genome Diversity Project: 数据集链接: Human Genome Diversity Project
Human Microbiome Project (HMP): 数据集链接: Human Microbiome Project (HMP)
ICOS PSP Benchmark: 数据集链接: ICOS PSP Benchmark
International HapMap Project: 数据集链接: International HapMap Project
Journal of Cell Biology DataViewer: 数据集链接: Journal of Cell Biology DataViewer
KEGG - KEGG is a database resource for understanding high-level functions [...]: 数据集链接: KEGG
MIT Cancer Genomics Data: 数据集链接: MIT Cancer Genomics Data
NCBI Proteins: 数据集链接: NCBI Proteins
NCBI Taxonomy: 数据集链接: NCBI Taxonomy
NCI Genomic Data Commons: 数据集链接: NCI Genomic Data Commons
NIH Microarray data: 数据集链接: NIH Microarray data
OpenSNP genotypes data: 数据集链接: OpenSNP genotypes data
Pathguid - Protein-Protein Interactions Catalog: 数据集链接: Pathguid - Protein-Protein Interactions Catalog
Protein Data Bank: 数据集链接: Protein Data Bank
Psychiatric Genomics Consortium: 数据集链接: Psychiatric Genomics Consortium
PubChem Project: 数据集链接: PubChem Project
PubGene (now Coremine Medical): 数据集链接: PubGene (now Coremine Medical)
Sanger Catalogue of Somatic Mutations in Cancer (COSMIC): 数据集链接: Sanger Catalogue of Somatic Mutations in Cancer (COSMIC)
Sanger Genomics of Drug Sensitivity in Cancer Project (GDSC): 数据集链接: Sanger Genomics of Drug Sensitivity in Cancer Project (GDSC)
Sequence Read Archive(SRA): 数据集链接: Sequence Read Archive(SRA)
Stanford Microarray Data: 数据集链接: Stanford Microarray Data
Stowers Institute Original Data Repository: 数据集链接: Stowers Institute Original Data Repository
Systems Science of Biological Dynamics (SSBD) Database: 数据集链接: Systems Science of Biological Dynamics (SSBD) Database
The Cancer Genome Atlas (TCGA), available via Broad GDAC: 数据集链接: The Cancer Genome Atlas (TCGA), available via Broad GDAC
The Catalogue of Life: 数据集链接: The Catalogue of Life
The Personal Genome Project: 数据集链接: The Personal Genome Project
UCSC Public Data: 数据集链接: UCSC Public Data
UniGene: 数据集链接: UniGene
Universal Protein Resource (UnitProt): 数据集链接: Universal Protein Resource (UnitProt)

气候+天气

Actuaries Climate Index: 数据集链接: Actuaries Climate Index
Australian Weather: 数据集链接: Australian Weather
Aviation Weather Center - Consistent, timely and accurate weather [...]: 数据集链接: Aviation Weather Center - Consistent, timely and accurate weather [...]
Brazilian Weather - Historical data (In Portuguese) - Data related to [...]: 数据集链接: Brazilian Weather - Historical data (In Portuguese) - Data related to [...]
Canadian Meteorological Centre: 数据集链接: Canadian Meteorological Centre
Climate Data from UEA (updated monthly): 数据集链接: Climate Data from UEA (updated monthly)
Dutch Weather - The KNMI Data Center (KDC) portal provides access to KNMI [...]: 数据集链接: Dutch Weather - The KNMI Data Center (KDC) portal provides access to KNMI [...]
European Climate Assessment & Dataset: 数据集链接: European Climate Assessment & Dataset
Global Climate Data Since 1929: 数据集链接: Global Climate Data Since 1929
NASA Global Imagery Browse Services: 数据集链接: NASA Global Imagery Browse Services
NOAA Bering Sea Climate: 数据集链接: NOAA Bering Sea Climate
NOAA Climate Datasets: 数据集链接: NOAA Climate Datasets
NOAA Realtime Weather Models: 数据集链接: NOAA Realtime Weather Models
NOAA SURFRAD Meteorology and Radiation Datasets: 数据集链接: NOAA SURFRAD Meteorology and Radiation Datasets
The World Bank Open Data Resources for Climate Change: 数据集链接: The World Bank Open Data Resources for Climate Change
UEA Climatic Research Unit: 数据集链接: UEA Climatic Research Unit
WU Historical Weather Worldwide: 数据集链接: WU Historical Weather Worldwide
WorldClim - Global Climate Data: 数据集链接: WorldClim - Global Climate Data

复杂网络

AMiner Citation Network Dataset: 数据集链接: AMiner Citation Network Dataset
CrossRef DOI URLs: 数据集链接: CrossRef DOI URLs
DBLP Citation dataset: 数据集链接: DBLP Citation dataset
DIMACS Road Networks Collection: 数据集链接: DIMACS Road Networks Collection
NBER Patent Citations: 数据集链接: NBER Patent Citations
NIST complex networks data collection: 数据集链接: NIST complex networks data collection
Network Repository with Interactive Exploratory Analysis Tools: 数据集链接: Network Repository with Interactive Exploratory Analysis Tools
Protein-protein interaction network: 数据集链接: Protein-protein interaction network
PyPI and Maven Dependency Network: 数据集链接: PyPI and Maven Dependency Network
Scopus Citation Database: 数据集链接: Scopus Citation Database
Small Network Data: 数据集链接: Small Network Data
Stanford GraphBase: 数据集链接: Stanford GraphBase
Stanford Large Network Dataset Collection: 数据集链接: Stanford Large Network Dataset Collection
Stanford Longitudinal Network Data Sources: 数据集链接: Stanford Longitudinal Network Data Sources
The Koblenz Network Collection: 数据集链接: The Koblenz Network Collection
The Laboratory for Web Algorithmics (UNIMI): 数据集链接: The Laboratory for Web Algorithmics (UNIMI)
UCI Network Data Repository: 数据集链接: UCI Network Data Repository
UFL sparse matrix collection: 数据集链接: UFL sparse matrix collection
WSU Graph Database: 数据集链接: WSU Graph Database

计算机网络

3.5B Web Pages from CommonCrawl 2012: 数据集链接: 3.5B Web Pages from CommonCrawl 2012
53.5B Web clicks of 100K users in Indiana Univ.: 数据集链接: 53.5B Web clicks of 100K users in Indiana Univ.
CAIDA Internet Datasets: 数据集链接: CAIDA Internet Datasets
CRAWDAD Wireless datasets from Dartmouth Univ.: 数据集链接: CRAWDAD Wireless datasets from Dartmouth Univ.
ClueWeb09 - 1B web pages: 数据集链接: ClueWeb09 - 1B web pages
ClueWeb12 - 733M web pages: 数据集链接: ClueWeb12 - 733M web pages
CommonCrawl Web Data over 7 years: 数据集链接: CommonCrawl Web Data over 7 years
Criteo click-through data: 数据集链接: Criteo click-through data
Internet-Wide Scan Data Repository: 数据集链接: Internet-Wide Scan Data Repository
OONI: Open Observatory of Network Interference - Internet censorship data: 数据集链接: OONI: Open Observatory of Network Interference - Internet censorship data
Open Mobile Data by MobiPerf: 数据集链接: Open Mobile Data by MobiPerf
**The Peer-to-Peer Trace Archive - Real

搜集汇总

数据集介绍

构建方式

该数据集是通过自动化脚本从多个来源收集和整理而来的，涵盖了农业、生物学、气候与天气、复杂网络、计算机网络、数据挑战、地球科学、经济学、教育、能源、金融和GIS等多个领域的数据集。

特点

数据集特点包括多样性、高质量、实时更新以及覆盖领域广泛。它不仅包含了大量免费的公共数据集，还包含了一些非免费的数据集，用户可以根据需要进行选择和使用。

使用方法

使用该数据集时，用户可以通过GitHub页面上的链接访问各个数据集的详情页，了解数据集的具体信息，并根据提供的说明和许可进行数据下载和使用。

背景与挑战

背景概述

多个数据集是一个汇总了众多领域公共数据集的资源库，由awesomedata团队创建于GitHub平台。该数据集涵盖了从农业、生物学到气候、计算机科学等众多学科领域，包含了如1000 Genomes、Hyperspectral benchmark dataset on soil moisture等高质量的数据集。这些数据集的收集与整理，旨在为研究者和开发者提供便捷的数据获取途径，推动各领域的数据共享与科学研究。

当前挑战

该数据集面临的挑战主要在于其多样性和复杂性。首先是数据集的多样性，涉及多个学科领域，每个领域的数据都有其特定的格式和特点，给数据集成和处理带来了困难。其次是数据集的复杂性，例如，在构建过程中需要处理的数据量庞大，数据清洗和整合的工作量巨大。此外，数据集的时效性和准确性也是持续的挑战，需要不断更新和维护。

常用场景

经典使用场景

该数据集涵盖了多个领域的大量公共数据集，其经典使用场景主要集中于为研究人员提供丰富的数据资源，以便于进行数据分析和挖掘。例如，在生物学领域，可用于基因表达分析、蛋白质结构研究等；在气候与天气领域，可用于气候模型构建、天气预测等；在复杂网络领域，可用于社会网络分析、知识图谱构建等。

衍生相关工作

该数据集衍生了众多相关工作，如基于数据集的学术研究论文、商业分析报告和开源项目。这些相关工作进一步推动了数据集的应用和领域研究的发展。

数据集最近研究