awesome-public-datasets

github2016-12-18 更新2024-05-31 收录

下载链接：

https://github.com/vdt/awesome-public-datasets

下载链接

链接失效反馈

官方服务：

资源简介：

一个包含高质量公开数据集的列表，涵盖多个领域，如农业、生物学、气候/天气等。

A compilation of high-quality open datasets spanning various domains, including agriculture, biology, climate/weather, and more.

创建时间：

2016-01-31

原始信息汇总

数据集概述

本数据集是一个综合性的公共数据集列表，涵盖了多个领域的数据资源。以下是根据提供的内容整理的关键信息：

数据集来源

数据集由用户响应、博客和问答社区中的信息整理而来。
大多数数据集是免费的，但也有部分数据集需要付费。

数据集分类

数据集被分为多个类别，包括但不限于：

农业
- U.S. Department of Agricultures PLANTS Database
生物学
- 1000 Genomes
- American Gut (Microbiome Project)
- Cell Image Library
- Collaborative Research in Computational Neuroscience (CRCNS)
- EBI ArrayExpress
- EBI Protein Data Bank in Europe
- ENCODE project
- Ensembl Genomes
- Gene Expression Omnibus (GEO)
- Gene Ontology (GO)
- Global Biotic Interactions (GloBI)
- Harvard Medical School (HMS) LINCS Project
- Human Microbiome Project (HMP)
- ICOS PSP Benchmark
- Journal of Cell Biology DataViewer
- MIT Cancer Genomics Data
- NIH Microarray data
- OpenSNP genotypes data
- Pathguid - Protein-Protein Interactions Catalog
- Protein Data Bank
- PubChem Project
- PubGene (now Coremine Medical)
- Sequence Read Archive(SRA)
- Stanford Microarray Data
- Stowers Institute Original Data Repository
- Systems Science of Biological Dynamics (SSBD) Database
- The Catalogue of Life
- The Personal Genome Project
- UCSC Public Data
- UniGene
气候/天气
- Australian Weather
- Brazilian Weather - Historical data (In Portuguese)
- Canadian Meteorological Centre
- Climate Data from UEA (updated monthly)
- European Climate Assessment & Dataset
- Global Climate Data Since 1929
- NASA Global Imagery Browse Services
- NOAA Bering Sea Climate
- NOAA Climate Datasets
- NOAA Realtime Weather Models
- The World Bank Open Data Resources for Climate Change
- UEA Climatic Research Unit
- WorldClim - Global Climate Data
- WU Historical Weather Worldwide
复杂网络
- CrossRef DOI URLs
- DBLP Citation dataset
- NBER Patent Citations
- NIST complex networks data collection
- Protein-protein interaction network
- PyPI and Maven Dependency Network
- Scopus Citation Database
- Small Network Data
- Stanford GraphBase (Steven Skiena)
- Stanford Large Network Dataset Collection
- The Koblenz Network Collection
- The Laboratory for Web Algorithmics (UNIMI)
- The Nexus Network Repository
- UCI Network Data Repository
- UFL sparse matrix collection
- WSU Graph Database
- Stanford Longitudinal Network Data Sources
计算机网络
- 3.5B Web Pages from CommonCraw 2012
- 53.5B Web clicks of 100K users in Indiana Univ.
- CAIDA Internet Datasets
- ClueWeb09 - 1B web pages
- ClueWeb12 - 733M web pages
- CommonCrawl Web Data over 7 years
- CRAWDAD Wireless datasets from Dartmouth Univ.
- Criteo click-through data
- Open Mobile Data by MobiPerf
- UCSD Network Telescope, IPv4 /8 net
上下文数据
- Context-aware data sets from five domains
数据挑战
- Challenges in Machine Learning
- CrowdANALYTIX dataX
- D4D Challenge of Orange
- DrivenData Competitions for Social Good
- ICWSM Data Challenge (since 2009)
- Kaggle Competition Data
- KDD Cup by Tencent 2012
- Localytics Data Visualization Challenge
- Netflix Prize
- Space Apps Challenge
- Telecom Italia Big Data Challenge
- Yelp Dataset Challenge
经济学
- American Economic Ass (AEA)
- EconData from UMD
- Economic Freedom of the World Data
- Historical MacroEconomic Statistics
- International Trade Statistics
- Internet Product Code Database
- Joint External Debt Data Hub
- Jon Haveman International Trade Data Links
- OpenCorporates Database of Companies in the World
- Our World in Data
- SciencesPo World Trade Gravity Datasets
- The Atlas of Economic Complexity
- The Center for International Data
- The Observatory of Economic Complexity
- UN Commodity Trade Statistics
- UN Human Development Reports
能源
- AMPds
- BLUEd
- COMBED
- Dataport
- ECO
- EIA
- HFED
- iAWE
- Plaid
- REDD
- UK-Dale
金融
- CBOE Futures Exchange
- Google Finance
- Google Trends
- NASDAQ
- OANDA
- OSU Financial data
- Quandl
- St Louis Federal
- Yahoo Finance
地质学
- Earth Models
- Smithsonian Institution Global Volcano and Eruption Database
- USGS Earthquake Archives
地理空间/GIS
- BODC - marine data of ~22K vars
- Cambridge, MA, US, GIS data on GitHub
- EOSDIS - NASAs earth observing system data
- Factual Global Location Data
- Geo Spatial Data from ASU
- Geo Wiki Project - Citizen-driven Environmental Monitoring
- GeoNames Worldwide
- Global Administrative Areas Database (GADM)
- International Institute for Systems Analysis - GIS Datasets
- Landsat 8 on AWS
- List of all countries in all languages
- Natural Earth - vectors and rasters of the world
- OpenAddresses
- OpenStreetMap (OSM)
- Reverse Geocoder using OSM data
- TIGER/Line - U.S. boundaries and roads
- TwoFishes - Foursquares coarse geocoder
- TZ Timezones shapfiles
- UN Environmental Data
- World countries in multiple formats
政府
- Alberta, Province of Canada
- Antwerp, Belgium
- Argentina (non official)
- Argentina
- Austin, TX, US
- Australia (abs.gov.au)
- Australia (data.gov.au)
- Austria (data.gv.at)
- Baton Rouge, LA, US
- Belgium
- Brazil
- Buenos Aires, Argentina
- Calgary, AB, Canada
- Cambridge, MA, US
- Canada
- Chicago
- Dallas Open Data
- DataBC - data from the Province of British Columbia
- Denver Open Data
- Durham, NC Open Data
- Edmonton, AB, Canada
- England LGInform
- EuroStat
- FedStats
- Finland
- France
- Fredericton, NB, Canada
- Gatineau, QC, Canada
- Germany
- Ghent, Belgium
- Glasgow, Scotland, UK
- Guardian world governments
- Halifax, NS, Canada
- Helsinki Region, Finland
- Houston Open Data
- Indian Government Data
- Indonesian Data Portal
- Laval, QC, Canada
- London Datastore, UK
- London, ON, Canada
- Los Angeles Open Data
- MassGIS, Massachusetts, U.S.
- Mexico
- Missisauga, ON, Canada
- Moncton, NB, Canada
- Montreal, QC, Canada
- Netherlands
- New Zealand
- NYC betanyc
- NYC Open Data
- OECD
- Oklahoma
- Open Government Data (OGD) Platform India
- Oregon
- Ottawa, ON, Canada
- Portland, Oregon
- Puerto Rico Government
- Quebec City, QC, Canada
- Quebec Province of Canada
- Regina SK, Canada
- Rio de Janeiro, Brazil
- Romania
- Russia
- San Francisco Data sets
- Saskatchewan, Province of Canada
- Seattle
- Singapore Government Data
- South Africa
- South Africa Trade Statistics
- State of Utah, US
- Switzerland
- Texas Open Data
- The World Bank
- Toronto, ON, Canada
- U.K. Government Data
- U.S. American Community Survey
- U.S. CDC Public Health datasets
- U.S. Census Bureau
- U.S. Department of Housing and Urban Development (HUD)
- U.S. Federal Government Agencies
- U.S. Federal Government Data Catalog
- U.S. Food and Drug Administration (FDA)
- U.S. National Center for Education Statistics (NCES)
- U.S. Open Government
- UK 2011 Census Open Atlas Project
- United Nations
- Uruguay
- Vancouver, BC Open Data Catalog
- Victoria, BC, Canada
医疗保健
- EHDP Large Health Data Sets
- Gapminder World demographic databases
- Medicare Coverage Database (MCD), U.S.
- Medicare Data Engine of medicare.gov Data
- Medicare Data File
- MeSH, the vocabulary thesaurus used for indexing articles for PubMed
- Number of Ebola Cases and Deaths in Affected Countries (2014)
- Open-ODS (structure of the UK NHS)
- The Cancer Genome Atlas project (TCGA)
- World Health Organization Global Health Observatory
图像处理
- 10k US Adult Faces Database
- 2GB of Photos of Cats
- Affective Image Classification
- Animals with attributes
- Face Recognition Benchmark
- ImageNet (in WordNet hierarchy)
- Indoor Scene Recognition
- International Affective Picture System, UFL
- Massive Visual Memory Stimuli, MIT
- Several Shape-from-Silhouette Datasets
- Stanford Dogs Dataset
- SUN database, MIT
- The Oxford-IIIT Pet Dataset

这些数据集为研究人员、开发者和数据分析师提供了丰富的资源，以支持各种研究和应用开发。

搜集汇总

数据集介绍

构建方式

该数据集通过从博客、回答和用户响应中收集和整理公共数据源而构建。大部分列出的数据集是免费的，但也有一些不是。其他令人惊叹的数据集列表可以在awesome-awesomeness和sindresorhus's awesome列表中找到。

使用方法

使用该数据集的方法包括：首先，访问GitHub页面上的数据集列表；其次，根据需要选择合适的数据集；最后，按照数据集的说明进行下载和使用。对于一些大型数据集，可能需要特定的工具或平台来处理和分析数据。

背景与挑战

背景概述

‘awesome-public-datasets’是一个收集和整理自博客、回答和用户响应的公开数据集列表。该数据集主要由sindresorhus维护，大部分列出的数据集是免费的，但也有一些不是。此列表的目的是为了方便研究人员和开发者快速找到并访问高质量的公共数据集，以推动科学研究和创新。

当前挑战

数据集的构建和整理面临的主要挑战包括：1) 如何确保数据集的质量和可靠性；2) 如何处理和整合不同来源和格式的数据；3) 如何保证数据集的更新和可持续发展；4) 如何在数据隐私和保护的前提下，提供开放访问。

常用场景

经典使用场景

awesome-public-datasets 数据集经典使用场景描述：该数据集主要被用于收集和整理各种公共数据源，涵盖了农业、生物学、气候、复杂网络、计算机网络、上下文数据、数据挑战、经济学、能源、金融、地质学、地理信息系统、政府、医疗保健、图像处理等多个领域。其经典使用场景包括为研究者提供一站式资源，以便于他们能够轻松地找到和访问所需的数据集。

解决学术问题

数据集解决学术问题描述：awesome-public-datasets 数据集解决了研究者寻找高质量公共数据集的难题，避免了数据源分散和难以访问的问题。它通过提供一个综合性的列表，使得研究者能够高效地定位到相关数据集，进而推动学术研究的进展。

实际应用

数据集实际应用情况描述：在实际应用中，该数据集被广泛用于数据科学项目、学术研究、决策支持系统等领域。它帮助研究者快速获取到所需的数据，提高了研究效率和数据质量，对于推动科技创新和社会发展具有重要意义。

数据集最近研究