momentslab/AstroCaptions

---
task_categories:
- image-to-text
language:
- en
pretty_name: AstroCaptions
size_categories:
- 10K<n<100K
license: cc-by-nc-3.0
---

AstroCaptions is an image captioning dataset made of both human labelled and synthetic captions. AstroCaptions is made of **44115** publicly available NASA archive images.
It contains both very recent photos and old archive pictures from the first Apollo missions. Many astronauts, NASA scientists and executives appear on these images.

Each image comes with a description, scraped from public NASA website. These provides both visual description of the image and contextual
information. The first sentence often contains the factual description of the image. Given the unstructured nature of the data, the final dataset still contains some dates and event names that could not be inferred by a captioning model.
That is why, with a few-shot approach, we also used OpenAI GPT-4 model to create new captions from the long descriptions.

The names of the people appearing in the dataset are also extracted from the captions using a named entity recognition BERT model.
A total of **13083** identifiable persons appears on this dataset.

### Dataset Fields
```
image_source_url: the URL to the source image
original_nasa_caption: the original scraped caption found in the image metadata
first_sentence_caption: the first sentence from the original_nasa_caption (NLTK is used for the splitting in sentences)
gpt4_caption: the synthetic caption generated by GPT4
identified_persons: the names of the identified persons in the associated image. Names are seperated by ','
```

If you use this dataset, please cite:

```
@misc{tevissen2024inserting,
title={Inserting Faces inside Captions: Image Captioning with Attention Guided Merging},
author={Yannis Tevissen and Khalil Guetari and Marine Tassel and Erwan Kerleroux and Frédéric Petitpont},
year={2024},
eprint={2405.02305},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```

原始信息汇总

数据集概述

名称: AstroCaptions

任务类别: 图像到文本（image-to-text）

语言: 英语（en）

大小: 10K<n<100K

许可: cc-by-nc-3.0

描述: AstroCaptions是一个包含44115张NASA档案图像的图像标注数据集，这些图像包括最新的照片和最早的阿波罗任务档案图片。数据集中的每张图像都附有从NASA公共网站上抓取的描述，提供图像的视觉描述和背景信息。此外，数据集还使用OpenAI GPT-4模型生成了新的标注。

数据集字段:

• image_source_url: 源图像的URL
• original_nasa_caption: 图像元数据中找到的原始抓取标注
• first_sentence_caption: 原始标注中的第一句话
• gpt4_caption: GPT-4生成的合成标注
• identified_persons: 图像中识别出的人名，以逗号分隔

人物识别: 数据集中共识别出13083个可识别的人物，使用BERT模型进行命名实体识别。