Recursiveness in Multimodal Generative Artificial Intelligence

The dataset contains images extracted from the COCO dataset that have been tested in the Recursive Modality Changes process: a caption is extracted from an image and used to generate a new image. The process is repeated in a loop. For the extraction of the description was used GPT-4o and for the generation of the images DALL-E3.

Description

The dataset contains experiments of applying the RMC of length 40 generations from images that contains elements of the following categories: apples, elephants, fire-hydrants, persons, toilets, and trains. In total, there are 10 RMC loops per category (40*10*6 = 2,400 images) and the comparison between the images and descriptions using the metrics LPIPS VGG, TF-IDF, BERT tokenizer, BLIP.

1_coco_dataset: information from COCO images of each category

2_categories: loops of each category

results_{category}

experiments

results_dall-e-3_hd_{style}_{category} -> hd (high definition), style (vivid or natural) -> all the experiments of that {style} and {category} 

results_all.xlsx -> similarity with metrics LPIPS, TF-IDF, BLIP, BERT aggregated

experiments

{date}_{style}_{coco_id} -> each experiment from a coco image

imgs -> images

imgs_resized -> resized images

experiment.json -> json with data of the experiment (description, number of generations, etc.)

results.xlsx -> metrics of each individual loop

images_all.xlsx -> summary of all the images generated and their descriptions

inter-experiments_results.xlsx -> aggregated results of inter-trajectory experiments of this {category}

intra-experiments_results.xlsx -> aggregated results of intra-trajectory experiments of this {category}

3_combined results: combined results of all categories

rsults_hd_labels_{style} -> all the results from images generated with the same style (vivid or natural)

images_all.xlsx -> summary of all images

inter-exeriments_results_all.xlsx

intra-experiments_results_all.xlsx

results_all_labels.xlsx -> summary of results per {category}

4_different_styles -> experiments comparing styles (natural and vivid)

Paper

Paper: 

Cite:

@misc{}