Mind captioning: Evolving descriptive text of mental content from human brain activity

This repository contains preprcessed fMRI data and semantic features used in:
Horikawa, T. Mind captioning: Evolving descriptive text of mental content from human brain activity. bioRxiv (2024). https://doi.org/10.1101/2024.04.23.590673
and
Horikawa, T. Mind captioning: Evolving descriptive text of mental content from human brain activity. Sci. Adv. 11, eadw1464 (2025). https://doi.org/10.1126/sciadv.adw1464

Preprocessed fMRI data of training and test data for all subjects (n = 6)'trainPerception_S1-6.mat''testPerception_S1-6.mat''testImagery_S1-6.mat'Feature data'feature.zip': Original features computed by deberta-large and timesformer computed for all videos.The directory also contains normalization parameters (mu, std).Decoded feature'decfeat_wb.zip': Brain-decoded features obtained from the whole brain activityEncoding analysis results'res_encoding.zip': Results to reproduce main results of encoding analysisText generation analysis results'res_textgen.zip': Results to reproduce main results of text generation analysisSupplementary information"Supplementary_Video1-3.mov": Supplementary videos showing examples of evolved descriptions of viewed and recalled content

Additional Information:
Raw fMRI data is available at OpenNeuro: https://doi.org/10.18112/openneuro.ds005191.v1.0.2The code that support the findings of the study is available from our GitHub repository: https://github.com/horikawa-t/MindCaptioning.