Visual image reconstruction

# Visual image reconstruction

Original paper: Miyawaki Y, Uchida H, Yamashita O, Sato M, Morito Y, Tanabe HC, Sadato N & Kamitani Y (2008) Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders. Neuron 60:915-929.

## Overview

This is the fMRI data from Miyawaki et al. (2008) "Visual image reconstruction from human brain activity using a combination of multiscale local image decoders". Neuron 60:915-29. In this study, we collected fMRI activity from subjects viewing images, and constructed decoders predicting local image contrast at multiple spatial scales. The combined decoders based on a linear model successfully reconstructed presented stimuli from fMRI activity.

## Task

The experiment consisted of human subjects viewing contrast-based images of 12 x 12 flickering patches. There were two types of image viewing tasks: (1) random image viewing and (2) figure image (geometric shape or alphabet letter) viewing. For image presentation, a block design was used with rest periods between the presentation of each image. For random image patch presentation, images were presented for 6 s, followed by 6 s rest. For figure image presentation, images were presented for 12 s, followed by 12 s rest. The data from random image viewing runs were used to train the decoding models, and the trained model were evaluated with the data from figure image viewing runs.

## Dataset

This dataset contains two subjects ('sub-01' and 'sub-02'). The subjects performed two sessions of fMRI experiments ('ses-01' and 'ses-02'). Each session is composed of several EPI runs (TR, 2000 ms; TE, 30 ms; flip angle, 80°; voxel size, 3 × 3 × 3 mm; FOV, 192 × 192 mm; number of slices, 30, slice gap, 0 mm) and inplane T2-weighted imaging (TR, 6000 ms; TE, 57 ms; flip angle, 90°; voxel size, 0.75 × 0.75 × 3.0 mm; FOV, 192 × 192 mm). The EPI images covered the entire occipital lobe. The dataset also includes a T1-weighted anatomical reference image for each subject (TR, 2250 ms; TE, 2.98 ms for sub-01 and 3.06 ms for sub-02; TI, 900 ms; flip angle, 9°; voxel size, 1.0 × 1.0 × 1.0 mm; FOV, 256 × 256 mm). The T1w images were obtained in sessions different from the fMRI experiment sessions and stored in 'ses-anat' directories. The T1w images were defaced by pydeface (<https://pypi.python.org/pypi/pydeface>). All DICOM files are converted to Nifti-1 files by mri_convert in FreeSurfer. In addition, the dataset contains mask images of manually defined ROIs for each subjects in `sourcedata` directory (See `README` in `sourcedata` for more details).

During fMRI runs, the subject viewed contrast-based images of 12 × 12 flickering image patches. Two types of runs ('viewRandom' and 'viewFigure') were included in the experiment. In 'viewRandom' runs, random images were presented as visual stimuli. Each 'viewRandom' runs consisted of 22 stimulus presentation trials and lasted for 298 s (149 volumes). The two subjects performed 20 'viewRandom' runs. In 'viewFigure' runs, either geometric shape pattern (square, small frame, large frame, plus, X) or alphabet letter pattern (n, e, u, r, o) was presented in each trial. In addition, data while the subject viewed thin and large alphabet letter patterns (n, e, u, r, o) are included in the dataset (they are not included in the results of the original study). Each 'viewFigure' run consisted of 10 stimulus presentation trials and lasted for 268 s (134 volumes). The 'sub-01' and 'sub-02' performed 12 and 10 'viewFigure' runs, respectively.

To help subjects suppress eye blinks and firmly fixate the eyes, the color of the fixation spot changed from white to red 2 s before each stimulus block started. To ensure alertness, subjects were instructed to detect the color change of the fixation (red to green, 100 ms) that occurred after a random interval of 3–5 s from the beginning of each stimulus block. Performances of the subject was monitored online during experiments, but were not recorded and omitted from the dataset.

### Task event files

The value of `trial_type` in the task event files (`*_events.tsv`) indicates the type of each trial (block) as below.

- `rest`: Rest trial (no visual stimulus).
- `stimulus_random`: Random pattern.
- `stimulus_shape`: Geometric shape pattern (square, small frame, large frame, plus, X).
- `stimulus_alphabet`: Alphabet pattern (n, e, u, r, o).
- `stimulus_alphabet_thin`: Thin alphabet pattern (n, e, u, r, o).
- `stimulus_alphabet_long`: Long alphabet pattern (n, e, u, r, o).

Note that the results from thin and long alphabet patterns are not included in the original paper although the data were obtained in the same sessions.

Additional column `stimulus_pattern` contains the pattern of stimuli (12 × 12) presented in each stimulus trial. It is vectorized in row-major order. Each element in the vector corresponds to a patch (1.15° × 1.15°) in a stimulus pattern. `1` and `0` represnets a flickering checkerboard and a gray area, respectively. For example, stimulus pattern of

000000000000000000000000000000000000000111111000000111111000000110011000000110011000000110011000000110011000000000000000000000000000000000000000

represents the following stimulus.

000000000000
000000000000
000000000000
000111111000
000111111000
000110011000
000110011000
000110011000
000110011000
000000000000
000000000000
000000000000

The column holds 'null' for rest trials.

### Comments added by Openfmri Curators ###
===========================================

General Comments
----------------

Defacing
--------
Pydeface was used on all anatomical images to ensure de-identification of subjects. The code
can be found at https://github.com/poldracklab/pydeface

Quality Control
---------------
MRIQC was run on the dataset. Results are located in derivatives/mriqc. Learn more about it here: https://mriqc.readthedocs.io/en/stable/

Where to discuss the dataset
----------------------------
1) www.openfmri.org/dataset/ds******/ See the comments section at the bottom of the dataset
page.
2) www.neurostars.org Please tag any discussion topics with the tags openfmri and dsXXXXXX.
3) Send an email to [e-mail]. Please include the accession number in
your email.

Known Issues
------------
-behavioral performance data is not accompanied this dataset as submitter didn't submit.