Multimodal Visual Search Dataset with EEG and Eye-tracking

This is the Multimodal dataset with Eye-tracking and EEG for Visual Search task. The dataset included 36 users, with each participant searching for targets in 140 scenes. Dataset includes 2 kinds of Stimulus: Workshop stimuli consists of wide range of scenes from assembly units, manufacturing and production facilities, industrial laboratories, and garageand repair workshops and Desktop stimuli featuring a broad variety of visual elements, including texts, abstract patterns,and human figures, each with diverse designs, colors, and motifs. EEG and Eye-tracking data is recorded throughout the data recording. All the markers and input modality are synchronized and we recorded the dataset using Lab Streaming Layer. All the events including markers, user interaction, meta data is stored and can be found in Marker stream. Marker stream, EEG, Eye-tracking data and time stamps can be found in respective streams. 

Hardware Setup:

EEG Recording:

Device: Enobio 20-channel system (Neuroelectrics)

Sampling rate: 500 Hz

Electrode placement: 10–20 international system

Eye-Tracking:

Device: Tobii Pro Nano (wireless)

Sampling rate: 60 Hz

Mounted on the lower bezel of the monitor

Viewing Conditions:

Viewing distance: ~60 cm

Visual angle: ~27.54°

Experimental Procedure

Preparation Phase

Participants were:

Briefed about the study and setup

Introduced to EEG and eye-tracking equipment

Asked to provide informed consent

Required to complete a demographics questionnaire

Setup details:

Participants seated comfortably

EEG cap fitted and electrodes filled with gel

Preparation time: ~20–30 minutes

Signal quality checked before recording

 A single trial of the experiment comprised three steps.

In Step 1, we presented the instructions to begin the study: “Press enter to start”, followed by a 5-second countdown.

In Step 2, we displayed the instruction “Search for this item” with the image of the target tool or desktop icon for 5 seconds.

Finally, in Step 3, the participant searched for the shown target tool or desktop icon within the scene until they found it. Participants were asked to click on the target using a mouse only after they had located it. This approach minimized distractions from the mouse pointer and reduced the impact of hand movement artifacts on the EEG during the search phase. Participants could skip the search by pressing the ’S’ key if they were unable to locate the target. We divided the 140 unique scenes into four sessions, including breaks between each session. The first two sessions included workshop scenes, while the last two sessions contained desktop scenes. The sequence of scenes and target objects within each session was randomized. On average, each participant completed the experiment in 120 minutes.