NeuMa (Raw): A multimodal Neuromarketing dataset

A novel multimodal Neuromarketing dataset that encompasses the data from 42 individuals who participated in an advertising brochure-browsing scenario is introduced here. In more detail, participants were exposed to a series of supermarket brochures (containing various products) and instructed to select the products they intended to buy. The data collected for each individual executing this protocol included: (i) encephalographic (EEG) recordings, (ii) eye tracking (ET) recordings, (iii) questionnaire responses (demographic, profiling and product related questions), and (iv) computer mouse data. The preprocessed version of this dataset can be found here: https://figshare.com/articles/dataset/NeuMa_PreProcessed_A_multimodal_Neuromarketing_dataset/22117124 Experimental Protocol The experimental paradigm simulated the browsing experience of a digital advertising brochure, where the participants had to select the products they intended to buy. The paradigm was designed accordingly so as to monitor the subject’s brain and ocular activity throughout the browsing experience, with the original scope being the identification of distinct brain and ocular activity patterns with respect to selected (i.e. intended to buy) and non selected products. Aiming to mimic this experience in a realistic way, the participants were instructed to select the products (using a computer mouse to click on them) they intended to buy in accordance to their regular buying habits, without having any global restrictions with respect to either the cost or the total number of products bought. During the experimental procedure, participants were seated in a comfortable armchair placed 50 cm from a 28 inch LCD monitor. Prior to the product presentation, resting state EEG was recorded for two minutes. Once the recording of resting state was completed, participants could freely browse among the provided brochures by pressing the keyboard’s left and right arrow to move forwards and backwards respectively. In total 6 brochures were provided. Each page encompassed a total number of 24 different supermarket products that belonged to the same product category (e.g. the first collection included dairy products, the second frozen products, etc.). During this task the participants were instructed to identify and select (by left clicking on) the products they intended to buy.Once the browsing/product selection experience was completed (ended by each participant on his/her own time), each participant was asked to fill in a questionnaire. The questionnaire included demographic questions (e.g. age, marital status, education, etc.), profiling questions (e.g. the big five personality traits, impulsive buying behaviour, etc.) and specific questions related to each of the selected products (i.e. reasons of selection, familiarity towards the selected product and whether the selected product constitutes a frequent purchase or not). Participant Demographics A total of 45 individuals participated in this study. However, the dataset in its final form includes 42 subjects (23 males and 19 females, aged 31.5 ± 8.84), denoted as S01, S02, …, S42, as the data from three subjects were excluded due to high levels of artifactual contamination. Acquisition setup The raw data was recorded using Wearable Sensing’s DSI 24 and Tobii Pro Fusion for the EEG and ET data respectively. The brain activity was recorded, with a sampling frequency of 300Hz, via 21 dry sensors, namely Fp1, Fp2, Fz, F3, F4, F7, F8, Cz, C3, C4, T7/T3, T8/T4, Pz, P3, P4, P7/T5, P8/T6, O1, O2, A1 and A2, that were placed at locations corresponding to the 10-20 International System. The Sensors A1 and A2 were the reference electrodes and were placed on the mastoids. Prior to the experimental procedure, impedance for all electrodes was set below 10KΩ and EEG signals were inspected to avoid any irregularities. The gaze data was recorded via Tobii’s screen based solution, with the eye movements being captured at a sampling frequency of 120Hz. Prior to initiation of the experimental procedure, a 9-point calibration was performed for each subject, using the Tobii Pro SDK, to formulate the individual’s eye model and gaze point.

本文提出一款新型多模态神经营销（Neuromarketing）数据集，其收录了42名参与广告宣传手册浏览场景实验的受试者的各类数据。具体而言，受试者被展示一系列包含各类商品的超市宣传手册，并被要求挑选出自身计划购买的商品。针对执行该实验流程的每名受试者，我们采集了以下四类数据：(i) 脑电图（EEG）记录；(ii) 眼动追踪（ET）记录；(iii) 问卷反馈（涵盖人口统计学信息、个人画像及商品相关问题）；(iv) 计算机鼠标操作数据。该数据集的预处理版本可通过以下链接获取：https://figshare.com/articles/dataset/NeuMa_PreProcessed_A_multimodal_Neuromarketing_dataset/22117124

## 实验范式
本实验范式模拟了数字广告宣传手册的浏览体验，受试者需从中挑选出计划购买的商品。该范式的设计初衷是在浏览过程中监测受试者的脑活动与眼动活动，核心目标是识别出与已选（即计划购买）及未选商品对应的特异性脑活动与眼动活动模式。为真实还原日常购物体验，受试者被要求按照自身常规购物习惯，通过计算机鼠标点击选中计划购买的商品，且未对商品单价或总购买数量设置任何全局限制。
实验过程中，受试者舒适端坐于扶手椅上，与一台28英寸液晶显示器的距离为50厘米。在商品展示前，先采集2分钟的静息态脑电图数据。静息态数据采集完成后，受试者可通过键盘左右箭头键分别向前或向后翻页，自由浏览提供的宣传手册。本次实验共提供6份宣传手册，每份手册包含24种隶属于同一商品类别的超市商品（例如第一组为乳制品，第二组为冷冻食品等）。在此任务阶段，受试者需识别并通过左键点击选中计划购买的商品。
待浏览与商品选择任务完成（由受试者自行终止任务）后，每名受试者需填写一份问卷。问卷内容包括人口统计学问题（如年龄、婚姻状况、受教育程度等）、个人画像问题（如大五人格特质、冲动性购买行为等），以及与每名选中商品相关的特定问题（例如选择该商品的原因、对选中商品的熟悉程度，以及该商品是否为高频购买商品）。

## 受试者人口统计学信息
本研究共招募45名受试者参与实验，但最终数据集仅包含42名受试者的数据（23名男性，19名女性，年龄均值±标准差为31.5±8.84），编号为S01、S02、…、S42。剔除3名受试者的原因是其数据存在严重的人工伪迹污染。

## 数据采集设置
原始数据分别采用Wearable Sensing公司的DSI 24设备与Tobii Pro Fusion设备采集脑电图（EEG）与眼动追踪（ET）数据。
脑电信号通过21个干电极采集，采样频率为300Hz，电极位置符合国际10-20系统（10-20 International System），具体为：Fp1、Fp2、Fz、F3、F4、F7、F8、Cz、C3、C4、T7/T3、T8/T4、Pz、P3、P4、P7/T5、P8/T6、O1、O2、A1及A2。其中A1与A2为参考电极，放置于乳突位置。实验开始前，所有电极的阻抗均设置为低于10KΩ，并对脑电信号进行检查以排除异常波动。
眼动数据通过Tobii的屏幕式眼动采集方案获取，眼动轨迹的采样频率为120Hz。实验正式开始前，使用Tobii Pro软件开发工具包（SDK）为每名受试者完成9点校准，以构建个人眼动模型与注视点映射关系。