Investigating the Impact of Visual Stimuli on the Auditory Selective Attention in a VR Classroom

General

The audio-visual Auditory Selective Attention - visual Priming (avASAvisPrim) project serves to investigate the impact of visual stimuli on auditory selective attention switch in a close-to-real-life classroom setting. This dataset consists of a Unity project and Matlab code used to collect data as well as the collected data and R scripts used for evaluation.

The dataset contains:

    Unity project for audiovisual display and the experiment structure    Matlab code for experiment preparation and HpFT measurement    Data collected in the experiment (experiment performance)    R code for data evaluation

Experiment preparation using Matlab

The code and software used to prepare the experiment is provided in the folder "matlab_avASAvisPrim".

The Matlab code used to prepare the trials for each participant as well as to measure the HpTFs. For the HpTF measurements, the  ITA Toolbox for Matlab was used and is provided (https://git.rwth-aachen.de/ita/toolbox). A developmental version of Virtual acoustics (VA) 2021a (https://www.virtualacoustics.org/VA/overview/) is provided.

Software requirements:

    Matlab 2020a or higher    ITA Toolbox for Matlab installed

Experiment conduction in Unity

The Unity project is provided in the folder "unity_avASAvisPrim".

Therefore, a virtual classroom with some basic furniture is provided. The used models, prefabs and plugins can be found in the Assets folder.

The acoustic stimuli for the task are taken from Loh and Fels 2023 "ChildASA dataset: Speech and Noise Material fpr Child-appropriate Paradigms on Auditory Selective Attention" https://doi.org/10.18154/RWTH-2023-00740. 

This Unity project was intended for the use in virtual reality using an HMD and respective controllers for input. However, it can also be used on a desktop pc. The mode can be changed using the "VRMode" toggle as described below.The audio reproduction is realized using the Unity plugin for Virtual Acoustics (VA, http://www.virtualacoustics.org/ and https://git.rwth-aachen.de/ita/vaunity_package).

Software requirements:

    Unity 2019.4.21.f1.    SteamVR 1.19.7    Virtual Acoustics v2021a, VAUnity: https://git.rwth-aachen.de/ita/VAUnity

Data evaluation

The collected data is provided in the folder "dataEvaluation_avASAvisPrim". This folder contains three types of data: the raw data collected in the experiment (reaction times and error rates). R code for the evaluation of the head tracking data and the questionnaires is provided.