Data from "CLUES: Cognitive Load Understanding through Experimental Sensing Dataset"

Multimodal dataset capturing synchronized physiological, behavioral, and self-report signals from adult participants performing cognitive tasks under relaxed, easy, and difficult conditions, designed for cognitive load (CL) estimation. Signals include facial and head measurements (smartglasses), heart-rate variability, skin temperature and electrodermal activity (wristband), gaze dynamics (eye-tracker), and RGB/thermal camera recordings, paired with subjective CL ratings (NASA-TLX and ISA) and task-performance measures. Collection period: Second half of 2024 - First half of 2025 Participants: 55 adults recruited across two sites (mean age 27.72, SD 6.58; range 19–47; 63.64% male). Study sites: Jožef Stefan International Postgraduate School (JSI; Ljubljana, Slovenia) and Università della Svizzera italiana (USI; Lugano, Switzerland). Ethics approval: CE-2024-27, USI. Access status: Restricted. The full dataset is shared for non-commercial academic use upon reasonable request, subject to a signed Dataset Sharing Agreement. Available Files (upon request approval) The dataset is organized hierarchically: one folder per participant, and within each participant folder, one folder per modality holding that modality's raw and/or processed data. All device streams were synchronized at recording time using LabRecorder (Lab Streaming Layer). CLUES/├── self_reports_and_performance.csv # One row per participant: questionnaires and task performance└── <subject_id>/ # One folder per participant ├── empatica/ # Empatica E4 wristband ├── ocosense/ # Emteq OCOsense smartglasses ├── eyetracker/ # Tobii Pro Spark ├── thermal/ # FLIR Lepton thermal camera └── rgb/ # Logitech BRIO 4K RGB camera self_reports_and_performance.csv has one row per participant; subject_id matches the folder names. age, sex, education: demographics focus, tiredness (1–10) and hours_slept: reported before the session nasa_tlx_<condition>: NASA-TLX, unweighted mean of the six subscales (0–100) isa_<condition>: ISA rating (1–5) score_<condition>: raw task score (correct responses/items; maximum 61 for n-back and Stroop, 7/15 for memory easy/difficult, 5 for image questions, 12/7 for difference images easy/difficult, 10 for pursuit) score_pct_<condition>: score_<condition> as a percentage of the maximum score_pct_overall: total score across all tasks as a percentage of the total maximum <condition> uses the same names as the label column of the sensor files: 2_back, 3_back, stroop_easy, stroop_difficult, memory_easy, memory_difficult, images_questions_easy, images_questions_difficult, difference_images_easy, difference_images_difficult, pursuit_easy, pursuit_difficult, and, for NASA-TLX and ISA only, the relaxation conditions rest, listen_music, passive_viewing. Missing answers are left empty. Notes Because of privacy constraints and consent scope (participants could opt out of specific data types, particularly video), the availability of certain modalities may vary by participant. Code Availability The code used for data preprocessing, feature extraction, machine learning experiments (person-dependent, person-independent, and task-independent CL estimation), explainability (SHAP) analysis, and figure generation is available in the repository connected to this dataset: https://github.com/USI-PCC/CLUES. Intended Uses Model and benchmark cognitive load estimation from multimodal physiological and behavioral signals, including person-dependent, person-independent, and task-independent settings to assess model generalization. Investigate how CL manifests across physiological systems and behavioral patterns, and identify the most informative signals/modalities via feature-importance and explainability analysis. Support cognition-aware and adaptive interface design, and ML personalization for interactive systems. Data Collection Protocol Summary Data were collected in a controlled laboratory session lasting approximately 60 minutes per participant (~40 minutes of active data collection). After reporting prior sleep duration and current tiredness/focus (1–10 scales), participants were fitted with the Emteq OCOsense smartglasses, Tobii eye-tracker (both calibrated), and the Empatica E4 wristband on the non-dominant hand; thermal and RGB cameras recorded facial and behavioral signals. The experiment comprised three sequential task domains - standardized CL tasks (N-back, Stroop), ecological memory tasks, and ecological visual-attention tasks (e.g. find-differences, line-tracking and enumeration) - each administered in an easy and a difficult version, with relaxation periods interleaved as baseline. After each task version, participants completed the NASA-TLX (six subscales, 0–100) and the single-item ISA as immediate subjective CL ratings. All device streams were synchronized and stored via LabRecorder. Access Instructions To obtain access to the dataset, please email BOTH authors with a short description of your research project and they will take care of starting the process to sign the Data Sharing Agreement: Francesco Bombassei De Bona — [e-mail] Ana Krstevska — [e-mail] Access will be granted for non-commercial academic use and requires signing the Dataset Sharing Agreement to protect participant privacy. Citation When citing the dataset, please use both of the following references: Ana Krstevska, Shivalika Goyal, Linda Fiorini, Francesco Bombassei De Bona, Daniil Kirilenko, Mohan Li, Sebastijan Trojer, Tomi Božak, Zoja Anžur, Mitja Luštrek, Hristijan Gjoreski, Marc Langheinrich, Gašper Slapničar, and Martin Gjoreski. 2026. CLUES: Cognitive Load Understanding through Experimental Sensing Dataset. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 10, 3, Article 116 (September 2026), 33 pages. https://doi.org/10.1145/3831656 Ana Krstevska, Shivalika Goyal, Linda Fiorini, Francesco Bombassei De Bona, Daniil Kirilenko, Mohan Li, Sebastijan Trojer, Tomi Božak, Zoja Anžur, Mitja Luštrek, Hristijan Gjoreski, Marc Langheinrich, Gašper Slapničar, and Martin Gjoreski. 2026. Data from "CLUES: Cognitive Load Understanding through Experimental Sensing Dataset" [Data set]. Zenodo. https://doi.org/10.5281/zenodo.21410787