Learning States

3. Learning statesWhile executing the proposed tasks, the participants’ physiological signals were monitored using two biosignalsplux devices from PLUX Wireless Biosignals, Lisbon, Portugal, with a sampling frequency of 100 Hz a resolution of 16 bits (24 bits in the case of fNIRS). Six different sensors were used: EEG and fNIRS positioned around the F7 and F8 of the 10–20 system (dorsolateral prefrontal cortex is often used to assess CW and fatigue as well as cognitive states); ECG monitored an approximation of Lead I of the Einthoven system, EDA placed on the palm of the non-dominant hand; ACC was positioned on the right side of the head to measure head movement and overall posture changes, and the RIP sensor was attached to the upper-abdominal area to measure the respiration cycles—the combination of the three allows to infer about the response of the Autonomic Nervous System (ANS) of the human body, namely, the response of the sympathetic and parasympathetic nervous system.3.1. Experimental designData collection was centered on a learning activity involving an ECG lesson presented in a Jupyter Notebook. This lesson included the same theoretical content as a prior study but incorporated open-ended questions designed to prompt active reflection on the learning process. Participants were required to recall and apply the material they had reviewed to formulate thoughtful responses. To monitor reading behaviors, all text and images were initially blurred, becoming visible only when participants hovered their mouse over specific sections. This method enabled precise tracking of their reading progress.3.2. Data recordingThis study involved 18 volunteers, including 13 females, aged between 18 and 28 (M = 21.7, SD = 3.2). These participants were enlisted from the NOVA School of Science and Technology. All were right-handed, reported no history of psychological disorders, and were not on any regular medication. Written informed consent was obtained before participating and all Ethical Procedures approved by the Ethics Committee of NOVA University of Lisbon were thoroughly followed.During data collection, electromagnetic noise saturated one of the EEG channels in one participant, and excessive sweat caused saturation of the EDA signals in two participants. These signals were excluded from further analysis. Additionally, HCI variables were not recorded for subject 2. In this case, the files don't have timestamps (because there's no need to synchronise the biosignals with the HCI features) but they do have nseq.3.3. Data labellingFor the data labelling, participants (biomedical engineering students or researchers) observed a video with their screen activity, facial expressions, and biosignals during the experiment. They were tasked with self-reporting their cognitive states by selecting from a predefined list of learning states, accompanied by their definition:<b>Neutral:</b> A state without any discernible emotion or cognitive engagement.<b>Interest/Flow:</b> A state of focused interest or deep involvement in the task.<b>Surprise:</b> The reaction to encountering something unexpected or new.<b>Boredom:</b> A lack of interest or disengagement with the task.<b>Distraction:</b> A loss of focus due to internal or external interruptions.<b>Confusion:</b> Difficulty understanding the content or task at hand.<b>Eureka:</b> A moment of insight or discovery after a period of struggle.<b>Frustration:</b> A feeling of dissatisfaction or disappointment caused by unmet expectations or goals.Participants were instructed to label their cognitive states whenever they experienced a notable change in their learning process or at regular two-minute intervals if no changes occurred. This self-reported labeling, documented in the learning_states_labels.txt files and overseen by the principal investigator, ensured that the recorded states closely reflected the participants' internal experiences.3.4. Data description<b>Biosignals</b> include EEG, fNIRS (not converted to oxi and deoxiHb), ECG, EDA, respiration (RIP), accelerometer (ACC), and push-button data (PB). All signals have already been converted to physical units. In each biosignal file, the first column corresponds to the timestamps.<br><b>HCI features</b> encompass keyboard, mouse, and screenshot data. Below is a Python code snippet for extracting screenshot files from the screenshots CSV file.​<pre>import base64<br>from os import mkdir<br>from os.path import join<br><br>file = '...'<br><br>with open(file, 'r') as f:<br>    lines = f.readlines()<br><br>for line in lines[1:]:<br>    timestamp = line.split(',')[0]<br>    code = line.split(',')[-1][:-2]<br>    imgdata = base64.b64decode(code)<br>    filename = str(timestamp) + '.jpeg'<br>   <br>    mkdir('screenshot')<br><br>    with open(join('screenshot', filename), 'wb') as f:<br>        f.write(imgdata)<br></pre>A characterization file containing age and gender information for all subjects in each dataset is provided within the respective dataset folder (e.g., <code>D3_subject-info.csv</code>). Other complementary files include (i) description of the pushbuttons to help segment the signals (e.g., <code>D3_S2_PB_description.json</code>) and (ii) labelling (e.g., <code>D3_S2_learning_states_labels.txt</code>).<br>