Table_2_An Approach to Aligning Categorical and Continuous Time Series for Studying the Dynamics of Complex Human Behavior.docx

An emerging perspective on human cognition and performance sees it as a kind of self-organizing phenomenon involving dynamic coordination across the body, brain and environment. Measuring this coordination faces a major challenge. Time series obtained from such cognitive, behavioral, and physiological coordination are often complicated in terms of non-stationarity and non-linearity, and in terms of continuous vs. categorical scales. Researchers have proposed several analytical tools and frameworks. One method designed to overcome these complexities is recurrence quantification analysis, developed in the study of non-linear dynamics. It has been applied in various domains, including linguistic (categorical) data or motion (continuous) data. However, most previous studies have applied recurrence methods individually to categorical or continuous data. To understand how complex coordination works, an integration of these types of behavior is needed. We aimed to integrate these methods to investigate the relationship between language (categorical) and motion (continuous) directly. To do so, we added temporal information (a time stamp) to categorical data (i.e., language), and applied joint recurrence analysis methods to visualize and quantify speech-motion coordination coupling during a rap performance. We illustrate how new dynamic methods may capture this coordination in a small case-study design on this expert rap performance. We describe a case study suggesting this kind of dynamic analysis holds promise, and end by discussing the theoretical implications of studying complex performances of this kind as a dynamic, coordinated phenomenon.

学界新兴的人类认知与行为表现视角将其视为一种自组织现象，涉及身体、大脑与环境之间的动态协同。对这类协同进行量化面临重大挑战：从这类认知、行为与生理协同中获取的时间序列，往往在非平稳性、非线性以及连续型与类别型量表的区分上存在复杂特性。研究者已提出多种分析工具与研究框架。其中，针对非线性动力学研究发展出的循环量化分析（Recurrence Quantification Analysis），正是为克服上述复杂性而设计的方法之一。该方法已被应用于多个领域，包括语言（类别型）数据与运动（连续型）数据。然而，既往多数研究仅将循环分析方法单独应用于类别型或连续型数据。若要厘清复杂协同的运作机制，需对这两类行为数据进行整合分析。本研究旨在整合此类方法，直接探究语言（类别型）与运动（连续型）之间的关联。为此，我们为类别型数据（即语言数据）添加了时序信息（时间戳），并应用联合循环分析方法，对说唱表演过程中的言语-运动协同耦合效应进行可视化与量化分析。我们通过针对该专业说唱表演的小型案例研究设计，展示了新型动态方法如何捕捉这类协同效应。本研究通过案例研究表明，这类动态分析方法颇具应用前景；最后我们还讨论了将此类复杂表演作为动态协同现象进行研究的理论意义。