Sounds and the city – Acoustic detection of open windows inindoor environments

(1) Background: Situated in the domain of urban sound scene classification by humans and machines, the research in this project will be a first step towards mapping urban noise pollution experienced indoors and finding ways to reduce its negative impact in peoples' homes. The acoustic distinction between outdoor and indoor scenes is an active research field and can be automated with some success. A much subtler difference is the change in the indoor soundscape induced by an open window. Being able to determine this, however, would allow applications in warning systems and be a prerequisite for an app-based urban sound mapping project. Acoustic detection requires neither line of sight nor sensors at the window frame or knowledge of the number of windows or their size. The task, however, varies substantially in difficulty with the amount of sound inside and outside. From the point of machine classification the lack of specificity is the most problematic aspect: Very few sounds if any can be assumed to originate exclusively from outside <em>and</em> be present at all times to aid automatic detection. The required generalisation ability, however, can be assumed for humans, who might also use very subtle cues in the change of reverberations. (2) Aims The aims are (a) to determine the degree of reliability with which an open window can be recognised by humans and machines under varying circumstances based only on acoustic cues; (b) to investigate whether the findings for humans and machines can inform each other and can be used for further application-related research, e.g., window noise cancellation. (3) Method: (a) Dataset acquisition: A recording kit consisting of a dedicated laptop and microphone will be given to volunteers. Custom-programmed software will remind the user to specify the window state (establishing the so-called ground truth). (b) Perception experiments: Thirty participants will judge whether in the recorded clips a window is open or closed. After an extended familiarisation phase, they will proceed through two testing phases: In the first phase, all clips will originate from the recording locations with which the participants have already familiarised themselves, in the second they will judge clips from locations they haven't been exposed to before (partial data sets used for the familiar/unfamiliar conditions will be counterbalanced across participants). (c) Machine recognition: We will develop a machine learning system using state-of-the-art deep learning methods (artificial neural networks with multiple layers). To encourage other researchers to also take up this research, we will organise a machine learning challenge. In the challenge, a training data set including correct labels (ground truth) and a test set without the labels are provided. Researchers from academia and industry across the world will develop their own systems and send their classification results on the test set to the organisers to evaluate and publish online.(1) 研究背景：本项目的研究锚定人类与机器的城市声场景分类领域，旨在为映射室内遭遇的城市噪声污染、探寻降低其对居家生活负面影响的路径迈出第一步。室外与室内声场景的声学区分已是当前活跃的研究方向，且已实现一定程度的自动化落地。而更为微妙的差异，是开窗操作引发的室内声景变化。不过，若能精准识别这一变化，即可应用于预警系统，同时也是开发基于应用程序的城市声景测绘项目的必要前提。声学检测无需视线路径、无需在窗框部署传感器，亦无需知晓窗户的数量与尺寸。但该任务的难度会随室内外声强的变化产生显著波动。从机器分类的视角来看，样本缺乏特异性是最棘手的问题：几乎没有声音可以被认定为仅来自室外，且始终存在，以辅助自动检测。不过，人类具备所需的泛化能力，或许还可利用混响变化这类极细微的线索。 (2) 研究目标：本研究的目标分为两点：(a) 明确仅依靠声学线索，人类与机器在不同场景下识别开窗状态的可靠程度；(b) 探究人类与机器的研究结果能否相互借鉴，并用于后续与应用相关的研究，例如开窗噪声抑制。 (3) 研究方法： (a) 数据集采集：将为志愿者配备由专用笔记本电脑与麦克风组成的录音套件。定制开发的软件会提醒用户记录窗户的状态（以此建立所谓的真实标签（ground truth））。 (b) 感知实验：招募30名参与者，判断录制的音频片段中窗户处于开启还是关闭状态。经过充分的熟悉阶段后，参与者将完成两个测试阶段：第一阶段中，所有音频片段均来自参与者已熟悉的录制地点；第二阶段则需判断来自此前未接触过的地点的音频片段（用于熟悉/不熟悉场景的部分数据集将在参与者间进行平衡处理）。 (c) 机器识别：我们将采用当前领先的深度学习方法（多层人工神经网络）开发机器学习系统。为鼓励其他研究者参与该领域的研究，我们将举办一场机器学习挑战赛。挑战赛将提供带正确标签（真实标签）的训练数据集，以及不带标签的测试数据集。全球学术界与工业界的研究者可开发各自的模型，并将测试集上的分类结果提交至主办方进行评估，最终结果将在线公布。