Sounds and the city – Acoustic detection of open windows inindoor environments

(1) Background:

Situated in the domain of urban sound scene classification by humans and machines, the research in this project will be a first step towards mapping urban noise pollution experienced indoors and finding ways to reduce its negative impact in peoples' homes. The acoustic distinction between outdoor and indoor scenes is an active research field and can be automated with some success. A much subtler difference is the change in the indoor soundscape induced by an open window. Being able to determine this, however, would allow applications in warning systems and be a prerequisite for an app-based urban sound mapping project.

Acoustic detection requires neither line of sight nor sensors at the window frame or knowledge of the number of windows or their size. The task, however, varies substantially in difficulty with the amount of sound inside and outside. From the point of machine classification the lack of specificity is the most problematic aspect: Very few sounds if any can be assumed to originate exclusively from outside and be present at all times to aid automatic detection. The required generalisation ability, however, can be assumed for humans, who might also use very subtle cues in the change of reverberations.

(2) Aims

The aims are

(a) to determine the degree of reliability with which an open window can be recognised by humans and machines under varying circumstances based only on acoustic cues;

(b) to investigate whether the findings for humans and machines can inform each other and can be used for further application-related research, e.g., window noise cancellation.

(3) Method:

(a) Dataset acquisition:

A recording kit consisting of a dedicated laptop and microphone will be given to volunteers. Custom-programmed software will remind the user to specify the window state (establishing the so-called ground truth).

(b)  Perception experiments:

Thirty participants will judge whether in the recorded clips a window is open or closed. After an extended familiarisation phase, they will proceed through two testing phases: In the first phase, all clips will originate from the recording locations with which the participants have already familiarised themselves, in the second they will judge clips from locations they haven't been exposed to before (partial data sets used for the familiar/unfamiliar conditions will be counterbalanced across participants). 

(c) Machine recognition:

We will develop a machine learning system using state-of-the-art deep learning methods (artificial neural networks with multiple layers). To encourage other researchers to also take up this research, we will organise a machine learning challenge. In the challenge, a training data set including correct labels (ground truth) and a test set without the labels are provided. Researchers from academia and industry across the world will develop their own systems and send their classification results on the test set to the organisers to evaluate and publish online.