Classifying sex and strain from mouse ultrasonic vocalizations using deep learning

Vocalizations are widely used for communication between animals. Mice use a large repertoire of ultrasonic vocalizations (USVs) in different social contexts. During social interaction recognizing the partner's sex is important, however, previous research remained inconclusive whether individual USVs contain this information.
Using deep neural networks (DNNs) to classify the sex of the emitting mouse from the spectrogram we obtain unprecedented performance (77%, vs. SVM: 56%, Regression: 51%). Performance was even higher (85%) if the DNN could also use each mouse's individual properties during training, which may, however, be of limited practical value. Splitting estimation into two DNNs, spectrogram-to-features and features-to-sex (60%) failed to reach single-step performance. Extending the features by each USVs spectral line, frequency and time marginal in a semi-convolutional DNN resulted in a performance mid-way (64%). Analyzing the network structure suggests an increase in sparsity of activation and correlation with sex, specifically in the fully-connected layers. A detailed analysis of the USV structure, reveals a subset of male vocalizations characterized by a few acoustic features, while the majority of sex differences appear to rely on a complex combination of many features. The same network architecture was also able to achieve above-chance classification for cortexless mice, which were considered indistinguishable before.
In summary, spectrotemporal differences between male and female USVs allow at least their partial classification, which enables sexual recognition between mice and automated attribution of USVs during analysis of social interactions.

发声行为广泛应用于动物的通讯交流之中。小鼠在各类社交场景中，可调用丰富的超声发声（ultrasonic vocalizations, USVs）库进行交流。社交互动中，识别同伴性别是关键环节，但既往研究尚未明确，单条超声发声是否携带性别相关信息。

本研究借助深度神经网络（deep neural networks, DNNs），基于声谱图对发声小鼠的性别进行分类，取得了前所未有的分类性能：准确率达77%，而支持向量机（Support Vector Machine, SVM）与回归模型的准确率分别为56%与51%。若训练阶段允许深度神经网络利用单只小鼠的个体特征，分类准确率可进一步提升至85%，但该方法的实际应用价值或较为有限。若将分类任务拆解为两个深度神经网络模块：分别实现声谱图到特征提取、特征到性别分类的功能，其分类准确率仅为60%，未能达到单步分类的性能水平。在半卷积深度神经网络中，为每条超声发声补充谱线、频率及时域边际特征后，分类准确率达到64%，性能介于前述两类方法之间。对网络结构的分析显示，模型激活的稀疏性以及与性别特征的关联度均有所提升，这一现象在全连接层中尤为突出。对超声发声结构的详细分析表明，部分雄性小鼠的发声仅需少数声学特征即可区分，而绝大多数性别差异则依赖于多种特征的复杂组合。同一网络架构还可对无皮层小鼠的超声发声实现高于随机猜测水平的性别分类——此前这类样本被认为无法通过发声区分性别。

综上，雌雄小鼠超声发声的谱时差异至少可实现部分性别分类，这一发现为小鼠间的性别识别，以及社交互动分析过程中超声发声的自动化性别归因提供了可行路径。