Phonetics Embedding Learning with Side Information

We show that it is possible to learn an efficient acoustic model using only asmall amount of easily available word-level similarity annotations. In contrastto the detailed phonetic labeling required by classical speech recognitiontechnologies, the only information our method requires are pairs ofspeech excerpts which are known to be similar (same word) and pairs ofspeech excerpts which are known to be different (different words). An acoustic model is obtained by training shallow and deep neural networks, using anarchitecture and a cost function well-adapted to the nature of the provided information. The resulting model is evaluated on an ABX minimal-pair discrimination task and is shown to perform much better (11.8% ABX errorrate) than raw speech features (19.6%), not far from a fully supervised baseline (best neural network: 9.2%, HMM-GMM: 11%).