Speech annotations of unscripted monologues

Contact information:

[e-mail], [e-mail]

Context:

This repository contains precisely time-resolved linguistic speech annotations derived from a published audiovisual dataset of unscripted monologues (Daeglau et al., 2023; https://doi.org/10.5281/zenodo.8082844). The annotations capture linguistic features on a word and phoneme level aligned at fine temporal resolution, enabling accurate mapping between speech content and its temporal structure. This speech annotation dataset provides a robust foundation for investigating the dynamics of natural speech in neurolinguistic research.

File naming convention:

Each folder contains files with a common overall structure to allow a direct link between the audiovisual recordings dataset and the corresponding linguistic speech annotations. For each subject and take the data are stored in separate files. The file naming convention is as follows:

Subject_<subject_ID>_Take<take_num>_<add_feature>.<extension>

subject_ID: unique integer number for each subject ranging from 1 to 6.

take_num: running integer index with leading zeros (01, 02, …) indicating the take number.

add_feature: in some takes an additional feature was introduced.

“MG” indicating when babble noise was played over in-ear phones

“LS” indicating that the subject wore lipstick during the take

“BR” indicating that the subject wore glasses during the take

extension: taking the value of .json or .tsv

Data description:

Each folder contains an _event.tsv file, a spreadsheet that describes the time-resolved linguistic annotation at the word and phoneme level. The structure of this event table is as follows:

Column name (Variable)
Description

onset
Onset of the event in seconds.

duration
Duration of the event in seconds.

event_type

The main category of the event."pause" - onset of a silent interval"word" - onset of a word"phoneme" - onset of a phoneme

mau
Phonetic segmentation of the event in XSampa.

mas
Phonetic segmentation of the event in XSampa including syllable segmentation, the point “.” marks a new syllable.

kan
Canonical phonological transcript (standard pronunciation) of the event in XSampa.

kas
Canonical phonological transcript (standard pronunciation) of the event in XSampa including syllable segmentation, the point “.” marks a new syllable.

ort
Orthographic representation of the event.

rftag
Part-of-speech Tags based on RFTagger.

lemma
Base form or dictionary form of the word.

token
Word index starting from 1.

It's important to note that the onset information of the event is given relative to the start of the audiovisual recording, while the duration information is given relative to the onset of the event. 

The _event.json file is a sidecar that describes the categorical and value columns of the event spreadsheet in json-format. Additionally, we included in each folder a _HED.tsv event file. It contains the same event information represented in the _event.tsv file but in machine-actionable HED format. HED (Hierarchical Event Descriptors) is a framework for annotating events in time-series data using a structured standardised vocabulary.