The Distant Listening Corpus

This is a README file for a data repository originating from the DCML corpus initiative and serves as welcome page for both the GitHub repo https://github.com/DCMLab/distant_listening_corpus and the corresponding documentation page https://dcmlab.github.io/distant_listening_corpus For information on how to obtain and use the dataset, please refer to this documentation page. The Distant Listening Corpus Cite as Version history Getting the data With full version history Without full version history Data Formats Opening Scores Opening TSV files in a spreadsheet Loading TSV files in Python How to read metadata.tsv File information Composition information Score information Identifiers Generating all TSV files from the scores Questions, Suggestions, Corrections, Bug Reports License Overview ABC bach_en_fr_suites bach_solo bartok_bagatelles beethoven_piano_sonatas c_schumann_lieder chopin_mazurkas corelli couperin_clavecin couperin_concerts cpe_bach_keyboard debussy_suite_bergamasque dvorak_silhouettes frescobaldi_fiori_musicali grieg_lyric_pieces handel_keyboard jc_bach_sonatas kleine_geistliche_konzerte kozeluh_sonatas liszt_pelerinage mahler_kindertotenlieder medtner_tales mendelssohn_quartets mozart_piano_sonatas pergolesi_stabat_mater peri_euridice pleyel_quartets poulenc_mouvements_perpetuels rachmaninoff_piano ravel_piano scarlatti_sonatas schubert_winterreise schulhoff_suite_dansante_en_jazz schumann_kinderszenen schumann_liederkreis sweelinck_keyboard tchaikovsky_seasons wagner_overtures wf_bach_sonatas The Distant Listening Corpus This corpus has been created within the DCML corpus initiative and employs the DCML harmony annotation standard. The publication covers the following public corpora (the DOI links always point at the latest version respectively): Ludwig van Beethoven - String Quartets Ludwig van Beethoven - Piano Sonatas Frédéric Chopin - Mazurkas Arcangelo Corelli – Trio Sonatas Claude Debussy - Suite Bergamasque Antonín Dvořák - Silhouettes Edvard Grieg - Lyric Pieces Franz Liszt - Années de Pèlerinage Nikolai Medtner - Tales Wolfgang Amadeus Mozart - Piano Sonatas Robert Schumann - Kinderszenen Pyotr Tchaikovsky - The Seasons Moreover, it covers the following corpora that will become public with publication of the DLC data report: J.S. Bach – English and French Suites J.S. Bach – Solo Pieces (A corpus of annotated scores) Béla Bartók – 14 Bagatelles, Op. 6 François Couperin – L'art de toucher le clavecin François Couperin – Concerts Royaux Clara Schumann – Lieder Carl Philipp Emanuel Bach – Works for Keyboard Girolamo Frescobaldi (1583-1643) – Fiori Musicali, op. 12 (1635) Georg Friedrich Händel – Grobschmied Variations (The Harmonious Blacksmith), HWV 430 J.C. Bach – Keyboard Sonatas Heinrich Schütz – Kleine Geistliche Konzerte Leopold Koželuch – Piano Sonatas Gustav Mahler – Kindertotenlieder Felix Mendelssohn – String Quartets Giovanni Battista Pergolesi – Stabat Mater (1736) Jacopo Peri – Euridice (1600) Ignaz Pleyel – String Quartets Francis Poulenc – Mouvements Perpetuels Sergei Rachmaninoff – Piano Pieces Maurice Ravel – Piano Pieces Domenico Scarlatti – Keyboard Sonatas Franz Schubert – Winterreise Erwin Schulhoff – Suite dansante en jazz Robert Schumann – Liederkreis Jan Sweelinck – Organ Pieces Richard Wagner – OverturesWilhelm Friedemann Bach – Piano Sonatas Cite as Version history See the GitHub releases. Getting the data With full version history The dataset is version-controlled via git. In order to download the files with all revisions they have gone through, git needs to be installed on your machine. Then you can clone this repository using the command git clone --recurse-submodules -j8 https://github.com/DCMLab/distant_listening_corpus.git -j8 is available from git >= 2.8 and fetches up to 8 submodules at a time in parallel Without full version history If you are only interested in the current version of the corpus, you can download the ZIP file of each submodule: ABC bach_en_fr_suites bach_solo bartok_bagatelles beethoven_piano_sonatas c_schumann_lieder chopin_mazurkas corelli couperin_clavecin couperin_concerts cpe_bach_keyboard debussy_suite_bergamasque dvorak_silhouettes frescobaldi_fiori_musicali grieg_lyric_pieces handel_keyboard jc_bach_sonatas kleine_geistliche_konzerte kozeluh_sonatas liszt_pelerinage mahler_kindertotenlieder medtner_tales mendelssohn_quartets mozart_piano_sonatas pergolesi_stabat_mater peri_euridice pleyel_quartets poulenc_mouvements_perpetuels rachmaninoff_piano ravel_piano scarlatti_sonatas schubert_winterreise schulhoff_suite_dansante_en_jazz schumann_kinderszenen schumann_liederkreis sweelinck_keyboard tchaikovsky_seasons wagner_overtures wf_bach_sonatas Data Formats Each piece in this corpus is represented by four files with identical names, each in its own folder. For example, the Prélude of J.S. Bach's first English Suite, BWV 806, has the following files: bach_en_fr_suites/MS3/BWV806_01_Prelude.mscx: Uncompressed MuseScore file including the music and annotation labels. bach_en_fr_suites/notes/BWV806_01_Prelude.tsv: A table of all note heads contained in the score and their relevant features (not each of them represents an onset, some are tied together) bach_en_fr_suites/measures/BWV806_01_Prelude.tsv: A table with relevant information about the measures in the score. bach_en_fr_suites/harmonies/BWV806_01_Prelude.tsv: A list of the included harmony labels (including cadences and phrases) with their positions in the score. Opening Scores After navigating to your local copy, you can open the scores in the folder MS3 with the free and open source score editor MuseScore. Please note that the scores have been edited, annotated and tested with MuseScore 3.6.2. MuseScore 4 has since been released and preliminary tests suggest that it renders them correctly. Opening TSV files in a spreadsheet Tab-separated value (TSV) files are like Comma-separated value (CSV) files and can be opened with most modern text editors. However, for correctly displaying the columns, you might want to use a spreadsheet or an addon for your favourite text editor. When you use a spreadsheet such as Excel, it might annoy you by interpreting fractions as dates. This can be circumvented by using Data --> From Text/CSV or the free alternative LibreOffice Calc. Other than that, TSV data can be loaded with every modern programming language. Loading TSV files in Python Since the TSV files contain null values, lists, fractions, and numbers that are to be treated as strings, you may want to use this code to load any TSV files related to this repository (provided you're doing it in Python). After a quick pip install -U ms3 (requires Python 3.10) you'll be able to load any TSV like this: import ms3 labels = ms3.load_tsv('bach_en_fr_suites/harmonies/BWV806_01_Prelude.tsv') notes = ms3.load_tsv('bach_en_fr_suites/notes/BWV806_01_Prelude.tsv')