A new, comprehensive database of all proceedings of the Australian Parliamentary Debates (1998-2022)

This database contains a CSV file for each sitting day in the Australian Parliament by the House of Representatives from 02 March 1998 to 08 September 2022. In addition to the individual CSV files, this version contains a ZIP file with all the CSVs (main-filled-csv-v2.zip), and a ZIP File with all the files in Parquet format (main-filled-parquet-v2.zip). These files were parsed entirely from the XML Hansard transcripts available on the Australian Parliament website. We developed four R scripts to parse and clean all of these XML files, and ran each file through two additional scripts: one to fill in missing speaker details, and one to validate each file using a suite of 7 automated tests. All scripts used to build this database are available at https://github.com/lindsaykatz/hansard-proj. Version 2 contains data on an additional 13 years of Hansard records, and all data in this version were parsed using a slightly different approach than what was used in version 1, which allows for better preservation of the correct chronological ordering of statements.