Wild Echo Australian Bird Species Classifier Benchmark v2.1 (May 2026)

This dataset contains the evaluation materials and results from the Wild Echo Australian Bird Species Classifier Benchmark v2.1, conducted in May 2026. The benchmark provides systematic performance measurements for automated bird species identification systems applied to Australian avifauna, including raw evaluation results, methodology documentation, and visualisations of comparative model performance across multiple classifier systems. Australia is home to over 800 bird species, approximately 45% of which are endemic to the continent. Global acoustic classification models are typically trained on Northern Hemisphere datasets and have not been rigorously evaluated against the full diversity of Australian species, particularly threatened and range-restricted endemics. Reliable automated identification is increasingly critical for passive acoustic monitoring programs supporting conservation planning, citizen science initiatives, and ecological impact assessments across Australia. The benchmark compares three classifiers on a curated set of Australian bird species: BirdNET v2.4 (Cornell Lab of Ornithology), Google Perch v2, and the Wild Echo AU-specific classifier trained exclusively on Australian acoustic data. Performance is evaluated on precision, recall, F1 score, and species coverage across taxonomic groups, habitat types, and conservation status categories. Key findings document substantial accuracy gaps in globally trained models for Australian endemic and threatened species, and demonstrate measurable improvements achieved through region-specific model training. Evaluation was conducted using field recordings representative of Australian soundscapes. Species lists and recording sets were drawn from publicly available sources and verified against current Australian taxonomic standards. The benchmark includes two evaluation batches with independent species sets, a v2 analysis incorporating additional threatened species, and a comprehensive summary report with supporting charts and statistical summaries. All materials are provided as a frozen snapshot as of 2026-05-26; see the associated GitHub repository for the latest version.