All or nothing: highly variable species-level performance of BirdNET’s bird classifier and a new Australian frog classifier

Effective conservation relies on accurate data on species distributions and their change over time. Recent advances in low-cost acoustic recorders and artificial intelligence (AI) now offer powerful new ways to collect these data, revitalising the use of ecoacoustics for monitoring fauna at scale. While open-source, multi-species AI models like BirdNET are now available for most bird species, similar tools for amphibians remain limited. Here, we present the first open-source, multi-species frog classifier for 16 native species from Victoria, Australia, and evaluate its performance alongside the bird classifier BirdNET using real-world data from freshwater wetlands. Both classifiers exhibited a characteristic bimodal distribution in precision, with most species detected either with very high (>90%) or very low (<25%) accuracy, and relatively few species in between. For our frog classifier, misclassifications were often associated with simultaneous calls from acoustically similar frog species. For the default bird classifier, precision tended to be higher for species with short, simple, downward-sloping calls. Despite being trained on a relatively small, targeted dataset, our frog classifier achieved low false-positive rates, suggesting that modest datasets can yield robust results. These classifiers are currently most effective for rapidly screening long recordings, allowing users to focus on segments most likely to contain relevant calls while maintaining low false-positive rates. To support further development and collaboration, we release our frog classifier as an open-access tool. Broader efforts to expand multi-species classifiers across taxa and regions will be critical to realising the full potential of AI-powered ecoacoustics in biodiversity monitoring.