Vocal complexity constrains the dear enemy effect: a comparative study of coal tits and green-backed tits

This study was conducted in the Liupanshan National Nature Reserve (35°15′－35°41′ N, 106°09′－106°30′ E), Ningxia Hui Autonomous Region, northwestern China. The reserve covers approximately 67,800 hectares and features an elevation range of 1,935 to 2,371 meters above sea level with forest coverage exceeding 70%. The climate is mid-temperate, transitioning semi-humid to semi-arid, characterized by a mean annual temperature of 5.8℃ and average annual precipitation ranging of 600-800 mm. Vegetation is diverse, primarily consisting of secondary coniferous forest, broadleaf mixed forests, and meadow grasslands. Key tree species include the Chinese red pine (Pinus tabuliformis) and Asian white birch (Betula platyphylla var. japonica) (Shi et al. 2009).The “dear enemy effect”, wherein territorial animals reduce aggression toward familiar neighbors while maintaining heightened responses to strangers, is a widespread strategy to minimize energy expenditure on territory defense. However, its expression across species with varying vocal complexities remains poorly understood. This study investigated neighbor-stranger discrimination in two sympatric tit species exhibiting contrasts in song repertoire complexity: coal tits (Periparus ater) with large, complex repertoires (21 distinct song types) and green-backed tits (Parus monticolus) with limited, simple repertoires (4 song types). Using playback experiments, we assessed behavioral responses to songs of familiar neighbors versus strangers. Coal tits exhibited robust discrimination, responding with significantly greater aggression (e.g., increased flight frequency within 5-meter radius of the nest box and closer approach distance to the speaker) to strangers, aligning with the dear enemy effect. In contrast, green-backed tits showed minimal and undifferentiated responses to both songs of familiar neighbors and strangers, indicating a lack of discrimination. These results demonstrate that the dear enemy effect is not universal and that vocal repertoire complexity plays a critical role in shaping individual recognition abilities. Our study underscores the importance of bioacoustics constrains in territorial behaviors and highlights how simplified vocal signals may limit the expression of complex social behaviors such as neighbor-stranger discrimination.