Experimental Synthesized Audio Samples for SGEAF

Speech synthesis has achieved human-like naturalness. However, generating speech with complex and mixed emotions remains a challenge. Discrete methods are constrained by their emotion representation paradigms, leading to unstructured or fragmented emotional feature spaces that fail to capture the spectrum of human emotion or the non-linear acoustic interactions. Continuous dimensional models, while structured, enforce mutual exclusivity between conflicting emotions, thereby struggling to represent emotions like bittersweet. This paper introduces score-guided emotion attention framework (SGEAF), a framework for controllable mixed-emotion speech synthesis that addresses these limitations. SGEAF constructs a compositional emotional feature space using a set of learnable emotion basis vectors, which represent fundamental, atomic emotion components. A multi-head attention module combines the emotion basis vectors, enabling control over mixed emotions. To imbue the emotion basis vectors with specific semantic meaning and ensure the coherence of the feature space, we propose a semantic alignment training strategy. The strategy leverages pre-computed semantic prototypes as stable regression targets, grounding the emotional feature space in perceptually meaningful semantics. Objective and subjective evaluations demonstrate that SGEAF outperforms baselines in synthesis quality and controllability. Acoustic analysis of synthesized speech demonstrates that the model captures non-linear acoustic interactions inherent in mixed emotions and enables control over single emotion intensity.