TextoMorph

Tumor synthesis can generate examples that AI models often miss or over-detect, thereby improving AI performance by training on these challenging cases. However, existing synthesis methods are typically unconditional\u2014generating images from random variables\u2014or conditioned only on tumor shapes, which limits their controllability over specific tumor characteristics such as texture, heterogeneity, boundaries, and pathology type. As a result, the generated tumors may be overly similar to or even duplicates of existing training data, failing to effectively address AI\u2019s weaknesses. To overcome these limitations, we propose a new text-driven tumor synthesis approach, termed TextoMorph, that enables textual control over tumor characteristics in conjunction with mask-based control. This approach is particularly beneficial for examples that are most challenging for AI, including early tumor detection (increasing Sensitivity by +8.5%), tumor segmentation for precise radiotherapy (increasing DSC by +6.3%), and classification between benign and malignant tumors (improving Sensitivity by +8.2%). By incorporating text mined from radiology reports into the synthesis process, we increase both the variability and controllability of synthetic tumors, allowing us to more precisely target AI\u2019s failure cases. Moreover, TextoMorph utilizes contrastive learning across different texts and CT scans, significantly reducing dependence on scarce image-report pairs (only 141 pairs are used in this study) by leveraging a large corpus of 34,035 radiology reports. Finally, we have developed rigorous evaluation protocols to assess the realism and diversity of our synthetic tumors, demonstrating that they are realistic and diverse in texture, heterogeneity, boundaries, and pathology. Code and models are available at https:\/\/github.com\/MrGiovanni\/TextoMorph.