A Multimodal Dataset for Descriptive Research on Dunhuang Murals

Constructing the Dunhuang mural dataset is the cornerstone of this study, laying a solid data foundation for subsequent image description generation tasks. The entire process systematically encompasses four key steps: data acquisition, data augmentation, data annotation, and data partitioning. While pursuing data breadth and diversity, it also optimizes data quality and application potential through technical means.In the data acquisition phase, we innovatively integrated both online and offline methods. Online, we utilized web scraping technology to efficiently collect a large number of high-resolution mural images from sources such as the official website of the Dunhuang Research Academy, major art exhibition platforms, digital museums, and professional image libraries. Offline, we actively established collaborations with museums, art institutions, and researchers. After obtaining formal permissions, we employed high-resolution photography equipment and scanners to meticulously capture physical murals, ensuring high image quality and rich detail. These two approaches complement each other, jointly creating an image data pool with broad representativeness and high diversity.During the data augmentation stage, to expand the dataset's scale and enhance the model's generalization ability, we introduced a series of advanced image processing techniques. These included geometric transformations (such as rotation, mirroring, and scaling to simulate different perspectives), color adjustments (modifying brightness, contrast, and saturation to adapt to varying lighting conditions), random noise addition (mimicking interference factors in real-world photography), and image synthesis (integrating mural elements into different backgrounds or scenes). These techniques not only effectively simulated the various states in which murals might appear in the real world, significantly enriching the dataset's diversity, but also made the image data encountered during model training more aligned with real-world application scenarios, thereby improving its robustness.Data annotation, as the core component of dataset construction, directly impacts the performance of subsequent models. We specifically invited senior experts in the field of Dunhuang murals to provide professional and detailed textual descriptions for each mural image in the dataset. These descriptions were meticulous, covering not only the specific content, themes, and artistic style characteristics of the murals but also delving into the symbolic meanings of symbols and elements within them. For example, a description like "The flying celestial being on the left wears an Indian-style crown, with the left leg bent forward and the right leg extended backward, holding a scarf in the left hand and a flower tray in the right" accurately records the posture and details of the celestial being. To ensure annotation consistency and high quality, we developed detailed annotation guidelines and conducted multiple rounds of strict quality review and verification for all annotated results.Ultimately, through this complete and closed-loop process of data acquisition, augmentation, annotation, and partitioning, we successfully constructed a high-quality Dunhuang mural dataset. This dataset not only contains rich and diverse image information paired with precise textual descriptions but also provides a solid and reliable data foundation for the training, optimization, and evaluation of deep learning-based image description generation models