Supporting data for: Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics

The dataset includes six archived files, 2018_CIMMYT_YT_1.zip, 2018_CIMMYT_YT_2.zip, 2018_CIMMYT_YT_3.zip,2018_CIMMYT_YT_4.zip, 2018_CIMMYT_EYT.zip, and Genotypes.zip. The first five zipped files (YT_1 to YT_4 and EYT) zipped files contain all the phenomics data of the YT and EYT trials, while the last zipped file includes the genomics data. The "Table_Headers_Info.xlsx" file explains the headers in each tabular data files. 2018 YT dataset There are four sub-folders and a CSV file in this dataset. The “DEM” folder stores the Digital Elevation Model (DEM) images of each breeding plot in Geo-Tiff format cropped from the DEM of the entire EYT field trial. The image file name indicates the plot ID. The “Multispec” folder stores the reflectance orthorectified images in Geo-Tiff format cropped from orthorectified raw images captured by the MicaSence RedEdge camera. Due to its large scale, the YT field trials are separated into three sections, named “1-60”, “61-141”, and “142-320” respectively. The image acquisition at each section might be implemented on different dates. The section information and the date information can be revealed from the folder name. Image files saved in each folder stores reflectance information in five raster bands, including “B” (blue band), “G” (green band), “R” (red band), “RE” (red-edge band), and “NIR” (near infrared band). The first five segments of the file name separated by a hyphen indicate the plot ID, while the next four segments indicate the image acquisition date, time, and the raw image sequence number. The “Plot-level_VI” folder stores the numerical value of plot-level traits extracted from the images saved in the “Multispec” folder. Other than the reflectance trait values of the five separated spectral bands, there are three vegetation indices - “GNDVI”, “NDVI”, and “NDRE” also extracted. The “Thermal” folder stores the thermal orthorectified images in Geo-Tiff format cropped from orthorectified raw images captured by the FLIR VUE Pro R camera, as well as the numerical value of plot-level trait extracted from those images. The key file links the phenotype metadata with the genotype metadata in the CSV file. 2018 CIMMYT EYT dataset There are five sub-folders in this dataset, including processed images and extracted trait values on five different dates. In each -sub-folder, there is a “DEM_crop” folder, an “Or_crop” folder, and a “Traits_csv” folder. The “DEM_crop” folder stores the Digital Elevation Model (DEM) images of each breeding plot in Geo-Tiff format cropped from the DEM of the entire EYT field trial. The image file name indicates the plot ID. The “Or_crop” folder stores the reflectance orthorectified images in Geo-Tiff format cropped from orthorectified raw images captured by the MicaSence RedEdge camera. Image file names ended with “B” (blue band), “G” (green band), “R” (red band), “RE” (red-edge band), and “NIR” (near infrared band) record single-band reflectance information. Image file names ended with “GNDVI”, “NDVI”, and “NDRE” record three vegetation indices information. The first five segments of the file name separated by a hyphen indicate the plot ID, while the next four segments indicate the image acquisition date, time, and the raw image sequence number. The “Traits_csv” folder stores the numerical value of plot-level traits extracted from the images in the “DEM_crop” and the “Or_crop” folder. Genotype dataset There are two zipped files (.gz files) and two spreadsheets in this dataset. The two zipped files store the SNP data of all the wheat breeding lines planted from the year of 2013 to the year of 2020 in CIMMYT Mexico. The first spreadsheet (Eyt1718_Gmat_982lines.xlsx) stores the genetic matrix information of the 2018 EYT field trials, while the second spreadsheet (Fieldbook_18-OBR-EYTBW-B5I.csv) includes meta data of each EYT plot. Table_Headers_Info.xlsx The Excel spreadsheet explains the columns' names in each tabular data files. "Header" represents the header name. "Notes" explains what the header name stands for, included unit if necessary.