Multimodal Terrestrial Cotton Seedling Dataset with Multiview RGB, RealSense Infrared and Depth, Thermal Imaging, and YOLO-Compatible Annotations for Agricultural Row Perception

This dataset contains multimodal terrestrial data of early-stage cotton seedlings collected under real open-field conditions in the Mexicali Valley, Baja California, Mexico. The data were acquired from a ground-level perspective and are intended to support research on crop-row perception, agricultural robotics, multimodal field sensing, row-center estimation, sensor association, and the development of perception methods for lightweight robotic platforms. Data collection was carried out along three field trajectories using an experimental terrestrial sensing platform. The sensing system included front, left, and right RGB cameras; an Intel RealSense D435i for color, depth, and left/right infrared acquisition; an MLX90640 thermal sensor; a GPS receiver; a 9-DOF inertial measurement unit; an optical-flow and ground-distance sensor; and environmental temperature and relative-humidity sensing. The central RGB camera, RealSense sensor, and thermal camera were oriented toward the direction of travel, while the lateral RGB cameras provided additional views of the surrounding crop rows. The recorded trajectories include several situations encountered during field operation, including row entry, row following, row ends, transitions between adjacent rows, sparse seedling emergence, weak row continuity, exposed soil, weed interference, shadows, changing illumination, partial occlusions, field objects, and areas with limited maneuvering space. These conditions provide data for studying crop-row perception under the visual variability found in early-stage cotton fields. The repository is organized by trajectory, modality, and processing level. It includes frontal and lateral RGB images and videos, RealSense color images, metric depth data and depth visualizations, left and right RealSense infrared images, RGB-depth composite images, thermal matrices and rendered thermal images, GPS records, inertial measurements, optical-flow and ground-distance measurements, environmental data, sensor time series, acquisition metadata, file inventories, quality-control reports, checksum files, documentation, and supporting code. The published dataset contains 151,092 files, including 149,769 image files, 306 videos, and 383 tabular files. It is distributed as three trajectory-based ZIP packages with a combined compressed size of approximately 41.9 GB. The organization allows each trajectory to be examined independently while preserving links among visual modalities, sensor records, metadata, and processed products. Multimodal records are associated using common capture-group identifiers and software-recorded timestamps. A total of 49 capture groups are documented across the three trajectories. These associations support temporal inspection and multimodal analysis; however, the sensing streams were not hardware synchronized and should therefore be interpreted according to the timestamps and metadata provided with the dataset. A manually annotated pilot subset is also included for crop-row perception. It contains 240 unique multimodal images, balanced across the three trajectories and four visual groups, and produces 339 image-label pairs with 1,678 annotated instances across the supported annotation tasks. The YOLO-compatible exports include segmentation, oriented bounding box (OBB), and pose-type structural annotations. Functional classes used in the segmentation and OBB annotations include crop_row, furrow, weed, CROP_ROW, and soil, depending on the corresponding task and modality. The pose annotation uses a single crop-row structural class and five ordered keypoints to represent the geometry of the visible row. Keypoint 1 identifies the beginning of the row closest to the observer or robotic platform, whereas keypoint 5 represents the farthest visible point along the row. Keypoints 2, 3, and 4 describe intermediate changes in row morphology and help represent straight and curved row structures. Together, the ordered points provide a continuous structural reference that can be used to estimate row direction and to maintain the row trajectory when part of the visual reference becomes weak or temporarily unavailable. In RGB-depth composite samples included in the pose subset, these structural keypoints are placed on the RGB portion of the composite, while the accompanying depth visualization provides additional geometric context. The Intel RealSense data include calibration and acquisition metadata, including intrinsic parameters, depth-to-color extrinsic information, and a documented depth scale of 0.001 m per depth unit. Raw metric depth information is distinguished from colorized depth visualizations intended for inspection. Likewise, the MLX90640 provides native 24 × 32 thermal measurements together with rendered thermal images; the rendered images should not be interpreted as the native spatial resolution of the thermal sensor. The repository also contains master file indexes, variable dictionaries, SHA-256 checksum information, modality-availability tables, image-label correspondence reports, annotation summaries, quality-control records, and software used for organization, inspection, post-processing, and dataset validation. Automated package inspection detected no corrupt or unreadable files in the published trajectory packages. The dataset can be reused for research on agricultural robotics, crop-row detection and tracking, row-center estimation, visual guidance and control, RGB-D analysis, infrared and thermal field inspection, multimodal perception, visual-inertial analysis, sensor fusion, temporal data association, local 3D reconstruction, semantic point-cloud exploration, dataset curation, and the preliminary development or evaluation of YOLO-compatible perception models for field-oriented robotic systems.