Viewpoints analysis for sweet peppers maturity classification

‘Photocell’ dataset 97 red sweet peppers (cultivar: Banji; seed company: Efal) and 102 yellow sweet peppers (cultivar: Liri; Seed company: Hazera) from maturity classes 1–4 were harvested from a commercial greenhouse in Kmehin, Israel in January 2019. Each pepper was manually placed inside a photocell, to ensure uniform illumination; each of the four photocell sides included three light-emitting diodes (LED) spots of 35 watts each [46], resulting in total illumination of 49 lux. Images were acquired using an IDS Ui-5250RE RGB color camera with a resolution of 1600 x 1200 pixels, placed 38 cm above the black cell floor. Images of each pepper were acquired from four viewpoints: three from the sides of the peppers, taken in no particular order, and the bottom viewpoint of the pepper. ‘Robotic’ dataset 69 red and 70 yellow peppers from maturity classes 2, 3, and 4 were harvested from the same commercial greenhouse in Kmehin, Israel, in November 2019. No peppers were collected from class 1 since we assumed that immature peppers that are entirely green would not be detected by a harvesting robot. Each pepper was individually attached to a pepper plant at a random orientation in a lab environment without controlled illumination. The peppers were attached to the plant hanging down straight in a way that does not create occlusion by leaves or stems and prevents overlap between peppers. Images for each pepper were acquired from three side viewpoints from the same height. No pictures of the bottom viewpoint were taken since it is not always feasible for harvesting robots due to plant parts that prevent the robot from reaching the pepper from the bottom. The images were acquired using an Intel RealSense D435 RGB-D (color + depth) camera mounted on a Sawyer robotic 7-degree-of-freedom torque-controlled arm from Rethink Robotics. Automatic exposure and white-balance were enabled. The use of the robotic arm enables images from the three viewpoints to be made from the same pose for all peppers. The dataset, which is published as part of this paper, resulted in 417 RGB-D images of the different sides of the peppers and the whole pepper ground truth information of their maturity class. ‘Orientation’ dataset As part of the ‘robotic’ dataset collection, 14 red peppers and 14 yellow peppers from maturity classes 2 and 3 (seven from each of class) were taken from the peppers harvested for the robotic dataset. Each pepper was placed hanging straight down in a random orientation on a pepper plant in the lab. Images from three viewpoints were acquired for each pepper three times, and each time the pepper was twisted 120˚ clockwise around the z-axis, resulting in a different surface of the pepper facing the camera from the initial viewpoint. This collection produced a dataset of 252 RGB images.‘光电池箱’数据集（Photocell dataset） 2019年1月，于以色列克梅辛（Kmehin）的一处商业温室中，采集了成熟度等级1至4级的红甜椒97个（品种：Banji；种子供应商：Efal）与黄甜椒102个（品种：Liri；种子供应商：Hazera）。将每个甜椒手动放置于光电池箱内以确保光照均匀；该箱体的四个侧面各设有3个功率为35瓦的发光二极管（LED）灯珠[46]，总光照强度为49勒克斯（lux）。采用分辨率为1600×1200像素的IDS Ui-5250RE RGB彩色相机采集图像，相机安装于黑色箱体底板上方38厘米处。每个甜椒从四个视角采集图像：三个无特定顺序的侧面视角，以及一个底部视角。 ‘机械臂’数据集（Robotic dataset） 2019年11月，于同一处以色列克梅辛商业温室中，采集了成熟度等级2、3、4级的红甜椒69个与黄甜椒70个。未采集等级1的甜椒，因研究团队假设完全呈绿色的未成熟甜椒无法被采收机器人识别。在实验室无受控光照的环境中，将每个甜椒以随机朝向单独固定于甜椒植株上；甜椒以垂直下垂的方式固定于植株，避免被叶片或茎秆遮挡，同时防止甜椒之间相互重叠。每个甜椒从三个相同高度的侧面视角采集图像，未采集底部视角的图像，因采收机器人常受植株结构阻碍，无法从底部拍摄甜椒。采用安装于Rethink Robotics公司生产的Sawyer七自由度力矩控制机械臂上的Intel RealSense D435 RGB-D（彩色+深度）相机采集图像，启用自动曝光与自动白平衡功能。借助机械臂可确保所有甜椒的三个视角均采用完全一致的拍摄姿态。本论文发表的该数据集共包含417张甜椒不同侧面的RGB-D图像，以及所有甜椒成熟度等级的完整标注真值（ground truth）信息。 ‘朝向’数据集（Orientation dataset） 作为‘机械臂’数据集采集流程的一部分，从该数据集的采集样本中选取成熟度等级2与3级的甜椒各7个（红、黄甜椒各14个）。在实验室环境中，将每个甜椒以随机朝向垂直下垂固定于甜椒植株上。为每个甜椒分三次从三个视角采集图像，每次采集前将甜椒绕z轴顺时针旋转120°，使初始视角下朝向相机的甜椒表面发生变化。本次采集共生成包含252张RGB图像的数据集。