Viewpoints analysis for sweet peppers maturity classification

‘Photocell’ dataset 97 red sweet peppers (cultivar: Banji; seed company: Efal) and 102 yellow sweet peppers (cultivar: Liri; Seed company: Hazera) from maturity classes 1–4 were harvested from a commercial greenhouse in Kmehin, Israel in January 2019. Each pepper was manually placed inside a photocell, to ensure uniform illumination; each of the four photocell sides included three light-emitting diodes (LED) spots of 35 watts each [46], resulting in total illumination of 49 lux. Images were acquired using an IDS Ui-5250RE RGB color camera with a resolution of 1600 x 1200 pixels, placed 38 cm above the black cell floor. Images of each pepper were acquired from four viewpoints: three from the sides of the peppers, taken in no particular order, and the bottom viewpoint of the pepper. ‘Robotic’ dataset 69 red and 70 yellow peppers from maturity classes 2, 3, and 4 were harvested from the same commercial greenhouse in Kmehin, Israel, in November 2019. No peppers were collected from class 1 since we assumed that immature peppers that are entirely green would not be detected by a harvesting robot. Each pepper was individually attached to a pepper plant at a random orientation in a lab environment without controlled illumination. The peppers were attached to the plant hanging down straight in a way that does not create occlusion by leaves or stems and prevents overlap between peppers. Images for each pepper were acquired from three side viewpoints from the same height. No pictures of the bottom viewpoint were taken since it is not always feasible for harvesting robots due to plant parts that prevent the robot from reaching the pepper from the bottom. The images were acquired using an Intel RealSense D435 RGB-D (color + depth) camera mounted on a Sawyer robotic 7-degree-of-freedom torque-controlled arm from Rethink Robotics. Automatic exposure and white-balance were enabled. The use of the robotic arm enables images from the three viewpoints to be made from the same pose for all peppers. The dataset, which is published as part of this paper, resulted in 417 RGB-D images of the different sides of the peppers and the whole pepper ground truth information of their maturity class. ‘Orientation’ dataset As part of the ‘robotic’ dataset collection, 14 red peppers and 14 yellow peppers from maturity classes 2 and 3 (seven from each of class) were taken from the peppers harvested for the robotic dataset. Each pepper was placed hanging straight down in a random orientation on a pepper plant in the lab. Images from three viewpoints were acquired for each pepper three times, and each time the pepper was twisted 120˚ clockwise around the z-axis, resulting in a different surface of the pepper facing the camera from the initial viewpoint. This collection produced a dataset of 252 RGB images.'光电池（Photocell）'数据集：2019年1月于以色列克梅欣（Kmehin）的一处商业温室中，采收了成熟度等级1至4的97个红色甜椒（品种：Banji；种子公司：Efal）与102个黄色甜椒（品种：Liri；种子公司：Hazera）。将每个甜椒手动放置于光电池舱内以保证光照均匀：该光电池舱的四个侧面各配有3个功率为35瓦的发光二极管（light-emitting diodes，LED）灯珠[46]，总光照强度为49勒克斯（lux）。采用分辨率为1600×1200像素的IDS Ui-5250RE RGB颜色相机采集图像，相机安装于黑色舱底上方38厘米处。每个甜椒从四个视角采集图像：三个无特定顺序的侧面视角，以及一个底部视角。 '机械臂（Robotic）'数据集：2019年11月于同一处以色列克梅欣商业温室中，采收了成熟度等级2、3、4的69个红色甜椒与70个黄色甜椒。未采集等级1的甜椒，因假设完全呈绿色的未成熟甜椒无法被采收机器人识别。将每个甜椒以随机朝向单独固定于实验室环境中的甜椒植株上，未设置受控光照条件。甜椒以垂直下垂的方式固定，避免叶片或茎秆造成遮挡，同时防止甜椒间相互重叠。从相同高度的三个侧面视角采集每个甜椒的图像，未采集底部视角，原因是采收机器人受植株结构限制，难以从底部拍摄甜椒。采用安装于Rethink Robotics公司Sawyer七自由度力矩控制机械臂上的Intel RealSense D435 RGB-D（彩色+深度）相机采集图像，启用自动曝光与自动白平衡功能。借助机械臂可保证所有甜椒的三个视角拍摄姿态一致。本数据集作为本文的一部分公开，共包含417张甜椒不同侧面的RGB-D图像，以及对应甜椒成熟度等级的完整真值标注信息。 '朝向（Orientation）'数据集：作为'机械臂'数据集采集工作的一部分，从用于构建机械臂数据集的采收甜椒中选取了成熟度等级2和3的14个红色甜椒与14个黄色甜椒（每个等级各7个）。将每个甜椒以随机朝向垂直下垂固定于实验室环境中的甜椒植株上，对每个甜椒从三个视角各采集一次图像，每次采集前将甜椒绕z轴顺时针旋转120°，从而使初始视角下朝向相机的甜椒表面各不相同。本次采集共生成252张RGB图像。