Overlapped fruit recognition for citrus harvesting robot in natural scenes

The overlapping of fruit is one of the main factors making automatic harvesting task a great challenge. A method based on convex hull was developed to achieve the localization and reconstruction of overlapped citrus fruits in natural scenes in order to guide the citrus harvesting robot more accurately. The central picking points and reconstruction radiuses were obtained due to the spherical shape of the citrus fruit. Firstly, the citrus fruit targets were detected based on the threshold segmentation method and basic morphological operations in HSV color space. Then, the overlapped citrus target was processed by subtracting it from its convex hull. Combining the minimum external affine transformation rectangle with distance transform method, the central points of each citrus fruits were determined. Finally, the reconstruction radiuses were determined by searching the gray scale gradient extremum along the connecting line of the central point of citrus fruits. The experimental results verify the effectiveness of the method, and it provides a basis for the path planning of fruit harvesting robot and the design of end-effector.