Details

Autor(en) / Beteiligte

• Li, Tengfei
• Fang, Wentai
• Zhao, Guanao
• Gao, Fangfang
• Wu, Zhenchao
• Li, Rui
• Fu, Longsheng
• Dhupia, Jaspreet

Titel

An improved binocular localization method for apple based on fruit detection using deep learning

Ist Teil von

• Information processing in agriculture, 2023-06, Vol.10 (2), p.276-287

Ort / Verlag

Elsevier B.V

Erscheinungsjahr

2023

Link zum Volltext

Quelle

Alma/SFX Local Collection

Beschreibungen/Notizen

• •Deep learning is applied to improve fruit detection for binocular localization of apples.•Three segmentation methods were developed to segment fruit in the bounding box of detected fruit.•Template matching with parallel polar line constraint was used to match apples in left and right images.•Average standard deviation and average localization precision of apples were 0.51 cm and 99.64%, respectively. Apple picking robot is now being developed as an alternative to hand picking due to a great demand for labor during apple harvest season. Accurate detection and localization of target fruit is necessary for robotic apple picking. Detection accuracy has a great influence on localization results. Although current researches on detection and localization of apples using traditional image algorithms can obtain good results under laboratory conditions, it is difficult to accurately detect and locate objects in natural field with complex environments. With the rapid development of artificial intelligence, accuracy of apple detection based on deep learning has been significantly improved. Therefore, a deep learning-based method was developed to accurately detect and locate the position of fruit. For different localization methods, binocular localization is a widely used localization method for its bionic principle and lower equipment cost. Hence, this paper proposed an improved binocular localization method for apple based on fruit detection using deep learning. First, apples of binocular images were detected by Faster R-CNN. After that, a segmentation based on chromatic aberration and chromatic aberration ratio was applied to segment apple and background pixels in bounding box of detected fruit. Furthermore, template matching with parallel polar line constraint was used to match apples in left and right images. Finally, two feature points on apples were selected to directly calculate three dimensional coordinates of feature points with the binocular localization principle. In this study, Faster R-CNN achieved an AP of 88.12% with an average detection speed of 0.32 s for an image. Meanwhile, standard deviation and localization precision of depth of two feature points on each apple were calculated to evaluate localization. Results showed that the average standard deviation and the average localization precision of 76 groups of datasets were 0.51 cm and 99.64%, respectively. Results indicated that the proposed improved binocular localization method is promising for fruit localization.