Cherry Tomato Detection for Harvesting Using Multimodal Perception and an Improved YOLOv7-Tiny Neural Network

Autor: Yingqi Cai, Bo Cui, Hong Deng, Zhi Zeng, Qicong Wang, Dajiang Lu, Yukang Cui, Yibin Tian
Jazyk: angličtina
Rok vydání: 2024
Předmět:
Zdroj: Agronomy, Vol 14, Iss 10, p 2320 (2024)
Druh dokumentu: article
ISSN: 2073-4395
DOI: 10.3390/agronomy14102320
Popis: Robotic fruit harvesting has great potential to revolutionize agriculture, but detecting cherry tomatoes in farming environments still faces challenges in accuracy and efficiency. To overcome the shortcomings of existing cherry tomato detection methods for harvesting, this study introduces a deep-learning-based cherry tomato detection scheme for robotic harvesting in greenhouses using multimodal RGB-D perception and an improved YOLOv7-tiny Cherry Tomato Detection (YOLOv7-tiny-CTD) network, which has been modified from the original YOLOv7-tiny by eliminating the “Objectness” output layer, introducing a new “Classness” method for the prediction box, and incorporating a new hybrid non-maximum suppression. Acquired RGB-D images undergo preprocessing such as color space transformation, point cloud normal vector angle computation, and multimodal regions of interest segmentation before being fed into the YOLOv7-tiny-CTD. The proposed method was tested using an AGV-based robot in a greenhouse cherry tomato farming facility. The results indicate that the multimodal perception and deep learning method improves detection precision and accuracy over existing methods while running in real time, and the robot achieved over 80% successful picking rates in two-trial mode in the greenhouse farm, showing promising potential for practical harvesting applications.
Databáze: Directory of Open Access Journals
Nepřihlášeným uživatelům se plný text nezobrazuje