An Autonomous Rover Counts Citrus Fruits with Ease, Thanks to OrangeYolo and OrangeSort
Designed to more accurately predict harvests, OrangeYolo and OrangeSort are laser-focused on spotting citrus fruits in video footage.
A team of scientists from the Beijing University of Technology, Chinese Academy of Agricultural Sciences, and the University of Tokyo has published details of two new algorithms designed specifically for detecting and tracking citrus fruit: OrangeYolo and OrangeSort.
"Fruit yield estimation is crucial for establishing fruit harvest and marketing strategies," the team explains. "Recently, computer vision and deep learning techniques have been used to estimate citrus fruit yield and have exhibited notable fruit detection ability. However, computer-vision- based citrus fruit counting has two key limitations: Inconsistent fruit detection accuracy and double-counting of the same fruit."
To fix that, the team has come up with two deep-learning approaches: OrangeYolo, based in part on the YOLOv3 algorithm from which it borrows part of its name, is optimized for detecting small citrus fruits at multiple scales; OrangeSort, meanwhile, uses specified tracking regions and a motion displacement estimation system to avoid double-counting when attempting to tally the number of fruits in a field.
To prove the system's capabilities, the team deployed it in-the-field using a custom autonomous rover built around the Robot Operating System (ROS) and equipped with an off-the-shelf DJI Osmo action camera for video capture. Once the rover had traversed the orchard, the fruit detection and tracking systems were fed the video on a desktop system with an Intel Core i7 processor and NVIDIA GeForce GTX 1080 Ti GPU.
"Using six video sequences taken from two fields containing 22 trees as the validation dataset," the team explains, "the proposed method showed the best performance (MAE [Mean Absolute Error] = 0.081, SD [Standard Deviation] = 0.08) relative to video- based manual counting. These results demonstrate the practical value of the proposed method compared with other existing algorithms.
"Future work can be aimed at using 3D technology to locate fruit spatial coordinates to enable more accurate counting," the team adds, "and line tail turns will be explored further in subsequent research work. Furthermore, a lightweight network model on edge devices can be designed and deployed to accomplish lightweight real-time field fruit counting and to link the actual fruit number by considering the unseen part of the tree to achieve actual yield estimation."
The team's work has been published in the journal Horticulture Research under open-access terms.