Dhaka University Journal of Applied Science & Engineering

Improved Detection and Analysis of Wheat Spikes U sing Multi-Stage Convolutional Neural Network

• M. A. Batin
  Department of Robotics and Mechatronics Engineering, University Of Dhaka, Dhaka 1000, Bangladesh
• Muhaiminul Islam
  Department of Robotics and Mechatronics Engineering, University Of Dhaka, Dhaka 1000, Bangladesh
• Md Mehedi Hasan*
  Department of Robotics and Mechatronics Engineering, University Of Dhaka, Dhaka 1000, Bangladesh
• Stanley J. Miklavcic
  Phenomics and Bioinformatics Research Centre, University of South Australia, Adelaide 5095 AUS

Plant Phenotyping, Neural Network, Spike detection, Field imaging, Machine Learning

High throughput plant phenotyping is the advanced scientific approach for rapid phenotyping of plant traits, especially high consumable grains or crops, which is designed to process a high volume of data in a short time for plant breeders and cultivars to utilize. Detection and counting of crop traits such as plants, fruits, wheat or rice spikes, sorghum head, and plant diseases is more advanced research in this field, where real-world data are collected using aerial and land-based imaging platforms equipped with a variety of geospatial sensors, and their statistical analysis is conducted using Artificial Intelligence (AI) and Deep Learning-based solutions. In this paper, we contributed to solving such a challenge of phenotyping by detecting and counting wheat spikes from land-based imaging by applying a Region-based Convolutional Neural Network (CNN) model. Our method employs the use of CNN to extract features from the imaging platform and the learning model is trained to detect and count wheat spikes in field images based on these extracted features. Using the publicly available SPIKE dataset to train and test our model, our proposed method achieved 98% average precision and 91% average F1 score on the test set. Our results show a significant improvement of 2.9% and 11.2% in detection accuracy as well as 1% and 3% in average precision metric over state-of-the-art Faster Regionbased Convolutional Neural Network (Faster-RCNN), and RetinaNet, respectively, and have the potential to significantly benefit plant breeders by facilitating the selection of wheat varieties with high yields.

1. “The Big Idea – Phenotyping – Envision – College of Agriculture Magazine at Purdue University.” https://shorturl. at/flpwL, 2017. [Online; accessed 23-June-2022].
2. M. P. Cendrero-Mateo, O. Muller, H. Al- brecht, A. Burkart, S. Gatzke, B. Janssen, B. Keller, N. Ko¨rber, T. Kraska, S. Matsubara, J. Li, M. Mu¨ller-Linow, R. Pieruschka, F. Pinto, P. Rischbeck, A. Schickling, A. Steier, M. Watt, U. Schurr, and U. Rascher, “Field Phenotyping,” in Terrestrial Ecosystem Research Infrastructures (A. Chabbi and H. W. Loescher, eds.), pp. 53–81, Boca Raton, FL : CRC Press, 2017.: CRC Press, 1 ed., Mar. 2017.
3. M. M. Hasan, J. P. Chopin, H. Laga, and S. J. Miklavcic, “Detection and analysis of wheat spikes using Convolutional Neural Networks,” Plant Methods, vol. 14, p. 100, Dec. 2018.
4. T. Misra, A. Arora, S. Marwaha, V. Chinnusamy, A. R. Rao, R. Jain, R. N. Sahoo, M. Ray, S. Ku- mar, D. Raju, R. R. Jha, A. Nigam, and S. Goel, “SpikeSegNet-a deep learning approach utilizing encoder-decoder network with hourglass for spike segmentation and counting in wheat plant from visual imaging,” Plant Methods, vol. 16, p. 40, Dec. 2020.
5. H. Xiong, Z. Cao, H. Lu, S. Madec, L. Liu, and C. Shen, “TasselNetv2: in-field counting of wheat spikes with contextaugmented local regression net- works,” Plant Methods, vol. 15, p. 150, Dec. 2019.
6. R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich Feature Hierarchies for Accurate Object De- tection and Semantic Segmentation,” pp. 580–587, 2014.
7. C. Wen, J. Wu, H. Chen, H. Su, X. Chen, Z. Li, and C. Yang, “Wheat Spike Detection and Counting in the Field Based on SpikeRetinaNet,” Frontiers in Plant Science, vol. 13, p. 821717, Mar. 2022.
8. J. Zhao, X. Zhang, J. Yan, X. Qiu, X. Yao, Y. Tian, Y. Zhu, and W. Cao, “A Wheat Spike Detec- tion Method in UAV Images Based on Improved YOLOv5,” Remote Sensing, vol. 13, p. 3095, Aug. 2021
9. S. Khaki, N. Safaei, H. Pham, and L. Wang, “WheatNet: A lightweight convolutional neural net- work for highthroughput image-based wheat head detection and counting,” Neurocomputing, vol. 489, pp. 78–89, June 2022.
10. Z. Cai and N. Vasconcelos, “Cascade R-CNN: Delving Into High Quality Object Detection,” pp. 6154–6162, 2018.
11. T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dolla´r, “Microsoft COCO: Common Objects in Context,” Tech. Rep. arXiv:1405.0312, arXiv, Feb. 2015. arXiv:1405.0312 [cs] type: arti- cle.
12. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” in Advances in Neural Information Processing Systems, vol. 28, Curran Associates, Inc., 2015.
13. T.-Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariha- ran, and S. Belongie, “Feature Pyramid Networks for Object Detection,” pp. 2117–2125, 2017.
14. K. He, X. Zhang, S. Ren, and J. Sun, “Deep Resid- ual Learning for Image Recognition,” Dec. 2015. Number: arXiv:1512.03385 arXiv:1512.03385 [cs].
15. J. Pang, K. Chen, J. Shi, H. Feng, W. Ouyang, and D. Lin, “Libra R-CNN: Towards Balanced Learning for Object Detection,” pp. 821–830, 2019.
16. K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, and D. Lin, “MMDetection: Open MMLab Detection Toolbox and Benchmark,” arXiv:1906.07155 [cs, eess], June 2019. arXiv: 1906.07155