Computer Science ›› 2023, Vol. 50 ›› Issue (3): 216-222.doi: 10.11896/jsjkx.211100203

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Leaf Classification and Ranking Method Based on Multi-granularity Feature Fusion

LIU Songyue, WANG Huan   

  1. School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China
  • Received:2021-11-19 Revised:2022-05-31 Online:2023-03-15 Published:2023-03-15
  • About author:LIU Songyue,born in 1998,postgra-duate.His main research interests include computer vision and deep lear-ning.
    WANG Huan,born in 1982,Ph.D,associate professor,is a member of China Computer Federation.His research interests include pattern recognition,image processing,infrared target detection,object tracking,robot vision navigation and artificial intelligence.
  • Supported by:
    National Natural Science Foundation of China(61703209).

Abstract: Much work has long been devoted to plant leaf classification,but these methods cannot be applied well in real applications,though they may achieve good results in public datasets.Moreover,they are hardly employed to more complex problems,e.g.leaf ranking,which requires the classification of leaves first and then ranking leaves of the same class.This paper proposes a new model for plant leaf classification as well as leaf ranking,which focuses on the granularity information of leaves and integrates multi-level granularity from coarse to fine.Specifically,the model contains two branches,coarse-grained and fine-grained,which are linked by a coarse-fine hybrid loss,prompting the model to progressively learn a coarse-to-fine representation.A multi-step training approach is used,with different levels of features extracted at each step,therefore enabling the fusion of shallow features with deep features.In addition,a geometric channel attention module,which consists of a spatial transformation and a bili-near attention pooling module,is proposed to allow our model to focus on more discriminative local regions in the image and extract more discriminative features.Our method achieves 99.8% and 99.7% classification accuracy on two publicly available leaf classification datasets,Flavia leaf and Swedish leaf,respectively,and 71.9% classification accuracy on our constructed tobacco leaf ranking dataset,both outperform the state-of-the-art methods.

Key words: Leaf classification, Leaf ranking, Multi-granularity fusion, Spatial transformation network, Bilinear attention pooling

CLC Number: 

  • TP391.41
[1]WANG S,CHEN X,WANG Y,et al.Progressive AdversarialNetworks for Fine-Grained Domain Adaptation[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE,2020:9213-9222.
[2]DU R,CHANG D,BHUNIA A K,et al.Fine-Grained VisualClassification via Progressive Multi-Granularity Training of Jigsaw Patches[C]//European Conference on Computer Vision.Cham:Springer,2020:153-168.
[3]SHAH M P,SINGHA S,AWATE S P.Leaf classification using marginalized shape context and shape + texture dual-path deep convolutional neural network[C]//2017 IEEE International Conference on Image Processing.NewYork:IEEE,2017:860-864.
[4]LEE S H,CHAN C S,MAYO S J,et al.How deep learning extracts and learns leaf features for plant classification[J].Pattern Recognition,2017,71:1-13.
[5]LEE S H,CHAN C S,MAYO S J.Multi-organ plant classification based on convolutional and recurrent neural networks[J].IEEE Transactions on Image Processing,2018,27(9):4287-4301.
[6]QUACH B M,CUONG D V,PHAM N,et al.Leaf Recognition Using Convolutional Neural Networks Based Features[EB/OL].(2021-08-04) [2021-09-10].https://arxiv.org/abs/2108.01808.
[7]HAN B,ZENG S W.Plant Leaf Image Recognition Based onMulti-feature Integration and Convolutional Neural Network[J].Computer Science,2021,48(6A):113-117.
[8]DASARI S K,PRASAD V.A novel and proposed comprehen-sive methodology using deep convolutional neural networks for flue cured tobacco leaves classification[J].International Journal of Information Technology,2019,11(1):107-117.
[9]MARZAN C S,RUIZ C R.Automated tobacco grading usingimage processing techniques and a convolutional neural network[J].International Journal of Machine Learning and Computing,2019,9(6):807.
[10]LIN T Y,ROYCHOWDHURY A,MAJI S.Bilinear cnn models for fine-grained visual recognition[C]//Proceedings of the IEEE International Conference on Computer Vision.Washington DC:IEEE,2015:1449-1457.
[11]XU Y,WEN G,HU Y,et al.Multiple attentional pyramid networks for Chinese herbal recognition[J].Pattern Recognition,2021,110:107558.
[12]HE K,ZHANG X,REN S,et al.Residual Learning for ImageRecognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:770-778.
[13]SELVARAJU R R,COGSWELL M,DAS A,et al.Grad-CAM:Visual Explanations from Deep Networks via Gradient-based Localization[J].International Journal of Computer Vision,2020,128(2):336-359.
[14]JADERBERG M,SIMONYAN K,ZISSERMAN A.Spatialtransformer networks[C]//Advances in Neural Information Processing Systems.Quebec:MIT Press,2015:2017-2025.
[15]HU T,QI H,HUANG Q,et al.See better before looking clo-ser:Weakly supervised data augmentation network for fine-grained visual classification[EB/OL].(2019-01-26) [2019-03-23].https://arxiv.org/abs/1901.09891.
[16]WU S G,BAO F S,XU E Y,et al.A leaf recognition algorithm for plant classification using probabilistic neural network[C]//2007 IEEE International Symposium on Signal Processing and Information Technology.Giza:IEEE,2007:11-16.
[17]SÖDERKVIST O.Computer vision classification of leaves from Swedish trees[D].Linköping:Linköping University,2001.
[18]HU R,JIA W,LING H,et al.Multiscale distance matrix for fastplant leaf recognition[J].IEEE Transactions on Image Proces-sing,2012,21(11):4667-4672.
[19]YANG C,YU Q.Multiscale fourier descriptor based on triangular features for shape retrieval[J].Signal Process,2019,71:110-119.
[20]YANG C.Plant Leaf Recognition by Integrating Shape and Texture Features[J].Pattern Recognition,2021,112(1):107809.
[21]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[EB/OL].(2014-09-04) [2015-04-10].https://arxiv.org/abs/1409.1556.
[22]HE J,CHEN J N,LIU S,et al.TransFG:A Transformer Archi-tecture for Fine-grained Recognition[EB/OL].(2021-03-14) [2021-12-01].https://arxiv.org/abs/2103.07976.
[1] QIN Qi-qi, ZHANG Yue-qin, WANG Run-ze, ZHANG Ze-hua. Hierarchical Granulation Recommendation Method Based on Knowledge Graph [J]. Computer Science, 2022, 49(8): 64-69.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!