计算机科学 ›› 2023, Vol. 50 ›› Issue (3): 216-222.doi: 10.11896/jsjkx.211100203

• 计算机图形学&多媒体 • 上一篇    下一篇


刘松岳, 王欢   

  1. 南京理工大学计算机科学与工程学院 南京 210094
  • 收稿日期:2021-11-19 修回日期:2022-05-31 出版日期:2023-03-15 发布日期:2023-03-15
  • 通讯作者: 王欢(wanghuanphd@njust.edu.cn)
  • 作者简介:(syliusy811@163.com)
  • 基金资助:

Leaf Classification and Ranking Method Based on Multi-granularity Feature Fusion

LIU Songyue, WANG Huan   

  1. School of Computer Science and Engineering,Nanjing University of Science and Technology,Nanjing 210094,China
  • Received:2021-11-19 Revised:2022-05-31 Online:2023-03-15 Published:2023-03-15
  • About author:LIU Songyue,born in 1998,postgra-duate.His main research interests include computer vision and deep lear-ning.
    WANG Huan,born in 1982,Ph.D,associate professor,is a member of China Computer Federation.His research interests include pattern recognition,image processing,infrared target detection,object tracking,robot vision navigation and artificial intelligence.
  • Supported by:
    National Natural Science Foundation of China(61703209).

摘要: 长期以来,已有很多工作致力于研究植物叶片分类,虽然它们在公开数据集上表现较好,但实际应用并不理想,且难以应用于更复杂的问题,如叶片分级,即要求在对叶片进行分类的基础上,再对同一类的叶片进行更细级别(质量等级)的划分。为此,提出了一种新的植物叶片分类以及分级模型,该模型关注叶片的多粒度信息,并将粗粒度与细粒度进行有效融合。该模型包含粗粒度和细粒度两个分支,由粒度混合损失将两个分支联系起来,促使模型逐步学习由粗到细的粒度表征。采用了多步骤训练方式,每一步提取不同层级的特征,实现浅层特征与深层特征的融合。此外,还提出了几何通道注意力模块,该模块由空间变换和双线性注意力池化组成,使模型可以关注图像中更具区分度的局部区域,提取出的特征更具区分性。所提方法在Flavia leaf和Swedish leaf两个公开的叶片分类数据集上分别达到了99.8%和99.7%的分类准确率,且在所构建的烟叶分级数据集上达到了71.9%的分级准确率,均超过了目前最优的方法。

关键词: 叶片分类, 叶片分级, 多粒度融合, 空间变换网络, 双线性注意力池化

Abstract: Much work has long been devoted to plant leaf classification,but these methods cannot be applied well in real applications,though they may achieve good results in public datasets.Moreover,they are hardly employed to more complex problems,e.g.leaf ranking,which requires the classification of leaves first and then ranking leaves of the same class.This paper proposes a new model for plant leaf classification as well as leaf ranking,which focuses on the granularity information of leaves and integrates multi-level granularity from coarse to fine.Specifically,the model contains two branches,coarse-grained and fine-grained,which are linked by a coarse-fine hybrid loss,prompting the model to progressively learn a coarse-to-fine representation.A multi-step training approach is used,with different levels of features extracted at each step,therefore enabling the fusion of shallow features with deep features.In addition,a geometric channel attention module,which consists of a spatial transformation and a bili-near attention pooling module,is proposed to allow our model to focus on more discriminative local regions in the image and extract more discriminative features.Our method achieves 99.8% and 99.7% classification accuracy on two publicly available leaf classification datasets,Flavia leaf and Swedish leaf,respectively,and 71.9% classification accuracy on our constructed tobacco leaf ranking dataset,both outperform the state-of-the-art methods.

Key words: Leaf classification, Leaf ranking, Multi-granularity fusion, Spatial transformation network, Bilinear attention pooling


  • TP391.41
[1]WANG S,CHEN X,WANG Y,et al.Progressive AdversarialNetworks for Fine-Grained Domain Adaptation[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE,2020:9213-9222.
[2]DU R,CHANG D,BHUNIA A K,et al.Fine-Grained VisualClassification via Progressive Multi-Granularity Training of Jigsaw Patches[C]//European Conference on Computer Vision.Cham:Springer,2020:153-168.
[3]SHAH M P,SINGHA S,AWATE S P.Leaf classification using marginalized shape context and shape + texture dual-path deep convolutional neural network[C]//2017 IEEE International Conference on Image Processing.NewYork:IEEE,2017:860-864.
[4]LEE S H,CHAN C S,MAYO S J,et al.How deep learning extracts and learns leaf features for plant classification[J].Pattern Recognition,2017,71:1-13.
[5]LEE S H,CHAN C S,MAYO S J.Multi-organ plant classification based on convolutional and recurrent neural networks[J].IEEE Transactions on Image Processing,2018,27(9):4287-4301.
[6]QUACH B M,CUONG D V,PHAM N,et al.Leaf Recognition Using Convolutional Neural Networks Based Features[EB/OL].(2021-08-04) [2021-09-10].https://arxiv.org/abs/2108.01808.
[7]HAN B,ZENG S W.Plant Leaf Image Recognition Based onMulti-feature Integration and Convolutional Neural Network[J].Computer Science,2021,48(6A):113-117.
[8]DASARI S K,PRASAD V.A novel and proposed comprehen-sive methodology using deep convolutional neural networks for flue cured tobacco leaves classification[J].International Journal of Information Technology,2019,11(1):107-117.
[9]MARZAN C S,RUIZ C R.Automated tobacco grading usingimage processing techniques and a convolutional neural network[J].International Journal of Machine Learning and Computing,2019,9(6):807.
[10]LIN T Y,ROYCHOWDHURY A,MAJI S.Bilinear cnn models for fine-grained visual recognition[C]//Proceedings of the IEEE International Conference on Computer Vision.Washington DC:IEEE,2015:1449-1457.
[11]XU Y,WEN G,HU Y,et al.Multiple attentional pyramid networks for Chinese herbal recognition[J].Pattern Recognition,2021,110:107558.
[12]HE K,ZHANG X,REN S,et al.Residual Learning for ImageRecognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:770-778.
[13]SELVARAJU R R,COGSWELL M,DAS A,et al.Grad-CAM:Visual Explanations from Deep Networks via Gradient-based Localization[J].International Journal of Computer Vision,2020,128(2):336-359.
[14]JADERBERG M,SIMONYAN K,ZISSERMAN A.Spatialtransformer networks[C]//Advances in Neural Information Processing Systems.Quebec:MIT Press,2015:2017-2025.
[15]HU T,QI H,HUANG Q,et al.See better before looking clo-ser:Weakly supervised data augmentation network for fine-grained visual classification[EB/OL].(2019-01-26) [2019-03-23].https://arxiv.org/abs/1901.09891.
[16]WU S G,BAO F S,XU E Y,et al.A leaf recognition algorithm for plant classification using probabilistic neural network[C]//2007 IEEE International Symposium on Signal Processing and Information Technology.Giza:IEEE,2007:11-16.
[17]SÖDERKVIST O.Computer vision classification of leaves from Swedish trees[D].Linköping:Linköping University,2001.
[18]HU R,JIA W,LING H,et al.Multiscale distance matrix for fastplant leaf recognition[J].IEEE Transactions on Image Proces-sing,2012,21(11):4667-4672.
[19]YANG C,YU Q.Multiscale fourier descriptor based on triangular features for shape retrieval[J].Signal Process,2019,71:110-119.
[20]YANG C.Plant Leaf Recognition by Integrating Shape and Texture Features[J].Pattern Recognition,2021,112(1):107809.
[21]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[EB/OL].(2014-09-04) [2015-04-10].https://arxiv.org/abs/1409.1556.
[22]HE J,CHEN J N,LIU S,et al.TransFG:A Transformer Archi-tecture for Fine-grained Recognition[EB/OL].(2021-03-14) [2021-12-01].https://arxiv.org/abs/2103.07976.
[1] 秦琪琦, 张月琴, 王润泽, 张泽华.
Hierarchical Granulation Recommendation Method Based on Knowledge Graph
计算机科学, 2022, 49(8): 64-69. https://doi.org/10.11896/jsjkx.210600111
Full text



No Suggested Reading articles found!