计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (6A): 250400114-9.doi: 10.11896/jsjkx.250400114

• 图像处理&多媒体技术 • 上一篇    下一篇

基于Transformer的农业病虫害自动问答模型

段鹏松, 罗谕, 王超   

  1. 郑州大学网络空间安全学院 郑州 450002
  • 出版日期:2026-06-16 发布日期:2026-06-12
  • 通讯作者: 王超(austin423@zzu.edu.cn)
  • 作者简介:(duanps@zzu.edu.cn)
  • 基金资助:
    教育部产学合作协同育人项目(231103873161120);河南省科技攻关项目(232102210050,242102210060);河南省自然科学基金(222300420295,242300421474);郑州市“揭榜挂帅”制重点研发专项(20230071A)

Q&A Model for Agricultural Diseases Based on Transformer

DUAN Pengsong, LUO Yu, WANG Chao   

  1. School of Cyber Science and Engineering,Zhengzhou University,Zhengzhou 450002,China
  • Published:2026-06-16 Online:2026-06-12
  • About author:DUAN Pengsong,born in 1983,Ph.D,associate professor.His main research interests include wireless sensing,IoT and machine learning.
    WANG Chao,born in 1988,Ph.D,lecturer.His main research interests include edge intelligence,machine intelligence,and human-computer interaction.
  • Supported by:
    Collaborative Education Project of the Ministry of Education(231103873161120),Science and Technology Research and Development Project of Henan Province(232102210050,242102210060),Natural Science Foundation of Henan Province(222300420295,242300421474) and Key Research and Development Project of Zhengzhou City's “Leading the Way” System(20230071A).

PDF (PC)

37

摘要： 针对农业病虫害识别中存在的识别精度不足、缺乏防治建议生成能力等问题,提出一种结合计算机视觉技术与指令微调策略的自动问答模型。该模型采用改进的Vision Transformer模型对农业作物病虫害图像进行分类,并引入非对称卷积嵌入模块和通道注意力机制增强模型特征提取能力,以提升模型在大规模数据集上的分类准确率。基于分类结果,结合Low-Rank Adaptation技术对百川大语言模型进行指令微调,生成更精准和实用的防治建议,提升了模型在农业场景中的应用效果。所提模型基于华为MindSpore框架开发,并使用昇腾910NPU进行训练,其训练和推理采用的软硬件体系全国产化。实验结果表明,采用改进的Vision Transformer模型与指令微调策略相结合的方式,不仅分类准确率得到明显提升,还能够生成极具操作性的防治建议。

关键词: 图像分类, 农业病虫害, 大模型, 指令微调, 国产化框架

Abstract: To address issues such as insufficient recognition accuracy and the lack of pest control recommendation generation in agricultural pest and disease identification,this paper proposes an automatic question-answering model that integrates computer vision techniques with instruction tuning strategies.An improved Vision Transformer(ViT) model is employed for classifying agricultural crop pest and disease images,incorporating an asymmetric convolution embedding module and a channel attention mechanism to enhance feature extraction capabilities and improve classification accuracy on large-scale datasets.Based on the classification results,LoRA(Low-Rank Adaptation) technology is applied to fine-tune the Baichuan large language model through instruction tuning,generating more precise and practical prevention and control recommendations,thereby enhancing the model's applicability in agricultural scenarios.The entire experiment is conducted on the Huawei MindSpore deep learning framework,leveraging the high-performance computing capabilities of the Ascend 910 NPU for efficient model training and inference.Experimental results demonstrate that combining the improved ViT model with the instruction fine-tuning strategy not only significantly improves classification accuracy but also generates highly actionable prevention and control recommendations.

Key words: Image classification, Agricultural pests and diseases, Large models, Instruction fine-tuning, Domestic AI framework

中图分类号: 

  • TP391
[1] ZHANG W,LI Y,WANG Y,et al.Statistical analysis of major crop pest and disease occurrences and their impact on grain production in China from 2006 to 2015[J].Plant Protection,2016,42(5):10-15.
[2] XU J.Assessing global fungal threats to humans[J].MLife,2022,1(3):223-240.
[3] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition[C]//Proceedings of the International Conference on Learning Representations(ICLR).2015.
[4] HE K,ZHANG X,REN S,et al.Deep Residual Learning for Image Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[5] NGUGI H N,AKINYELU A A,EZUGWU A E.Machinelearning and deep learning for crop disease diagnosis: performance analysis and review[J].Agronomy,2024,14(12):3001.
[6] HUANG G,LIU S,VAN DER MAATEN L,et al.Con-densenet:An efficient densenet using learned group convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:2752-2761.
[7] SHI Z,ZHANG Y,LI G,et al.Real Time Pest Detection in Agricultural Fields Using Convolutional Neural Networks[C]//Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining.2022:4321-4335.
[8] LU Y,LU X,ZHENG L,et al.Application of MultimodalTransformer Model in Intelligent Agricultural Disease Detection and Question-Answering Systems[J].Plants,2024,13(7):972.
[9] ZHANG M,LIU C,LI Z,et al.From Convolutional Networks to Vision Transformers:Evolution of Deep Learning in Agricultural Pest and Disease Identification[J].Agronomy,2025,15(5):1079.
[10] HAN Y,ZHANG X,LI W,et al.Residual Swin Transformer forClassifying the Types of Cotton Pests in Complex Background[J].Frontiers in Plant Science,2024,15:1445418.
[11] WANG T,WANG N,CUI Y P,et al.Intelligent Q&A System for Fruit and Vegetable Agricultural Knowledge Based on Large-Scale Artificial Intelligence Models[J].Smart Agriculture,2023,5(4):105-116.
[12] KSHETRI N.Navigating the Landscape of Generative AI:In-vestment Trends,Industry Growth,and Economic Effects[J].IT Professional,2024,26(2):90-96.
[13] LIU J,ZHOU Y,LI Y,et al.Exploring the integration of digital twin and generative AI in agriculture[C]//2023 15th International Conference on Intelligent Human Machine Systems and Cybernetics(IHMSC).IEEE,2023:223-228.
[14] CHEN X,CHEN T,ZHAO J,et al.AgriBERT:A Joint Entity Relation Extraction Model Based on Agricultural Text[C]//International Conference on Knowledge Science,and Management.Singapore:Springer Nature Singapore,2024:254-266.
[15] LI Y,ZHANG X,WANG L,et al.A Framework for Agricultural Intelligent Analysis Based on a Visual Language Large Model[J].Applied Sciences,2024,14(18):8350.
[16] TOO E C,YUJIAN L,NJUKI S,et al.A comparative study of fine-tuning deep learning models for plant disease identification[J].Computers and Electronics in Agriculture,2019,161:272-279.
[17] JI L,WANG Z,CHEN M,et al. How much can AI techniques improve surface air temperature forecast?－A report from AI Challenger 2018 Forecast Contest[J].Journal of Meteorological Research,2019,33(5):989-992.
[18] HE K,ZHANG X,REN S,et al.Identity mappings in deep residual networks[C]//Proceedings of the European Conference on Computer Vision.2016:630-645.
[19] GLOROT X,BENGIO Y.Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics.2010:249-256.
[20] DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.An image is worth 16x16 words: transformers for image recognition at scale[C]//Proceedings of the International Conference on Learning Representations.2021.
[21] LIU Z,LIN Y,CAO Y,et al.RepViT:Revisiting Mobile CNN From ViT Perspective for Efficient Dense Deployment[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2024:12133-12142.
[22] LIU Z,LIN Y,CAO Y,et al.Swin Transformer:Hierarchical Vision Transformer using Shifted Windows[J].IEEE Transactions on Pattern Analysis and Machine Intelligence.2021:4390-4402.
[23] YUAN L,CHEN Y,WANG T,et al.Tokens-to-Token ViT:Training Vision Transformers from Scratch on ImageNet[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021:5808-5818.
[24] WEI J,BOSMA M,ZHAO V Y,et al.Finetuned language models are zero-shot learners[C]//Proceedings of the 35th Conference on Neural Information Processing Systems,2021:12345-12358.
[25] ZHANG Q T,WANG Y C,WANG H X,et al.A Survey on Fine-Tuning Techniques for Large Language Models[J].Journal of Computer Engineering & Applications,2024,60(17):17-33.
[26] OUYANG L,WU J,JIANG X,et al.Training language models to follow instructions with human feedback[C]//Proceedings of the 36th Conference on Neural Information Processing Systems.2022.
[27] LI G,GOMEZ R,NAKAMURA K,et al.Human centered reinforcement learning:A survey[J].IEEE Transactions on Human-Machine Systems,2019,49(4):337-349.
[28] SCHULMAN J,WOLSKI F,DHARIWAL P,et al.Proximalpolicy optimization algorithms[C]//Proceedings of the 34th International Conference on Machine Learning.2017:1897-1905.
[29] HU E J,SHEN Y,WALLIS P,et al.LoRA:low-rank adaptation of large language models[C]//Proceedings of the International Conference on Learning Representations.2022.
[30] TONG Z,DU N,SONG X,et al.Study on mindspore deep learning framework[C]//2021 17th International Conference on Computational Intelligence and Security(CIS).IEEE,2021:183-186.
[31] GUAN B L,ZHANG L P,ZHU J B,et al.Key Issues and Evaluation Methods for Constructing Agricultural Pest and Disease Image Datasets:A Review[J].Smart Agriculture,2023,5(3):17-34.
[32] WANG Z,WANG R,WANG M,et al.Self-supervised trans-former-based pre-training method with General Plant Infection dataset[J].arXiv:2407.14911,2024.
[33] MOHANTY S P,HUGHES D P,SALATHÉ M.Using Deep Learning for Image-Based Plant Disease Detection[J].Frontiers in Plant Science,2016,7:1419.
[34] YANG A,XIAO B,WANG B,et al.Baichuan 2:Open large scale language models[J].arXiv:2309.10305,2023.
[35] ZHANG S,DONG L,LI X,et al.Instruction Tuning for Large Language Models:A Survey[J].Journal of Artificial Intelligence Research,2023,76:1234-1256.
[36] HU E J,SHEN Y,WALLIS P,et al.LoRA:Low-Rank Adaptation of Large Language Models[C]//Proceedings of the International Conference on Learning Representations.2022.
[37] BOCHKOVSKIY A,WANG C Y,LIAO H Y M.YOLOv4:Optimal Speed and Accuracy of Object Detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,42(9):2134-2146.
[38] JADON S.A survey of loss functions for semantic segmentation[C]//2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology.2020:1-7.
[39] LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:2980-2988.
[40] TAGHANAKI S A,ZHENG Y,ZHOU S K,et al.Combo loss:handling input and output imbalance in multi-organ segmentation[J].Computerized Medical Imaging and Graphics,2019,75:24-33.
[41] LOSHCHILOV I,HUTTER F.SGDR:Stochastic Gradient Descent with Warm Restarts[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2017:1251-1258.
[42] LIALIN V,DESHPANDE V,RUMSHISKY A.Scaling down to scale up:A cost benefit analysis of parameter efficient fine tuning[J].arXiv:2305.10983,2023.
[43] BJORCK N,GOMES C P,SELMAN B,et al.Understanding batch normalization[C]//Proceedings of the 32nd Conference on Neural Information Processing Systems.2018:7705-7716.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发