计算机科学 ›› 2022, Vol. 49 ›› Issue (8): 113-119.doi: 10.11896/jsjkx.210700153

• 计算机图形学& 多媒体 • 上一篇    下一篇

基于注意力机制的医学影像深度哈希检索算法

朱承璋1,2,3,4, 黄嘉儿1,3,4, 肖亚龙1,2, 王晗1,3,4, 邹北骥1,3,4   

  1. 1 中南大学计算机学院 长沙 4100832
    2 中南大学文学与新闻传播学院 长沙 410083
    3 “移动医疗”教育部-中国移动联合实验室 长沙 410083
    4 湖南省机器视觉与智慧医疗工程技术研究中心 长沙 410083
  • 收稿日期:2021-07-14 修回日期:2021-10-23 发布日期:2022-08-02
  • 通讯作者: 肖亚龙(ylxiao@csu.edu.cn)
  • 作者简介:(anandawork@126.com)
  • 基金资助:
    科技创新2030-“新一代人工智能”重大项目(2018AAA0102100);湖南省高新技术产业科技创新引领计划(2020GK2021)

Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism

ZHU Cheng-zhang1,2,3,4, HUANG Jia-er1,3,4, XIAO Ya-long1,2, WANG Han1,3,4, ZOU Bei-ji1,3,4   

  1. 1 School of Computer Science and Engineering,Central South University,Changsha 410083,China
    2 School of Literature and Journalism,Central South University,Changsha 410083,China
    3 Mobile Health Ministry of Education-China Mobile Joint Laboratory,Changsha 410083,China
    4 Hunan Engineering Research Center of Machine Vision and Intelligent Medicine,Changsha 410083,China
  • Received:2021-07-14 Revised:2021-10-23 Published:2022-08-02
  • About author:ZHU Cheng-zhang,born in 1978,Ph.D,associate professor,master supervisor.Her main research interests include pattern recognition,computer vision,and image processing.
    XIAO Ya-long,born in 1985,Ph.D,associate professor,is a member of China Computer Federation.His main research interests include wireless sensing and computational communication.
  • Supported by:
    National Key R & D Program of China(2018AAA0102100) and Hunan Province High-tech Industry Science and Technology Innovation Leading Program(2020GK2021).

摘要: 针对现阶段医学影像检索中检索性能差、精度低、缺乏可解释性等一系列问题,提出了一种结合了注意力机制的医学影像检索算法。以深度卷积神经网络为基础,以贝叶斯模型为框架,所提算法引入了由语义特征引导的注意力机制模块,通过分类网络的引导,生成包含语义信息的局部特征描述子,同时使用全局特征与富含语义信息的局部特征作为哈希网络的输入,引导哈希网络从全局和局部的角度关注重要特征区域,增强了哈希编码的特征表达能力,并引入加权似然估计函数解决了正负样本对数量不均衡的问题。采用MAP和NDCG作为评价指标,选择ChestX-ray14数据集进行实验,将所提算法与目前常用的深度哈希方法进行对比。实验结果表明,本文算法在哈希编码不同码位下的MAP值和NDCG值都远优于现有的深度哈希方法,证明了其有效性。

关键词: 贝叶斯框架, 卷积神经网络, 深度哈希, 医学影像检索, 注意力机制

Abstract: A medical image retrieval method combining attention mechanism is proposed for a series of problems such as poor retrieval performance,low accuracy and lack of interpretability in current medical image retrieval.Based on deep convolutional neural networks and taking Bayesian models as the framework,the proposed algorithm introduces an attention mechanism module guided by semantic features.Local feature descriptors containing semantic information are generated under the guidance of the classification network.Both global features and local features rich in semantic information are used as inputs to the hash network,which enhances the feature representation capability of hash coding by guiding the hash network to pay attention to important feature regions from both global and local perspectives.And the weighted likelihood estimation function is introduced to solve the problem of the unbalanced number of positive and negative sample pairs.MAP and NDCG are used as evaluation metrics,and the ChestX-ray14 dataset is selected for experiments.The proposed algorithm is compared with the current commonly used deep ha-shing methods.Experiment results show that the MAP and NDCG values are much better than the existing deep hashing methods at different code levels of hash coding,which proves the effectiveness of the proposed algorithm.

Key words: Attention mechanism, Bayesian framework, Convolutional neural networks, Deep hashing, Medical image retrieval

中图分类号: 

  • TP391
[1]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems.Red Hook,USA:Curran Asso-ciates Inc.,2012:1097-1105.
[2]RONNEBERGER O,FISCHER P,BROX T.U-Net:Convolu-tional networks for biomedical image segmentation[C]//2015 Medical Image Computing and Computer Assisted Intervention 2015.Cham,Switzerland:Springer,2015:234-241.
[3]LI Y,JIANG H N,WANG H B,et al.Generating GF-3 orthophoto image based on indirect rectification[J].Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition),2020,32(3):441-451.
[4]WANG J D,ZHANG T K,SONG J K,et al.A survey on lear-ning to hash[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(4):769-790.
[5]MNIH V,HEESS N,GRAVES A,et al.Recurrent models ofvisual attention[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge,USA:MIT Press,2014:2204-2212.
[6]HE K M,ZHANG X Y,REN S Q,et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,USA:IEEE Press,2016:770-778.
[7]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//2015 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,USA:IEEE Press,2015:3431-3440.
[8]XIA R K,PAN Y,LAI H J,et al.Supervised hashing for image retrieval via image representation learning[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2014:2156-2162.
[9]LAI H J,PAN Y,LIU Y,et al.Simultaneous feature learning and hash coding with deep neural networks[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,USA:IEEE Press,2015:3270-3278.
[10]ZHU H,LONG M S,WANG J M,et al.Deep hashing network for efficient similarity retrieval[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2016:2415-2421.
[11]CAO Z J,LONG M S,WANG J M,et al.HashNet:Deep lear-ning to hash by continuation[C]//2017 IEEE International Conference on Computer Vision.Piscataway,USA:IEEE Press,2017:5609-5618.
[12]CAO Y,LONG M S,LIU B,et al.Deep cauchy hashing for hamming space retrieval[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway,USA:IEEE Press,2018:1229-1237.
[13]YUAN L,WANG T,ZHANG X P,et al.Central similarityquantization for efficient image and video retrieval[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway,USA:IEEE Press,2020:3080-3089.
[14]WANG R K,WANG R P,QIAO S S,et al.Deep position-aware hashing for semantic continuous image retrieval[C]//2020 IEEE Winter Conference on Applications of Computer Vision.Piscataway,USA:IEEE Press,2020:2482-2491.
[15]CAI Y H,LI Y Y,QIU C Y,et al.Medical image retrieval based on convolutional neural network and supervised hashing[J].IEEE Access,2019,7:51877-51885.https://ieeexplore.ieee.org/document/8692349.
[16]PENG T Y,BOXBERG M,WEICHERT W,et al.Multi-task learning of a deep k-nearest neighbour network for histopathological image classification and retrieval[C]//2019 Medical Image Computing and Computer Assisted Intervention.Cham,Switzerland:Springer,2019:676-684.
[17]CHEN Z X,CAI R J,LU J W,et al.Order-sensitive deep ha-shing for multimorbidity medical image retrieval[C]//2018 Me-dical Image Computing and Computer Assisted Intervention.Cham,Switzerland:Springer,2018:620-628.
[18]ZHENG Y S,JIANG B N,SHI J,et al.Encoding histopathological WSIs using GNN for scalable diagnostically relevant regions retrieval[C]//2019 Medical Image Computing and Computer Assisted Intervention.Cham,Switzerland:Springer,2019:550-558.
[19]SILVA W,POELLINGER A,CARDOSO J S,et al.Interpre-tability-guided content-based medical image retrieval[C]//23th Medical Image Computing and Computer Assisted Intervention.Cham,Switzerland:Springer,2020:305-314.
[20]FANG J S,FU H Z,LIU J.Deep triplet hashing network for case-based medical image retrieval[J].Medical Image Analysis,2021,69:101981.https://www.sciencedirect.com/science/article/pii/S136184152100027X.
[21]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway,USA:IEEE Press,2018:7132-7141.
[22]WOO S,PARK J,LEE J,et al.CBAM:Convolutional block attention module[C]//Proceedings of the 15th EuropeanConfe-rence on Computer Vision.Piscataway,USA:Springer,2018:3-19.
[23]SELVARAJU R R,COGSWELL M,DAS A,et al.Grad-CAM:Visual explanations from deep networks via gradient-based localization[C]//2017 IEEE International Conference on Compu-ter Vision.Piscataway,USA:IEEE Press,2017:618-626.
[24]OTSU N.A threshold selection method from gray-level histograms[J].IEEE Transactions on Systems,Man,and Cyberne-tics,1979,9(1):62-66.
[25]RAJPURKAR P,IRVIN J,ZHU K L,et al.CheXNet:Radiologist-level pneumonia detection on chest X-Rays with deep lear-ning[EB/OL].(2019-06-12)[2021-08-01].http://arxiv.org/abs/1711.05225.
[26]LIU H M,WANG R P,SHAN S G,et al.Deep supervised ha-shing for fast image retrieval[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway,USA:IEEE Press,2016:2064-2072.
[27]LI Y Q,PEI W J,ZHA Y F,et al.Push for quantization:Deep fisher hashing[C]//30th British Machine Vision Conference.Cardiff,United Kingdom:BMVC,2019:1-12.
[28]FAN L X,NG K W,JU C,et al.Deep polarized network for supervised learning of accurate binary hashing codes[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2020:825-831.
[29]LI W J,WANG S,KANG W C.Feature learning based deep supervised hashing with pairwise labels[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2016:1711-1717.
[30]ZHANG Z,ZOU Q,LIN Y W,et al.Improved deep hashing with soft pairwise similarity for multi-label image retrieval[J].IEEE Transactions on Multimedia,2020,22(2):540-553.
[31]WANG X F,SHI Y,KITANI K M.Deep supervised hashingwith triplet labels[C]//13th Asian Conference on Computer Vision.Cham,Switzerland:Springer,2017:70-84.
[32]ZHU H,GAO S H.Locality constrained deep supervised ha-shing for image retrieval[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.Palo Alto,USA:AAAI Press,2017:3567-3573.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 周芳泉, 成卫青.
基于全局增强图神经网络的序列推荐
Sequence Recommendation Based on Global Enhanced Graph Neural Network
计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[3] 戴禹, 许林峰.
基于文本行匹配的跨图文本阅读方法
Cross-image Text Reading Method Based on Text Line Matching
计算机科学, 2022, 49(9): 139-145. https://doi.org/10.11896/jsjkx.220600032
[4] 周乐员, 张剑华, 袁甜甜, 陈胜勇.
多层注意力机制融合的序列到序列中国连续手语识别和翻译
Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion
计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[5] 熊丽琴, 曹雷, 赖俊, 陈希亮.
基于值分解的多智能体深度强化学习综述
Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization
计算机科学, 2022, 49(9): 172-182. https://doi.org/10.11896/jsjkx.210800112
[6] 李宗民, 张玉鹏, 刘玉杰, 李华.
基于可变形图卷积的点云表征学习
Deformable Graph Convolutional Networks Based Point Cloud Representation Learning
计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[7] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[8] 陈泳全, 姜瑛.
基于卷积神经网络的APP用户行为分析方法
Analysis Method of APP User Behavior Based on Convolutional Neural Network
计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[9] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[10] 檀莹莹, 王俊丽, 张超波.
基于图卷积神经网络的文本分类方法研究综述
Review of Text Classification Methods Based on Graph Convolutional Network
计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[11] 闫佳丹, 贾彩燕.
基于双图神经网络信息融合的文本分类方法
Text Classification Method Based on Information Fusion of Dual-graph Neural Network
计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[12] 汪鸣, 彭舰, 黄飞虎.
基于多时间尺度时空图网络的交通流量预测模型
Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction
计算机科学, 2022, 49(8): 40-48. https://doi.org/10.11896/jsjkx.220100188
[13] 金方焱, 王秀利.
融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取
Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM
计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[14] 熊罗庚, 郑尚, 邹海涛, 于化龙, 高尚.
融合双向门控循环单元和注意力机制的软件自承认技术债识别方法
Software Self-admitted Technical Debt Identification with Bidirectional Gate Recurrent Unit and Attention Mechanism
计算机科学, 2022, 49(7): 212-219. https://doi.org/10.11896/jsjkx.210500075
[15] 彭双, 伍江江, 陈浩, 杜春, 李军.
基于注意力神经网络的对地观测卫星星上自主任务规划方法
Satellite Onboard Observation Task Planning Based on Attention Neural Network
计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!