Computer Science ›› 2020, Vol. 47 ›› Issue (6A): 148-152.doi: 10.11896/JsJkx.190700046

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Application of Deep Learning in Photoacoustic Imaging

SUN Zheng and WANG Xin-yu   

  1. Department of Electronic and Communication Engineering,North China Electric Power University,Baoding,Hebei 071003,China
  • Published:2020-07-07
  • About author:SUN Zheng, born in 1977, Ph.D, professor.Her main research interests include biomedical imaging and signal proces-sing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61372042) and Fundamental Research Funds for the Central Universities of Ministry of Education of China (2014AD31).

Abstract: Photoacoustic imaging (PAI) is a multi-physics coupled non-invasive biomedical functional imaging technology.It combines the high contrast of pure optical imaging with the high spatial resolution of ultrasonic imaging,and can obtain the morpho-logy and functional components information of target tissues at the same time.In recent years,deep learning (DL) has been widely applied in medical image processing.The PAI imaging algorithms based on DL have attracted more and more attention of researchers.This paper reviewed the current application of DL in PAI image reconstruction,summarized the existing algorithms,analyzed their limits and forecasted the possible improvements in the future.

Key words: Photoacoustic imaging, Deep learning, Convolutional neural network, Image reconstruction, Limited-angle scanning

CLC Number: 

  • TP391.4
[1] YAO J,WANG LV.Recent progress in photoacoustic molecular imaging.Current Opinion in Chemical Biology,2018,45:104-112.
[2] POUDEL J,LOU Y,ANASTASIO M A.A survey of computational frameworks for solving the acoustic inverse problem in three-dimensional photoacoustic computed tomography.doi:10.1088/1361-6560/ab2017.
[3] BU S,YAMAKAWAY M,SHIINA T.Interpolation method for model-based 3-D planar photoacoustic tomography reconstruction//MBE,医用?生体工学研究会.2011:139-142.
[4] SYED TA,KRISHNAN VP,SIVASWAMY J.Numerical inversion of circular arc Radon transform.IEEE Transactions on Computational Imaging,2016,2(4):540-549.
[5] WANG J,WANG Y.An Efficient compensation method for limited-view photoacoustic imaging reconstruction based on Gerchberg-Papoulis extrapolation.Applied Sciences,2017,7(5):505.
[6] SUNNEGRDH J,DANIELSSON P E.Regularized iterative weighted filtered backproJection for helical cone-beam CT.Medical Physics,2008,35(9):4173-4185.
[7] ANTHOLZER S,SCHWAB J,BAUER-MARSCHALLINGER J,et al.NETT regularization for compressed sensing photoacoustic tomography//Proceedings of SPIE International Conference on Photons Plus Ultrasound:Imaging and Sensing 2019.2019:108783B.
[8] SYED T A,KRISHNAN V P,SIVASWAMY J.Numerical inversion of circular arc radon transform .IEEE Transactions on Computational Imaging,2017,2(4):540-549.
[9] GUO W.Research on Reconstruction Algorithms of CT with Incomplete ProJection Data .Changshun:Jilin University,2011.
[10] SU B L,ZHANG Y H,PENG L H,et al.Simultaneous iterative reconstruction technique for electrical capacitance tomography .Journal of Tsinghua University(Science and Technology),2000(9):90-92.
[11] CHEN X,HU H L,GAO X X,et al.Comparison of Algebraic Reconstruction Technique and Simultaneous Iterative Reconstruction Technique in Electrical Capacitance Tomography Image Reconstruction .Journal of Xi’an Jiaotong University,2011,45(4):25-29.
[12] YANG D W,XING D,ZHAO X,et al.A combined reconstruction algorithm for limited-view multi-element photoacoustic imaging .Chinese Physics Letters,2010,27(5):144-147.
[13] CHAUDHARY G,ROUMELIOTIS M,CARSON J J L,et al.Comparison of reconstruction algorithms for sparse-array detection photo-acoustic tomography //Proceedings of SPIE International Conference on Photons Plus Ultrasound:Imaging and Sensing.2010,7564:756434.
[14] WANG Q,WU Y N.New Analytical Solution to Extrapolation Problem for Band-Limited Signals .Journal of Electronics and Information Technology,1999,21(6):825-828.
[15] LIU X Y,PENG D,GUO W,et al.Compressed sensing photoacoustic imaging based on fast alternating direction algorithm .International Journal of Biomedical Imaging,2012,2012:206-214.
[16] VASWANI N,LU W.Modified-CS:modifying compressive sensing for problems with partially known support .IEEE Transactions on Signal Processing,2010,58(9):4595-4607.
[17] MENG J,WANG LV,YING L,et al.Compressed-sensing photoacoustic computed tomography in vivo with partially known support .Medical & Biological Imaging.2012,20(15):16510-16523.
[18] LIANG D,ZHANG H F,YING L.Compressed-sensing photoacoustic imaging based on random optical illumination .International Journal of Functional Informatics & Personalised Medicine,2009,2(4):394-406.
[19] ARRIDGE S,BEARD P,BETCKE M,et al.Accelerated high-resolution photoacoustic tomography via compressed sensing .Physics in Medicine & Biology,2016,61(24):8908.
[20] MENG J,JIANG Z,WANG LV,et al.High-speed,sparse-sampling three-dimensional photoacoustic computed tomography in vivo based on principal component analysis.Journal of Biomedical Lptics,2016,21(7):076007.
[21] LITJENS G,KOOI T,BEJNORDI B E,et al.A survey on deep learning in medical image analysis.Medical Image Analysis,2017,42(9):60-88.
[22] WANG G,YE J C,MUELLER K,et al.Image reconstruction is a new frontier of machine learning.IEEE Transactions on Medical Imaging,2018,37(6):1289-1296.
[23] LUCAS A,ILIADIS M,MOLINA R,et al.Using deep neural networks for inverse problems in imaging:beyond analytical methods.IEEE Signal Processing Magazine,2018,35(1):20-36.
[24] HALTMEIER M,ANTHOLZER S,SCHWAB J,et al.Photoacoustic image reconstruction via deep learning//Proceedings of SPIE International Conference on Photons Plus Ultrasound:Imaging and Sensing 2018.2018:104944U.
[25] KELLY B,MATTHEWS T P,ANASTASIO M A.Deep lear-ning-guided image reconstruction from incomplete data//Proceedings of 31st Annual Conference on Neural Information Processing Systems (NIPS 2017).Long Beach,CA,USA,2017.
[26] HAUPTMANN A,LUCKA F,BETCKE M,et al.Model based learning for accelerated,limited-view 3D photoacoustic tomography.IEEE Transactions on Medical Imaging,2018,37(6):1382-1393.
[27] HALTMEIER M,ANTHOLZER S,SCHWAB J,et al.Photoacoustic image reconstruction via deep learning//Proceedings of SPIE International Conference on Photons Plus Ultrasound:Imaging and Sensing 2018.2018:104944U.
[28] SCHWAB J,ANTHOLZER S,NUSTER R,et al.Real-time photoacoustic proJection imaging using deep learning.arXiv:1801.06693,2018.
[29] SCHWAB J,ANTHOLZER S,HALTMEIER M.Learned backproJection for sparse and limited view photoacoustic tomography//Proceedings of SPIE International Conference on Photons Plus Ultrasound:Imaging and Sensing 2019.2019:1087837
[30] SUN Z,HAN D,YUAN Y.2-D image reconstruction of photoacoustic endoscopic imaging based on time-reversal.Compu-ters in Biology and Medicine,2016,76:60-68.
[31] WAIBEL D J E.Photoacoustic image reconstruction to solve the acoustic inverse problem with deep learning.University of Heidelberg,2018.
[32] WAIBEL D,GRHL J,ISENSEE F,et al.Reconstruction of ini-tial pressure from limited view photoacoustic images using deep learning//Proceedings of SPIE International Conference on Photons Plus Ultrasound:Imaging and Sensing 2018.2018:104942S.
[33] SCHWAB J,ANTHOLZER S,NUSTER R,et al.DALnet: high-resolution photoacoustic proJection imaging using deep learning.arXiv:1801.06693,2018.
[34] ANTHOLZER S,HALTMEIER M,SCHWAB J.Deep learning for photoacoustic tomography from sparse data.Inverse Problems in Science and Engineering,2018,27(6):1-19.
[35] SCHWAB J,ANTHOLZER S,HALTMEIER M.Deep null space learning for inverse problems:Convergence analysis and rates.arXiv:1806.06137,2018.
[36] ANTHOLZER S,SCHWAB J,HALTMEIER M.Deep learning versus l1-minimization for compressed sensing photoacoustic tomography.arXiv:1901.06510,2019.
[37] AWASTHI N,PRABHAKAR K R,KALVA S K,et al.PAFuse:deep supervised approach for the fusion of photoacoustic images with distinct reconstruction characteristics.Biomedi-cal Optics Express,2019,10(5):2227-2243.
[38] ADLER J,KTEM O.Solving ill-posed inverse problems using iterative deep neural networks.Inverse Problems,2017,33(12):124007.
[39] LI H,SCHWAB J,ANTHOLZER S,et al.NETT_solving inverse problems with deep neural networks.arXiv:1803.00092,2018.
[40] ARRIDGE S,BEARD P,BETCKE M,et al.Accelerated high-resolution photoacoustic tomography via compressed sensing.Physics in Medicine and Biology,2016,61(24):8908-8940.
[41] SCHWAB J,ANTHOLZER S,HALTMEIER M.Big in Japan:regularizing networks for solving inverse problems.arXiv:1812.00965,2018.
[42] ANTHOLZER S,SCHWAB J,MARSCHALLINGER J B,et al.NETT regularization for compressed sensing photoacoustic tomography.arXiv:1901.11158v1,2019.
[43] SCHWAB J,ANTHOLZER S,NUSTER R,et al.Deep learning of truncated singular values for limited view photoacoustic tomography//Proceedings of SPIE International Conference on Photons Plus Ultrasound:Imaging and Sensing 2019.2019:1087836.
[44] ANDRYCHOWICZ M,DENIL M,GOMEZ S,et al.Learning to learn by gradient descent by gradient descent//Proceedings of 30th Annual Conference on Neural Information Processing Systems 2016 (NIPS2016).Barcelona,Spain,2016:3981-3989.
[45] HAUPTMANN A,COX B,LUCKA F,et al.Approximate kspace models and deep learning for fast photoacoustic reconstruction//Machine Learning for Medical Image Reconstruction (MLMIR 2018).Lecture Notes in Computer Science.Cham:Springer,2018:103-111.
[46] SUN Z,ZHENG L.Progress in Quantitative photoacoustictomography.Chinese Journal of Luminescence,2017,38(9):1222-1232.
[47] JIN H,ZHANG R,LIU S,et al.A single sensor dual-modality photoacoustic fusion imaging for compensation of light fluence variation.IEEE Transactions on Biomedical Engineering,2019,66(6):1810-1813.
[48] CAI C,DENG K,MA C,et al.End-to-end deep neural network for optical inversion in quantitative photoacoustic imaging.Optics Letters,2018,43(12):2752-2755.
[49] KIRCHNER T,GRHL J,MAIER-HEIN L.Context encoding enables machine learning-based quantitative photoacoustics.Journal of Biomedical Optics,2018,23(5):056008.
[50] GRHL J,KIRCHNER T,ADLER T,et al.Confidence estimation for machine learning-based quantitative photoacoustics.Journal of Imaging,2018,4:147.
[51] TREEBY B E,COX B T.k-Wave:MATLAB toolbox for the simulation and reconstruction of photoacoustic wave fields.Journal of Biomedical Optics,2010,15(2):1-12.
[52] JACQUES S L.Coupling 3D Monte Carlo light transport in optically heterogeneous tissues to photoacoustic signal generation.Photoacoustics,2014,2(4):137-142.
[53] SUN Z,YUAN Y,HAN D.A computer-based simulator for intravascular photoacoustic images.Computers in biology and medicine,2017,81:176-187.
[1] SHAN Mei-jing, QIN Long-fei, ZHANG Hui-bing. L-YOLO:Real Time Traffic Sign Detection Model for Vehicle Edge Computing [J]. Computer Science, 2021, 48(1): 89-95.
[2] HE Yan-hui, WU Gui-xing, WU Zhi-qiang. Domain Alignment Based Object Detection of X-ray Images [J]. Computer Science, 2021, 48(1): 175-181.
[3] LI Ya-nan, HU Yu-jia, GAN Wei, ZHU Min. Survey on Target Site Prediction of Human miRNA Based on Deep Learning [J]. Computer Science, 2021, 48(1): 209-216.
[4] WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM [J]. Computer Science, 2021, 48(1): 226-232.
[5] YU Wen-jia, DING Shi-fei. Conditional Generative Adversarial Network Based on Self-attention Mechanism [J]. Computer Science, 2021, 48(1): 241-246.
[6] TONG Xin, WANG Bin-jun, WANG Run-zheng, PAN Xiao-qin. Survey on Adversarial Sample of Deep Learning Towards Natural Language Processing [J]. Computer Science, 2021, 48(1): 258-267.
[7] DING Yu, WEI Hao, PAN Zhi-song, LIU Xin. Survey of Network Representation Learning [J]. Computer Science, 2020, 47(9): 52-59.
[8] HE Xin, XU Juan, JIN Ying-ying. Action-related Network:Towards Modeling Complete Changeable Action [J]. Computer Science, 2020, 47(9): 123-128.
[9] ZHANG Jia-jia, ZHANG Xiao-hong. Multi-branch Convolutional Neural Network for Lung Nodule Classification and Its Interpretability [J]. Computer Science, 2020, 47(9): 129-134.
[10] YE Ya-nan, CHI Jing, YU Zhi-ping, ZHAN Yu-liand ZHANG Cai-ming. Expression Animation Synthesis Based on Improved CycleGan Model and Region Segmentation [J]. Computer Science, 2020, 47(9): 142-149.
[11] ZHU Ling-ying, SANG Qing-bing, GU Ting-ting. No-reference Stereo Image Quality Assessment Based on Disparity Information [J]. Computer Science, 2020, 47(9): 150-156.
[12] DENG Liang, XU Geng-lin, LI Meng-jie, CHEN Zhang-jin. Fast Face Recognition Based on Deep Learning and Multiple Hash Similarity Weighting [J]. Computer Science, 2020, 47(9): 163-168.
[13] CUI Tong-tong, WANG Gui-ling, GAO Jing. Ship Trajectory Classification Method Based on 1DCNN-LSTM [J]. Computer Science, 2020, 47(9): 175-184.
[14] LIU Hai-chao, WANG Li. Graph Classification Model Based on Capsule Deep Graph Convolutional Neural Network [J]. Computer Science, 2020, 47(9): 219-225.
[15] BAO Yu-xuan, LU Tian-liang, DU Yan-hui. Overview of Deepfake Video Detection Technology [J]. Computer Science, 2020, 47(9): 283-292.
Full text



[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8] GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[9] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[10] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .