Computer Science ›› 2021, Vol. 48 ›› Issue (9): 181-186.doi: 10.11896/jsjkx.200800064

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Multi-focus Image Fusion Method Based on PCANet in NSST Domain

HUANG Xiao-sheng, XU Jing   

  1. School of Software Engineering,East China Jiaotong University,Nanchang 330013,China
  • Received:2020-08-11 Revised:2020-12-25 Online:2021-09-15 Published:2021-09-10
  • About author:HUANG Xiao-sheng,born in 1972,Ph.D,associate professor,is a member of China Computer Federation.His main research interests include digital image processing,computer vision and computer graphics.
    XU Jing,born in 1996,postgraduate.Her main research interests include image fusion and so on.
  • Supported by:
    National Natural Science Foundation of China(61763011,61762037,61962021) and Natural Science Foundation of Jiangxi Province(20192BBE50079)

Abstract: The deep learning model based image fusion methods have attracted much attention in recently years.But the traditio-nal deep learning model usually needs a time-consuming and complex training process and a difficulty parameters tuning process on large datasets.To overcome these problems,a simple deep learning model PCANet based multi-focus image fusion method in NSST domain is proposed.Firstly,multi-focus images are used to train two-stage PCANet to extract image features.Then,the input source image is decomposed by NSST to obtain the multi-scale and multi-directional representation of the source image.The low frequency subband uses the trained PCANet to extract its image features,and uses the kernel norm to construct an effective feature space for image fusion.High frequency subbands are fused using the fusion rule of regional energy maximization.Finally,the frequency coefficients fused according to different fusion rules are reconstructed by NSST to obtain a clear target image.The experimental results show that the training and fusion speed of the algorithm is 43% higher than that of the CNN-based method.The average gradient,spatial frequency and entropy of the proposed algorithm are 5.744,15.560 and 7.059 respectively,which can be comparable to or superior to the existing fusion methods.

Key words: CNN, Deep learning, Multi-focus image fusion, NSST, PCANet

CLC Number: 

  • TP391
[1]LI S,KANG X,FANG L,et al.Pixel-level image fusion:A survey of the state of the art[J].Information Fusion,2016,33:100-112.
[2]DOU J F,QIN Q,TU Z M.Image fusion based on wavelettransform with genetic algorithms and human visual system[J].Multimedia Tools and Applications,2019,78(9):12491-12517.
[3]MAO R,FU X,NIU P,et al.Multi-directional Laplacian Pyramid Image Fusion Algorithm[C]//2018 3rd International Conference on Mechanical,Control and Computer Engineering (ICMCCE).IEEE Computer Society,2018.
[4]DO M N,VETTERLI M.The Contourlet Transform:An Efficient Directional Multiresolution Image Representation[J].IEEE Transactions on Image Processing,2006,14(12):2091-2106.
[5]ZHU D R,XU L,WANG F B,et al.Multi-focus image fusion Algorithm based on Fast Finite Shear Wave Transform and Guided Filter[J].Advances in Laser and Optoelectronics,2018,55(1):190-197.
[6]LIU Y,LIU S,WANG Z.A general framework for image fusion based on multi-scale transform and sparse representation[J].Information Fusion,2015,24:147-164.
[7]LIU Y,CHEN X,WANG Z,et al.Deep learning for pixel-level image fusion:Recent advances and future prospects[J].Information Fusion,2018,42:158-173.
[8]LI P,WANG H,LI X,et al.A novel Image Fusion Framework based on Non-Subsampled Shearlet Transform (NSST) Domain[C]//2019 Chinese Control And Decision Conference (CCDC).2019.
[9]YIN M,LIU X,LIU Y,et al.Medical Image Fusion With Pa-rameter-Adaptive Pulse Coupled Neural Network in Nonsubsampled Shearlet Transform Domain[J].Instrumentation & Measurement IEEE Transactions on,2019,68(1):49-64.
[10]WU Y,TAO F.Multispectral and panchromatic image fusionbased on improved projected gradient NMF in NSST domain[J].Guangxue Xuebao/acta Optica Sinica,2015,35(4):0410005.
[11]LIU Y,CHEN X,PENG H,et al.Multi-focus image fusion with a deep convolutional neural network[J].Information Fusion,2017,36:191-207.
[12]DU J K,LONG H L,DING D D,et al.CNN image fusion algorithm based on non-subsampled Shearlet transform[J].Audio Engineering,2019,43(3):37-41,73.
[13]CHAN T H,JIA K,GAO S,et al.PCANet:A Simple DeepLearning Baseline for Image Classification?[J].IEEE Transactions on Image Processing,2015,24(12):5017-5032.
[14]ZHANG D D,LI L.Face detection system based on PCANET-RF[J].Computer Technology and Development,2016(2):31-34.
[15]SONG X,WU X J.Multi-focus Image Fusion with PCA Filters of PCANet[M]//Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction.2019.
[16]LABATE D,LIM W Q,KUTYNIOK G,et al. Sparse multidimensional representation using shearlets[C]//2005 Proceedings of SPIE-The International Society for Optical Engineering.Bellingham,WA,USA,2005.
[17]EASLEY G,LABATE D,LIM W Q.Sparse directional image representations using the discrete shearlet transform[J].Applied & Computational Harmonic Analysis,2008,25(1):25-46.
[18]ZHANG L T,LV X L,SUN M X,et al.Image Fusion Method Based on Wavelet Transform[J].Electro-Optic Technology Application,2016:204-210.
[19]LIU J S,JIANG W.Improved image fusion algorithm based on nonsubsampled Contourlet transform[J].Journal of Computer Applications,2018,38(z1):194-197.
[20]WANG H,AHMED M S.Multi-focus image fusion algorithmbased on non-subsampled shearlet transform and focus measure[C]//International Conference on Robotics & Machine Vision.2017.
[21]LIU S Q,WANG J,AN Y L,et al.Multi-focus image fusion in non-subsampled shear Wave domain based on CNN[J].Journal of Zhengzhou University:Engineering Science,2019,40(4):36-41.
[1] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[9] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[10] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[11] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[12] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
[15] SUN Fu-quan, CUI Zhi-qing, ZOU Peng, ZHANG Kun. Brain Tumor Segmentation Algorithm Based on Multi-scale Features [J]. Computer Science, 2022, 49(6A): 12-16.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!