Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 231100005-7.doi: 10.11896/jsjkx.231100005

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Stereo Matching Network Based on Enhanced Superpixel Sampling

XU Haidong1,2, ZHANG Zili 1,2,3, HU Xinrong1,2,3, PENG Tao1,2,3 , ZHANG Jun4   

  1. 1 Engineering Research Center of Hubei Province for Clothing Information,Wuhan 430200,China
    2 School of Computer Science and Artificial Intelligence,Wuhan Textile University,Wuhan 430200,China
    3 Hubei Provincial Engineering Research Center for Intelligent Textile and Fashion,Wuhan 430200,China
    4 School of Computer Science and Engineering,Wuhan Institute of Technology,Wuhan 430205,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:XU Haidong,born in 1999,postgra-duate,is a member of CCF(No.Q6975G).His main research interests include machine learning and image processing.
    ZHANG Zili,born in 1981,Ph.D,lectu-rer,is a member of CCF(No.99006M).His main research interests include machine learning and image processing.
  • Supported by:
    Science and Technology Research Project of Education Department of Hubei Province(B2017066).

PDF (PC)

187

Abstract: Aiming at the accuracy challenges in stereo matching related to details,occlusion,and textureless regions,a stereo matching method based on improved superpixel sampling is proposed.Initially,an enhanced superpixel sampling method is employed to downsample the high-resolution input images used for stereo matching.Subsequently,the downsampled image pairs are input into the stereo matching network,where a convolutional network with shared weights is utilized for feature extraction.Using 3D convolution,a feature-fused Cost Volume is generated,leading to the creation of a disparity map.The outputted disparity map is then upsampled to reconstruct the final disparity map.To tackle the issue of potential detail loss during the superpixel sampling process,two innovations are introduced:the feature pyramid attention module(FPA)and an improved residual structure.Based on these two innovations,a stereo matching network named FPSMnet(feature pyramid stereo matching network)is proposed.This paper selects and partitions the image datasets BSDS500 and NYUv2 for training,validation,and testing of superpixel sampling.Experimental results in stereo matching demonstrate that,compared to the baseline method,the proposed algorithm achieves a reduction of 0.25 and 0.52 in average pixel errors on the SceneFlow and HR-VS datasets,respectively.These improvements are achieved without compromising runtime efficiency.

Key words: Deep learning, Superpixels, Stereo matching, Attention mechanism

CLC Number: 

  • TP391
[1]YAO A Q,XU J M.Electric vehicle charging port recognitionand positioning system based on binocular vision[J].Sensors and Microsystems,2021,40(7):81-84.
[2]QI Y F,MA Z Y.Multi-loss head pose estimation based on deep residual networks[J].Computer Engineering,2020,46(12):247-253.
[3]ZENATI N,ZERHOUNI N.Dense stereo matching with application to augmented reality[C]//2007 IEEE International Conference on Signal Processing and Communications.IEEE,2007:1503-1506.
[4]SCHARSTEIN D,SZELISKI R.A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J].International Journal of Computer Vision,2002,47(1/2/3):7-42.
[5]CHANG J R,CHEN Y S.Pyramid Stereo Matching Network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition:[Volume 8 of 13].IEEE,2018:5410-5418.
[6]YANG F,SUN Q,JIN H,et al.Superpixel segmentation with fully convolutional networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:13964-13973.
[7]SONG X Y,ZHOU L L,LI Z G,et al.A comprehensive survey of superpixel methods in image segmentation[J].Journal of Image and Graphics,China,2015,20(5):599-608.
[8]JAMPANI V,SUN D Q,LIU M Y,et al.Superpixel Sampling Networks[C]//Computer Vision－ECCV 2018:15th European Conference.Springer,2018:363-380.
[9]LI P,MA W.OverSegNet:A convolutional encoder-decoder net-work for image over-segmentation[J].Computers and Electrical Engineering,2023,107:108610.
[10]LI H,XIONG P,AN J,et al.Pyramid attention network for semantic segmentation[J].arXiv:1805.10180,2018.
[11]HU J,SHEN L,SUN G.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:7132-7141.
[12]HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[13]LONG J,SHELHAMER E,DARRELL T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:3431-3440.
[14]TIAN X,WANG L,DING Q.A review of image semantic segmentation methods based on deep learning[J].Journal of Software,2019,30(2):440-468.
[15]WANG Y R,CHEN Q L,WU J J.A review of image semantic segmentation methods for complex environments[J].Computer Science,2019,46(9):36-46.
[16]ZHAO H,SHI J,QI X,et al.Pyramid scene parsing network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:2881-2890.
[17]VAN DEN BERGH M,BOIX X,ROIG G,et al.Seeds:Superpixels extracted via energy-driven sampling[C]//Computer Vision－ECCV 2012:12th European Conference on Computer Vision,Florence,Italy,Part VII 12.Springer Berlin Heidelberg,2012:13-26.
[18]ARBELAEZ P,MAIRE M,FOWLKES C,et al.Contour detection and hierarchical image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,33(5):898-916.
[19]SILBERMAN N,HOIEM D,KOHLI P,et al.Indoor segmentation and support inference from rgbd images[C]//Computer Vision－ECCV 2012:12th European Conference on Computer Vision,Florence,Italy.Springer Berlin Heidelberg,2012:746-760.
[20]STUTZ D,HERMANS A,LEIBE B.Superpixels:An evaluation of the state-of-the-art[J].Computer Vision and Image Understanding,2018,166:1-27.
[21]ACHANTA R,SHAJI A,SMITH K,et al.SLIC Super-pixels Compared to State-of-the-Art Superpixel Methods[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(11):2274-2282.
[22]LIU M Y,TUZEL O,RAMALINGAM S,et al.Entropy rate superpixel segmentation[C]//CVPR 2011.IEEE,2011:2097-2104.
[23]MAYER N,ILG E,HAUSSER P,et al.A large dataset to train convolutional networks for disparity,optical flow,and scene flow estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:4040-4048.
[24]YANG G,MANELA J,HAPPOLD M,et al.Hierarchical deep stereo matching on high-resolution images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:5515-5524.
[25]PANG J H,SUN W X,JIMMY S J R,et al.Cascade Residual Learning:A Two-stage Convolutional Neural Network for Ste-reo Matching[C]//2017 IEEE International Conference on Computer Vision Workshops(ICCVW 2017).Venice,Italy,2[v.2].:Institute of Electrical and Electronics Engineers,2017:878-886.
[26]KENDALL A,MARTIROSYAN H,DASGUPTA S,et al.End-to-End Learning of Geometry and Context for Deep Stereo Regression[C]//ICCV 2017.IEEE,2017.
[1] DU Yu, YU Zishu, PENG Xiaohui, XU Zhiwei. Padding Load:Load Reducing Cluster Resource Waste and Deep Learning Training Costs [J]. Computer Science, 2024, 51(9): 71-79.
[2] XU Jinlong, GUI Zhonghua, LI Jia'nan, LI Yingying, HAN Lin. FP8 Quantization and Inference Memory Optimization Based on MLIR [J]. Computer Science, 2024, 51(9): 112-120.
[3] LI Yunchen, ZHANG Rui, WANG Jiabao, LI Yang, WANG Ziqi, CHEN Yao. Re-parameterization Enhanced Dual-modal Realtime Object Detection Model [J]. Computer Science, 2024, 51(9): 162-172.
[4] HU Pengfei, WANG Youguo, ZHAI Qiqing, YAN Jun, BAI Quan. Night Vehicle Detection Algorithm Based on YOLOv5s and Bistable Stochastic Resonance [J]. Computer Science, 2024, 51(9): 173-181.
[5] LIU Qian, BAI Zhihao, CHENG Chunling, GUI Yaocheng. Image-Text Sentiment Classification Model Based on Multi-scale Cross-modal Feature Fusion [J]. Computer Science, 2024, 51(9): 258-264.
[6] LI Zhe, LIU Yiyang, WANG Ke, YANG Jie, LI Yafei, XU Mingliang. Real-time Prediction Model of Carrier Aircraft Landing Trajectory Based on Stagewise Autoencoders and Attention Mechanism [J]. Computer Science, 2024, 51(9): 273-282.
[7] LIU Qilong, LI Bicheng, HUANG Zhiyong. CCSD:Topic-oriented Sarcasm Detection [J]. Computer Science, 2024, 51(9): 310-318.
[8] YAO Yao, YANG Jibin, ZHANG Xiongwei, LI Yihao, SONG Gongkunkun. CLU-Net Speech Enhancement Network for Radio Communication [J]. Computer Science, 2024, 51(9): 338-345.
[9] SUN Yumo, LI Xinhang, ZHAO Wenjie, ZHU Li, LIANG Ya’nan. Driving Towards Intelligent Future:The Application of Deep Learning in Rail Transit Innovation [J]. Computer Science, 2024, 51(8): 1-10.
[10] KONG Lingchao, LIU Guozhu. Review of Outlier Detection Algorithms [J]. Computer Science, 2024, 51(8): 20-33.
[11] LIU Sichun, WANG Xiaoping, PEI Xilong, LUO Hangyu. Scene Segmentation Model Based on Dual Learning [J]. Computer Science, 2024, 51(8): 133-142.
[12] TANG Ruiqi, XIAO Ting, CHI Ziqiu, WANG Zhe. Few-shot Image Classification Based on Pseudo-label Dependence Enhancement and NoiseInterferenceReduction [J]. Computer Science, 2024, 51(8): 152-159.
[13] ZHANG Rui, WANG Ziqi, LI Yang, WANG Jiabao, CHEN Yao. Task-aware Few-shot SAR Image Classification Method Based on Multi-scale Attention Mechanism [J]. Computer Science, 2024, 51(8): 160-167.
[14] WANG Qian, HE Lang, WANG Zhanqing, HUANG Kun. Road Extraction Algorithm for Remote Sensing Images Based on Improved DeepLabv3+ [J]. Computer Science, 2024, 51(8): 168-175.
[15] XIAO Xiao, BAI Zhengyao, LI Zekai, LIU Xuheng, DU Jiajin. Parallel Multi-scale with Attention Mechanism for Point Cloud Upsampling [J]. Computer Science, 2024, 51(8): 183-191.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.