卷积神经网络低层特征辅助的图像实例分割方法

doi:10.11896/jsjkx.191200063

Abstract

Abstract: The popular instance segmentation network,Mask R-CNN,has rough target segmentation boundaries and segmentation contours when performing instance segmentation,which leads to low segmentation accuracy.To solve this problem,a high-precision instance segmentation method is proposed by introducing the low-level features of the network into the segmentation branch of Mask R-CNN.Specifically,it selects the convolutional features from lower layers of feature extraction network at first.And then,it resizes the features to a fixed scale (1/8 of the input image) by interpolation algorithm to form the low-level features.It concatenates the features of original segmentation branch of Mask R-CNN with the features extracted by RoI Align ope-ration from low-level features for current target.Since low-level features introduce more low-level texture and contour information,it can effectively improve the accuracy of instance segmentation.Compared with Mask R-CNN,the proposed method obtains 1.2% relative average precision (AP) improvement on the COCO2017 dataset by using ResNet-101-FPN as the feature extraction network.Experimental results show that the proposed method is robust and effective when using different feature extraction networks.

Key words: Deep learning, Deep neural network, Feature fusion, Instance segmentation, Low-level feature

CLC Number:

TP391.4

FAN Wei, LIU Ting, HUANG Rui, GUO Qing, ZHANG Bao. Low-level CNN Feature Aided Image Instance Segmentation[J].Computer Science, 2020, 47(11): 186-191.

References

[1] LIU S,QI L,QIN H,et al.Path Aggregation Network for Instance Segmentation[J].arXiv:1803.01534.
[2] LUO J,SAVAKIS A E,SINGHAL A.A Bayesian network-based framework for semantic image understanding[J].Pattern Recognition,2005,38(6):919-934.
[3] LI L,JIANG S Q,HUANG Q M.Learning Hierarchical Semantic Description Via Mixed-Norm Regularization for Image Understanding[J].IEEE Transactions on Multimedia,2012,14(5):1401-1413.
[4] LOZANO S,MÖLLER K,BRENDLE A,et al.AUTOPILOT-BT:A system for knowledge and model based mechanical ventilation[J].Technology and Health Care,2008,16(1):1-11.
[5] THEIS J,OSSMANN D,THIELECKE F,et al.Robust autopilot design for landing a large civil aircraft in crosswind[J].Control Engineering Practice,2018,76:54-64.
[6] ZHU J,LAO Y W,ZHENG Y F.Object Tracking in Structured Environments for Video Surveillance Applications[J].IEEE Transactions on Circuits and Systems for Video Technology,2010,20(2):223-235.
[7] GILBERT A L,GILES M K,FLACHS G M,et al.A Real-Time Video Tracking System[J].IEEE Transactions on Pattern Ana-lysis and Machine Intelligence,1980(1):10.
[8] SALTI S,CAVALLARO A,DI STEFANO L.Adaptive Ap-pearance Modeling for Video Tracking:Survey and Evaluation[J].IEEE Transactions on Image Processing,2012,21(10):4334-4348.
[9] YEE K P,SWEARINGEN K,LI K,et al.Faceted metadata for image search and browsing[C]//Proceedings of the 2003 Conference on Human Factors in Computing Systems(CHI 2003).Ft.Lauderdale,Florida,USA,2003.
[10] WANG M,LI H,TAO D C,et al.Multimodal Graph-BasedReranking for Web Image Search[J].IEEE Transactions on Ima-ge Processing,2012,21(11):4649-4661.
[11] LI X,LIU Z,LUO P,et al.Not All Pixels Are Equal:Difficulty-aware Semantic Segmentation via Deep Layer Cascade[J].ar-Xiv:1704.01344.
[12] LIU Z,LI X,LUO P,et al.Semantic Image Segmentation viaDeep Parsing Network[J].arXiv:1509.02634.
[13] PINHEIRO P O,COLLOBERT R,DOLLAR P.Learning toSegment Object Candidates[J].arXiv:1506.06204.
[14] PINHEIRO P O,LIN T Y,COLLOBERT R,et al.Learning to Refine Object Segments[J].arXiv:1603.08695.
[15] DAI J,HE K,LI Y,et al.Instance-sensitive Fully Convolutional Networks[J].arXiv:1603.08678.
[16] DAI J,HE K,SUN J.Instance-aware Semantic Segmentation via Multi-task Network Cascades[J].arXiv:1512.04412.
[17] HAYDER Z,HE X,SALZMANN M.Boundary-Aware Instance Segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2017.
[18] DAI J,LI Y,HE K,et al.R-FCN:Object Detection via Region-based Fully Convolutional Networks[J].arXiv:1605.06409.
[19] GIRSHICK R.Fast r-cnn[C]//2015 IEEE International Confe-rence on Computer Vision (ICCV).IEEE,2016.
[20] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2017,39(6):1137-1149.
[21] LI Y,QI H,DAI J,et al.Fully Convolutional Instance-Aware Semantic Segmentation[C]//2017 IEEE Conference on Compu-ter Vision and Pattern Recognition (CVPR).Honolulu,HI:IEEE,2017:4438-4446.
[22] CHEN L C,HERMANS A,PAPANDREOU G,et al.Mask-Lab:Instance Segmentation by Refining Object Detection with Semantic and Direction Features[J].arXiv:1712.04837.
[23] HE K,GKIOXARI G,PIOTR DOLLÁ R,et al.Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision (ICCV).IEEE,2017.
[24] HUANG Z,HUANG L,GONG Y,et al.Mask Scoring R-CNN[J].arXiv:1903.00241.
[25] CHEN K,PANG J,WANG J,et al.Hybrid Task Cascade forInstance Segmentation[J].arXiv:1901.07518.
[26] CAI Z,VASCONCELOS N.Cascade R-CNN:Delving into High Quality Object Detection[J].arXiv:1712.00726.
[27] SUN Y,P P S K,SHIMAMURA J,et al.Concatenated Feature Pyramid Network for Instance Segmentation[J].arXiv:1904.00768.
[28] LIN T Y,DOLLÁR P,GIRSHICK R,et al.Feature PyramidNetworks for Object Detection[J].arXiv:1612.03144.
[29] LIANG X,LIN L,WEI Y,et al.Proposal-free Network for Instance-level Object Segmentation[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2018,40(12):2978-2991.
[30] BAI M,URTASUN R.Deep Watershed Transform for Instance Segmentation[J].arXiv:1611.08303.
[31] BEUCHER S,C ANTUÉL.Use of Watersheds in Contour Detection[C]//International workshop on image processing,real-time edge and motion detection.CCETT,1979.
[32] KIRILLOV A,LEVINKOV E,ANDRES B,et al.InstanceCut:from Edges to Instances with MultiCut[J].arXiv:1611.08272.
[33] JIN L,CHEN Z,TU Z.Object Detection Free Instance Segmentation With Labeling Transformations[J].arXiv:1611.08991.
[34] LIU S,JIA J,FIDLER S,et al.SGN:Sequential Grouping Networks for Instance Segmentation[C]//2017 IEEE International Conference on Computer Vision (ICCV).Venice:IEEE,2017:3516-3524.
[35] REN M,ZEMEL R S.End-to-End Instance Segmentation with Recurrent Attention[C]//Computer Vision & Pattern Recognition.IEEE,2017.
[36] ROMERA-PAREDES B,TORR P H S.Recurrent Instance Segmentation[J].Computer Science,2016,9910(10):312-329.
[37] HOCHREITER S,SCHMIDHUBER J.Long Short-Term Memory[J].Neural Computation,1997,9(8):1735-1780.
[38] SHI X,CHEN Z,WANG H,et al.Convolutional LSTM Net-work:A Machine Learning Approach for Precipitation Nowcas-ting[J].arXiv:1506.04214.
[39] ZEILER M D,FERGUS R.Visualizing and Understanding Convolutional Networks[J].arXiv:1311.2901.
[40] LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft COCO:Common Objects in Context[C]//European Conference on Computer Vision.Springer International Publishing,2014.
[41] MASSA F,GIRSHICK R.maskrcnn-benchmark:Fast,modularreference implementation of Instance Segmentation and Object Detection algorithms in PyTorch[OL].https://github.com/facebookresearch/maskrcnn-benchmark.

Related Articles 15

[1]	RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[2]	TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[3]	XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[4]	WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5]	HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6]	JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7]	SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8]	HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[9]	ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[10]	CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[11]	HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[12]	ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[13]	SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[14]	ZHU Wen-tao, LAN Xian-chao, LUO Huan-lin, YUE Bing, WANG Yang. Remote Sensing Aircraft Target Detection Based on Improved Faster R-CNN [J]. Computer Science, 2022, 49(6A): 378-383.
[15]	WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Low-level CNN Feature Aided Image Instance Segmentation

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0