计算机视觉下的旋转目标检测研究综述

doi:10.11896/jsjkx.221000148

Abstract

Abstract: Traditional object detector locates objects by horizontal bounding box(HBB),which often have low accuracy and poor generalization ability when detecting objects with arbitrary orientation angle,dense distribution,large aspect ratio and complex background.The above problems can be effectively solved by adding rotating target boxes with different rotation angles in the bounding boxes.This method is widely used in the fields of remote sensing images,scene text images,shelf goods images and other target detection,and has important research value.Most of the current works aim at constructing different models for rotating object detection,and there are fewer review works for summarizing and analyzing existing models in depth.Therefore,this paper provides a detailed review of existing research results on rotating object detection.Firstly,according to the current popular way of target box characterization,the target boxes are classified into three types of oriented bounding box(OBB),quadrilateral bounding boxes(QBB) and point set for generalized analysis,and simultaneously compare the advantages and disadvantages,network structures and performance of different rotating object detection algorithms.Secondly,the commonly used rotating object detection datasets and performance evaluation metrics are analyzed.Finally,the problems in the current study are briefly summarized and discussed,and the future development trend is prospected.

Key words: Computer vision, Deep learning, Object detection, Rotating object, Performance comparison

CLC Number:

TP311

WANG Xu, WU Yanxia, ZHANG Xue, HONG Ruize, LI Guangsheng. Survey of Rotating Object Detection Research in Computer Vision[J].Computer Science, 2023, 50(8): 79-92.

References

[1]GIRSHICK R.Fast R-CNN [C]//GIRSHICK R.GIRSHICK R[C]//IEEE International Conference on Computer Vision.Piscataway:IEEE Press,2015:1440-1448.
[2]REN S,HE K,GIRSHICK R,et al.Faster r-cnn:Towards real-time object detection with region proposal networks[J].Advances in Neural Information Processing Systems,2015,28:91-99.
[3]HE K,ZHANG X,REN S,et al.Spatial pyramid pooling in deepconvolutional networks for visual recognition[J].IEEE Tran-sactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.
[4]REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:Unified real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE Comp. Soc.,2016:779-788.
[5]LIU W,ANGUELOV D,ERHAN D,et al.Ssd:Single shotmultibox detector[C]//European Conference on Computer Vision.Berlin:Springer,2016:21-37.
[6]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice,Italy:ICCV,2017:2980-2988.
[7]TIAN Z,HUANG W,HE T,et al.Detecting text in naturalimage with connectionisttext proposal network[C]//European Conference on Computer Vision.Berlin:Springer,2016:56-72.
[8]LIAO M,SHI B,BAI X,et al.TextBoxes:A Fast Text Detector with a Single Deep Neural Network[C]//Proceedings of the AAAI Conference on Artificial Intelligence.San Francisco,USA:ACM,2017,31(1):4161-4167.
[9]SHEN Y,LIU D,ZHANG F,et al.Fast and accurate multi-class geospatial object detection with large-size remote sensing ima-gery using CNN and Truncated NMS[J].ISPRS Journal of Photogrammetry and Remote Sensing,2022,191:235-249.
[10]SHI P,ZHAO Z,FAN X,et al.Remote Sensing Image Object Detection Based on Angle Classification[J].IEEE Access,2021,9:118696-118707.
[11]ZHANG L,ZHANG Y S,YU Y,et al.Survey on object detection in tilting box for remote sensing images[J].National Remote Sensing Bulletin,2022,26(9):1723-1743.
[12]MA J,SHAO W,YE H,et al.Arbitrary-oriented scene text detection via rotation proposals[J].IEEE Transactions on Multimedia,2018,20(11):3111-3122.
[13]YANG X,YAN J,FENG Z,et al.R3det:Refined single-stagedetector with feature refinement for rotating object[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.Online:AAAI,2021,35(4):3163-3171.
[14]ZHANG G,LU S,ZHANG W.CAD-Net:Acontext-aware de-tection network for objects in remote sensing imagery[J].IEEE Transactions on Geoscience and Remote Sensing,2019,57(12):10015-10024.
[15]MING Q,MIAO L,ZHOU Z,et al.Task interleaving and orientation estimation for high-precision oriented object detection in aerial images[J].ISPRS Journal of Photogrammetry and Remote Sensing,2023,196:241-255.
[16]YANG X,SUN H,SUN X,et al.Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network[J].IEEE Access,2018,6:50839-50849.
[17]SHI X,SHAN S,KAN M,et al.Real-time rotation-invariantface detection with progressive calibration networks[C]//Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition.Salt Lake City,UT,USA:IEEE,2018:2295-2303.
[18]LIU T,LI W G,GUAN J H.A Review of Object DetectionMethods in Optical Remote Sensing Image Based on Deep Learning [J].Radio Communication Technology,2020,46(6):624-634.
[19]NIE G T,HUANG H.A survey of object detection in optical re-mote sensing images[J].Acta Automatica Sinica,2021,47(8):1749-1768.
[20]SONG Z N,SUI H G,LI Y C.A survey on ship detection technology in high-resolution optical remote sensing images[J].Geomatics and Information Science of Wuhan University,2021,46(11):1703-1715.
[21]WANG J X,WANG Z Y,TIAN X.A Review of Text Detection and Recognition in Natural Scenes Based on Deep Learning [J].Journal of Software,2020,31(5):1465-1496.
[22]ZHAO T Z,YANG C W,LIU W.Remote sensing image target detection method based on non-local feature enhancement [J].Journal of Huazhong University of Science and Technology(Natural Science Edition),2021,49(9):47-51.
[23]SCHILLING H,BULATOV D,NIESSNER R,et al.Detection of vehicles in multisensor data via multibranch convolutional neural networks[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2018,11(11):4299-4316.
[24]LIAO M,SHI B,BAI X.Textboxes++:A single-shot oriented scene text detector[J].IEEE Transactions on Image Processing,2018,27(8):3676-3690.
[25]XU Y,FU M,WANG Q,et al.Gliding vertex on the horizontal bounding box for multi-oriented object detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(4):1452-1459.
[26]SUN P,ZHENG Y,ZHOU Z,et al.R4 Det:Refined single-stage detector with feature recursion and refinement for rotating object detection in aerial images[J].Image and Vision Computing,2020,103:104036.
[27]QIAN W,YANG X,PENG S,et al.Learning modulated loss for rotated object detection[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.Online:AAAI,2021,35(3):2458-2466.
[28]YANG X,YAN J,MING Q,et al.Rethinking rotated object detection with gaussian wasserstein distance loss[C]//International Conference on Machine Learning.PMLR,2021:11830-11841.
[29]MING Q,MIAO L,ZHOU Z,et al.Optimization for arbitrary-oriented object detection via representation invariance loss[J].IEEE Geoscience and Remote Sensing Letters,2021,19:1-5.
[30]LAW H,DENG J.Cornernet:Detecting objects as paired key-points[C]//Proceedings of the European Conference on Computer Vision(ECCV).Munich:ECCV,2018:734-750.
[31]YANG Z,LIU S,HU H,et al.Reppoints:Point set representation for object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul,Korea (South):IEEE,2019:9657-9666.
[32]HOU L,LU K,YANG X,et al.G-Rep:Gaussian Representation for Arbitrary-Oriented Object Detection[J].arXiv:2205.11796,2022.
[33]YANG X,YAN J.Arbitrary-oriented object detection with circular smooth label[C]//European Conference on Computer Vision.Berlin:Springer,2020:677-694.
[34]DING J,XUE N,LONG Y,et al.Learning RoI transformer for oriented object detection in aerial images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seoul,Korea (South):ICCV,2019:2849-2858.
[35]HAN J,DING J,XUE N,et al.Redet:A rotation-equivariant detector for aerial object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Montreal Canada:ICCV,2021:2786-2795.
[36]MA J,SHAO W,YE H,et al.Arbitrary-oriented scene text detection via rotation proposals[J].IEEE Transactions on Multimedia,2018,20(11):3111-3122.
[37]XIE X,CHENG G,WANG J,et al.Oriented R-CNN for object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Montreal Canada:ICCV,2021:3520-3529.
[38]YANG X,YANG J,YAN J,et al.Scrdet:Towards more robust detection for small,cluttered and rotated objects[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul,Korea (South):ICCV,2019:8232-8241.
[39]LYU C,ZHANG W,HUANG H,et al.Rtmdet: An empirical study of designing real-time object detectors[J].arXiv:2212.07784,2022.
[40]AN S B,LOU H R,CHEN S W,et al.Research progress of rotating target detection method based on deep learning [J].Electronic Measurement Technology,2021,44(21):168-178.
[41]JIANG Y,ZHU X,WANG X,et al.R2CNN:Rotational region CNN for orientation robust scene text detection[J].arXiv:1706.09579,2017.
[42]YANG X,SUN H,FU K,et al.Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks[J].Remote Sensing,2018,10(1):132.
[43]ZHU Y,DU J,WU X.Adaptive period embedding for representing oriented objects in aerial images[J].IEEE Transactions on Geoscience and Remote Sensing,2020,58(10):7247-7257.
[44]AZIMI S M,VIG E,BAHMANYAR R,et al.Towards multi-class object detection in unconstrained remote sensing imagery[C]//Asian Conference on Computer Vision.Berlin:Springer,2018:150-165.
[45]WANG J,DING J,GUO H,et al.Mask OBB:A semantic attention-based mask oriented bounding box representation for multi-category object detection in aerial images[J].Remote Sensing,2019,11(24):2930.
[46]LI Y,HUANG Q,PEI X,et al.RADet:Refine feature pyramid network and multi-layer attention network for arbitrary-oriented object detection of remote sensing images[J].Remote Sensing,2020,12(3):389.
[47]KHOSHBORESH M M,SHAH-HOSSEINI R.A hybrid deep learning-based model for automatic car extraction from high-re-solution airborne imagery[J].Applied Geomatics,2020,12(2):107-119.
[48]AUDEBERT N,LE SAUX B,LEFÈVRE S.Segment-before-detect:Vehicle detection and classification through semantic segmentation of aerial images[J].Remote Sensing,2017,9(4):368.
[49]YANG X,HOU L,ZHOU Y,et al.Dense label encoding forboundary discontinuity free rotation detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Montreal,Canada:ICCV,2021:15819-15829.
[50]WANG H,HUANG Z,CHEN Z,et al.Multi-Grained AngleRepresentation for Remote Sensing Object Detection[J].arXiv:2209.02884,2022.
[51]YANG X,YAN J,YANG X,et al.SCRDet++:DetectingSmall,Cluttered and Rotated Objects via Instance-Level Feature Denoising and Rotation Loss Smoothing[J].arXiv:2004.13316,2020.
[52]CHEN Z,CHEN K,LIN W,et al.PIoU Loss:Towards Accurate Oriented Object Detection in Complex Environments[C]//European Conference on Computer Vision.Berlin:Springer,2020:195-211.
[53]ZHANG L,WANG H,WANG L,et al.Constraint Loss for Rotated Object Detection in Remote Sensing Images[J].Remote Sensing,2021,13(21):4291.
[54]HAN J,DING J,LI J,et al.Align deep features for oriented object detection[J].IEEE Transactions on Geoscience and Remote Sensing,2021,60:1-11.
[55]YANG X,ZHOU Y,ZHANG G,et al.The KFIoU Loss for Rotated Object Detection[J].arXiv:2201.12558,2022.
[56]HE W,ZHANG X Y,YIN F,et al.Deep direct regression for multi-oriented scene text detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice,Italy:ICCV,2017:745-753.
[57]ZHOU X,YAO C,WEN H,et al.East:an efficient and accurate scene text detector[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Venice,Italy:ICCV,2017:5551-5560.
[58]ZHANG C,LIANG B,HUANG Z,et al.Look more than once:An accurate detector for text of arbitrary shapes[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seoul,Korea(south):ICCV,2019:10552-10561.
[59]FENG P,LIN Y,GUAN J,et al.Toso:Student’s distribution aided one-stage orientation target detection in remote sensing images[C]//2020 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2020).Barcelona,Spain:IEEE,2020:4057-4061.
[60]LIU Y,HE T,CHEN H,et al.Exploring the capacity of an orderless boxdiscretization network for multi-orientation scene text detection[J].International Journal of Computer Vision,2021,129(6):1972-1992.
[61]WEI H,ZHANG Y,CHANG Z,et al.Oriented objects as pairs of middle lines[J].ISPRS Journal of Photogrammetry and Remote Sensing,2020,169:268-279.
[62]ZHOU L,WEI H,LI H,et al.Arbitrary-oriented object detection in remote sensing images based on polar coordinates[J].IEEE Access,2020,8:223373-223384.
[63]GUO Z,ZHANG X,LIU C,et al.Convex-hull Feature Adaptation for Oriented and Densely Packed Object Detection[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,32(8):5252-5265.
[64]LI W,CHEN Y,HU K,et al.Oriented reppoints for aerial object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans,Louisiana:CVPR,2022:1829-1838.
[65]ZHOU Q,YU C.Point RCNN:An Angle-Free Framework forRotated Object Detection[J].Remote Sensing,2022,14(11):2605.
[66]BRADSKI G.The OpenCV library[J].Dr.Dobb’s Journal:Software Tools for the Professional Programmer,2000,25(11):120-123.
[67]HOU L,LU K,YANG X,et al.G-Rep:Gaussian Representation for Arbitrary-Oriented Object Detection[J].arXiv:2205.11796,2022.
[68]CHEN H B,JIANG S,HE G,et al.TEANS:A target enhancement and attenuated nonmaximum suppression object detector for remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2020,18(4):632-636.
[69]HOU L P,LU K,XUE J,et al.Cascade detector with featurefusion for arbitrary-oriented objects in remote sensing images[C]//Proceedings of 2020 IEEE International Conference on Multimedia and Expo.Piscataway:IEEE Press,2020:1-6.
[70]LU X,JI J,XING Z,et al.Attention and feature fusion SSD for remote sensing object detection[J].IEEE Transactions on Instrumentation and Measurement,2021,70:1-9.
[71]FU K,CHANG Z,ZHANG Y,et al.Rotation-aware and multi-scale convolutional neural network for object detection in remote sensing images[J].ISPRS Journal of Photogrammetry and Remote Sensing,2020,161:294-308.
[72]DENG Z,SUN H,ZHOU S,et al.Multi-scale object detection in remote sensing imagery with convolutional neural networks[J].ISPRS Journal of Photogrammetry and Remote Sensing,2018,145:3-22.
[73]DONG Z,WANG M,WANG Y,et al.Object detection in high resolution remote sensing imagery based on convolutional neural networks with suitable object scale features[J].IEEE Transactions on Geoscience and Remote Sensing,2019,58(3):2104-2114.
[74]WANG C,BAI X,WANG S,et al.Multiscale visual attention networks for object detection in VHR remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2018,16(2):310-314.
[75]XIA G S,BAI X,DING J,et al.DOTA:A large-scale dataset for object detection in aerial images[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT,USA:CVPR,2018:3974-3983.
[76]ZHU H,CHEN X,DAI W,et al.Orientation robust object detection in aerial images using deep convolutional neural network[C]//2015 IEEE International Conference on Image Processing (ICIP).Quebec City,QC,Canada:IEEE,2015:3735-3739.
[77]LIU Z,WANG H,WENG L,et al.Ship rotated bounding boxspace for ship extraction from high-resolution optical satellite images with complex backgrounds[J].IEEE Geoscience and Remote Sensing Letters,2016,13(8):1074-1078.
[78]XU C A,SU H,LI J W,et al.RSDD-SAR:SAR ship oblique frame detection dataset [J].Journal of Radar,2022,11(4):581-599.
[79]ZHANG T,ZHANG X,LI J,et al.Sar ship detection dataset(ssdd):Official release and comprehensive data analysis[J].Remote Sensing,2021,13(18):3690.
[80]KARATZAS D,GOMEZ-BIGORDA L,NICOLAOU A,et al.ICDAR 2015 competition on robust reading[C]//2015 13th International Conference on Document Analysis and Recognition(ICDAR).Tunis,Tunisia:IEEE,2015:1156-1160.
[81]YAO C,BAI X,LIU W,et al.Detecting texts of arbitrary orientations in natural images[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition.Providence,RI,USA:IEEE,2012:1083-1090.
[82]VEIT A,MATERA T,NEUMANN L,et al.Coco-text:Dataset and benchmark for text detection and recognition in natural images[J].arXiv:1601.07140,2016.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Survey of Rotating Object Detection Research in Computer Vision

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[2]	SONG Xinyang, YAN Zhiyuan, SUN Muyi, DAI Linlin, LI Qi, SUN Zhenan. Review of Talking Face Generation [J]. Computer Science, 2023, 50(8): 68-78.
[3]	ZHOU Ziyi, XIONG Hailing. Image Captioning Optimization Strategy Based on Deep Learning [J]. Computer Science, 2023, 50(8): 99-110.
[4]	ZHANG Xiao, DONG Hongbin. Lightweight Multi-view Stereo Integrating Coarse Cost Volume and Bilateral Grid [J]. Computer Science, 2023, 50(8): 125-132.
[5]	WANG Yu, WANG Zuchao, PAN Rui. Survey of DGA Domain Name Detection Based on Character Feature [J]. Computer Science, 2023, 50(8): 251-259.
[6]	LI Kun, GUO Wei, ZHANG Fan, DU Jiayu, YANG Meiyue. Adversarial Malware Generation Method Based on Genetic Algorithm [J]. Computer Science, 2023, 50(7): 325-331.
[7]	WANG Mingxia, XIONG Yun. Disease Diagnosis Prediction Algorithm Based on Contrastive Learning [J]. Computer Science, 2023, 50(7): 46-52.
[8]	SHEN Zhehui, WANG Kailai, KONG Xiangjie. Exploring Station Spatio-Temporal Mobility Pattern:A Short and Long-term Traffic Prediction Framework [J]. Computer Science, 2023, 50(7): 98-106.
[9]	HUO Weile, JING Tao, REN Shuang. Review of 3D Object Detection for Autonomous Driving [J]. Computer Science, 2023, 50(7): 107-118.
[10]	ZHOU Bo, JIANG Peifeng, DUAN Chang, LUO Yuetong. Study on Single Background Object Detection Oriented Improved-RetinaNet Model and Its Application [J]. Computer Science, 2023, 50(7): 137-142.
[11]	MAO Huihui, ZHAO Xiaole, DU Shengdong, TENG Fei, LI Tianrui. Short-term Subway Passenger Flow Forecasting Based on Graphical Embedding of Temporal Knowledge [J]. Computer Science, 2023, 50(7): 213-220.
[12]	LI Yuqiang, LI Linfeng, ZHU Hao, HOU Mengshu. Deep Learning-based Algorithm for Active IPv6 Address Prediction [J]. Computer Science, 2023, 50(7): 261-269.
[13]	HAN Junling, LI Bo, KANG Xiaodong, YANG Jingyi, LIU Hanqing, WANG Xiaotian. Cardiac MRI Image Segmentation Based on Faster R-CNN and U-net [J]. Computer Science, 2023, 50(6A): 220600047-9.
[14]	LIU Haowei, YAO Jingchi, LIU Bo, BI Xiuli, XIAO Bin. Two-stage Method for Restoration of Heritage Images Based on Muti-scale Attention Mechanism [J]. Computer Science, 2023, 50(6A): 220600129-8.
[15]	QI Xuanlong, CHEN Hongyang, ZHAO Wenbing, ZHAO Di, GAO Jingyang. Study on BGA Packaging Void Rate Detection Based on Active Learning and U-Net++ Segmentation [J]. Computer Science, 2023, 50(6A): 220200092-6.