计算机视觉下的旋转目标检测研究综述

doi:10.11896/jsjkx.221000148

摘要/Abstract

摘要： 传统目标检测器通过水平边界框(Horizontal Bounding Box,HBB)定位目标,在检测方向角任意、分布密集、长宽比大、背景复杂的目标时,往往精度较低、泛化能力较差。在边界框中增加不同旋转角度的旋转目标框可有效解决上述问题,其被广泛应用在遥感图像、场景文本图像、货架商品图像等目标检测领域,具有重要研究价值。目前大多数工作旨在构建不同的旋转目标检测模型,对现有模型的归纳总结及深入分析的综述性工作较少。为此,对旋转目标检测现有研究成果进行了详细综述。首先根据当前流行的目标框表征方式,将目标框分为旋转矩形框(Oriented Bounding Box,OBB)、四边形边界框(Quadrilateral Bounding Box,QBB)和点集(Point set) 3种类型,并比较了不同旋转目标检测算法的优缺点、网络结构和性能;其次分析了目前常用的旋转目标检测数据集和性能评价指标;最后对目前研究中存在的问题进行简要总结和讨论,并对未来的发展趋势进行展望。

关键词: 计算机视觉, 深度学习, 目标检测, 旋转目标, 性能比较

Abstract: Traditional object detector locates objects by horizontal bounding box(HBB),which often have low accuracy and poor generalization ability when detecting objects with arbitrary orientation angle,dense distribution,large aspect ratio and complex background.The above problems can be effectively solved by adding rotating target boxes with different rotation angles in the bounding boxes.This method is widely used in the fields of remote sensing images,scene text images,shelf goods images and other target detection,and has important research value.Most of the current works aim at constructing different models for rotating object detection,and there are fewer review works for summarizing and analyzing existing models in depth.Therefore,this paper provides a detailed review of existing research results on rotating object detection.Firstly,according to the current popular way of target box characterization,the target boxes are classified into three types of oriented bounding box(OBB),quadrilateral bounding boxes(QBB) and point set for generalized analysis,and simultaneously compare the advantages and disadvantages,network structures and performance of different rotating object detection algorithms.Secondly,the commonly used rotating object detection datasets and performance evaluation metrics are analyzed.Finally,the problems in the current study are briefly summarized and discussed,and the future development trend is prospected.

Key words: Computer vision, Deep learning, Object detection, Rotating object, Performance comparison

中图分类号:

TP311

王旭, 吴艳霞, 张雪, 洪瑞泽, 李广生. 计算机视觉下的旋转目标检测研究综述[J]. 计算机科学, 2023, 50(8): 79-92. https://doi.org/10.11896/jsjkx.221000148

WANG Xu, WU Yanxia, ZHANG Xue, HONG Ruize, LI Guangsheng. Survey of Rotating Object Detection Research in Computer Vision[J]. Computer Science, 2023, 50(8): 79-92. https://doi.org/10.11896/jsjkx.221000148

参考文献

[1]GIRSHICK R.Fast R-CNN [C]//GIRSHICK R.GIRSHICK R[C]//IEEE International Conference on Computer Vision.Piscataway:IEEE Press,2015:1440-1448.
[2]REN S,HE K,GIRSHICK R,et al.Faster r-cnn:Towards real-time object detection with region proposal networks[J].Advances in Neural Information Processing Systems,2015,28:91-99.
[3]HE K,ZHANG X,REN S,et al.Spatial pyramid pooling in deepconvolutional networks for visual recognition[J].IEEE Tran-sactions on Pattern Analysis and Machine Intelligence,2015,37(9):1904-1916.
[4]REDMON J,DIVVALA S,GIRSHICK R,et al.You only look once:Unified real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas,NV,USA:IEEE Comp. Soc.,2016:779-788.
[5]LIU W,ANGUELOV D,ERHAN D,et al.Ssd:Single shotmultibox detector[C]//European Conference on Computer Vision.Berlin:Springer,2016:21-37.
[6]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice,Italy:ICCV,2017:2980-2988.
[7]TIAN Z,HUANG W,HE T,et al.Detecting text in naturalimage with connectionisttext proposal network[C]//European Conference on Computer Vision.Berlin:Springer,2016:56-72.
[8]LIAO M,SHI B,BAI X,et al.TextBoxes:A Fast Text Detector with a Single Deep Neural Network[C]//Proceedings of the AAAI Conference on Artificial Intelligence.San Francisco,USA:ACM,2017,31(1):4161-4167.
[9]SHEN Y,LIU D,ZHANG F,et al.Fast and accurate multi-class geospatial object detection with large-size remote sensing ima-gery using CNN and Truncated NMS[J].ISPRS Journal of Photogrammetry and Remote Sensing,2022,191:235-249.
[10]SHI P,ZHAO Z,FAN X,et al.Remote Sensing Image Object Detection Based on Angle Classification[J].IEEE Access,2021,9:118696-118707.
[11]ZHANG L,ZHANG Y S,YU Y,et al.Survey on object detection in tilting box for remote sensing images[J].National Remote Sensing Bulletin,2022,26(9):1723-1743.
[12]MA J,SHAO W,YE H,et al.Arbitrary-oriented scene text detection via rotation proposals[J].IEEE Transactions on Multimedia,2018,20(11):3111-3122.
[13]YANG X,YAN J,FENG Z,et al.R3det:Refined single-stagedetector with feature refinement for rotating object[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.Online:AAAI,2021,35(4):3163-3171.
[14]ZHANG G,LU S,ZHANG W.CAD-Net:Acontext-aware de-tection network for objects in remote sensing imagery[J].IEEE Transactions on Geoscience and Remote Sensing,2019,57(12):10015-10024.
[15]MING Q,MIAO L,ZHOU Z,et al.Task interleaving and orientation estimation for high-precision oriented object detection in aerial images[J].ISPRS Journal of Photogrammetry and Remote Sensing,2023,196:241-255.
[16]YANG X,SUN H,SUN X,et al.Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network[J].IEEE Access,2018,6:50839-50849.
[17]SHI X,SHAN S,KAN M,et al.Real-time rotation-invariantface detection with progressive calibration networks[C]//Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition.Salt Lake City,UT,USA:IEEE,2018:2295-2303.
[18]LIU T,LI W G,GUAN J H.A Review of Object DetectionMethods in Optical Remote Sensing Image Based on Deep Learning [J].Radio Communication Technology,2020,46(6):624-634.
[19]NIE G T,HUANG H.A survey of object detection in optical re-mote sensing images[J].Acta Automatica Sinica,2021,47(8):1749-1768.
[20]SONG Z N,SUI H G,LI Y C.A survey on ship detection technology in high-resolution optical remote sensing images[J].Geomatics and Information Science of Wuhan University,2021,46(11):1703-1715.
[21]WANG J X,WANG Z Y,TIAN X.A Review of Text Detection and Recognition in Natural Scenes Based on Deep Learning [J].Journal of Software,2020,31(5):1465-1496.
[22]ZHAO T Z,YANG C W,LIU W.Remote sensing image target detection method based on non-local feature enhancement [J].Journal of Huazhong University of Science and Technology(Natural Science Edition),2021,49(9):47-51.
[23]SCHILLING H,BULATOV D,NIESSNER R,et al.Detection of vehicles in multisensor data via multibranch convolutional neural networks[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2018,11(11):4299-4316.
[24]LIAO M,SHI B,BAI X.Textboxes++:A single-shot oriented scene text detector[J].IEEE Transactions on Image Processing,2018,27(8):3676-3690.
[25]XU Y,FU M,WANG Q,et al.Gliding vertex on the horizontal bounding box for multi-oriented object detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(4):1452-1459.
[26]SUN P,ZHENG Y,ZHOU Z,et al.R4 Det:Refined single-stage detector with feature recursion and refinement for rotating object detection in aerial images[J].Image and Vision Computing,2020,103:104036.
[27]QIAN W,YANG X,PENG S,et al.Learning modulated loss for rotated object detection[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.Online:AAAI,2021,35(3):2458-2466.
[28]YANG X,YAN J,MING Q,et al.Rethinking rotated object detection with gaussian wasserstein distance loss[C]//International Conference on Machine Learning.PMLR,2021:11830-11841.
[29]MING Q,MIAO L,ZHOU Z,et al.Optimization for arbitrary-oriented object detection via representation invariance loss[J].IEEE Geoscience and Remote Sensing Letters,2021,19:1-5.
[30]LAW H,DENG J.Cornernet:Detecting objects as paired key-points[C]//Proceedings of the European Conference on Computer Vision(ECCV).Munich:ECCV,2018:734-750.
[31]YANG Z,LIU S,HU H,et al.Reppoints:Point set representation for object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul,Korea (South):IEEE,2019:9657-9666.
[32]HOU L,LU K,YANG X,et al.G-Rep:Gaussian Representation for Arbitrary-Oriented Object Detection[J].arXiv:2205.11796,2022.
[33]YANG X,YAN J.Arbitrary-oriented object detection with circular smooth label[C]//European Conference on Computer Vision.Berlin:Springer,2020:677-694.
[34]DING J,XUE N,LONG Y,et al.Learning RoI transformer for oriented object detection in aerial images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seoul,Korea (South):ICCV,2019:2849-2858.
[35]HAN J,DING J,XUE N,et al.Redet:A rotation-equivariant detector for aerial object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Montreal Canada:ICCV,2021:2786-2795.
[36]MA J,SHAO W,YE H,et al.Arbitrary-oriented scene text detection via rotation proposals[J].IEEE Transactions on Multimedia,2018,20(11):3111-3122.
[37]XIE X,CHENG G,WANG J,et al.Oriented R-CNN for object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Montreal Canada:ICCV,2021:3520-3529.
[38]YANG X,YANG J,YAN J,et al.Scrdet:Towards more robust detection for small,cluttered and rotated objects[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul,Korea (South):ICCV,2019:8232-8241.
[39]LYU C,ZHANG W,HUANG H,et al.Rtmdet: An empirical study of designing real-time object detectors[J].arXiv:2212.07784,2022.
[40]AN S B,LOU H R,CHEN S W,et al.Research progress of rotating target detection method based on deep learning [J].Electronic Measurement Technology,2021,44(21):168-178.
[41]JIANG Y,ZHU X,WANG X,et al.R2CNN:Rotational region CNN for orientation robust scene text detection[J].arXiv:1706.09579,2017.
[42]YANG X,SUN H,FU K,et al.Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks[J].Remote Sensing,2018,10(1):132.
[43]ZHU Y,DU J,WU X.Adaptive period embedding for representing oriented objects in aerial images[J].IEEE Transactions on Geoscience and Remote Sensing,2020,58(10):7247-7257.
[44]AZIMI S M,VIG E,BAHMANYAR R,et al.Towards multi-class object detection in unconstrained remote sensing imagery[C]//Asian Conference on Computer Vision.Berlin:Springer,2018:150-165.
[45]WANG J,DING J,GUO H,et al.Mask OBB:A semantic attention-based mask oriented bounding box representation for multi-category object detection in aerial images[J].Remote Sensing,2019,11(24):2930.
[46]LI Y,HUANG Q,PEI X,et al.RADet:Refine feature pyramid network and multi-layer attention network for arbitrary-oriented object detection of remote sensing images[J].Remote Sensing,2020,12(3):389.
[47]KHOSHBORESH M M,SHAH-HOSSEINI R.A hybrid deep learning-based model for automatic car extraction from high-re-solution airborne imagery[J].Applied Geomatics,2020,12(2):107-119.
[48]AUDEBERT N,LE SAUX B,LEFÈVRE S.Segment-before-detect:Vehicle detection and classification through semantic segmentation of aerial images[J].Remote Sensing,2017,9(4):368.
[49]YANG X,HOU L,ZHOU Y,et al.Dense label encoding forboundary discontinuity free rotation detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Montreal,Canada:ICCV,2021:15819-15829.
[50]WANG H,HUANG Z,CHEN Z,et al.Multi-Grained AngleRepresentation for Remote Sensing Object Detection[J].arXiv:2209.02884,2022.
[51]YANG X,YAN J,YANG X,et al.SCRDet++:DetectingSmall,Cluttered and Rotated Objects via Instance-Level Feature Denoising and Rotation Loss Smoothing[J].arXiv:2004.13316,2020.
[52]CHEN Z,CHEN K,LIN W,et al.PIoU Loss:Towards Accurate Oriented Object Detection in Complex Environments[C]//European Conference on Computer Vision.Berlin:Springer,2020:195-211.
[53]ZHANG L,WANG H,WANG L,et al.Constraint Loss for Rotated Object Detection in Remote Sensing Images[J].Remote Sensing,2021,13(21):4291.
[54]HAN J,DING J,LI J,et al.Align deep features for oriented object detection[J].IEEE Transactions on Geoscience and Remote Sensing,2021,60:1-11.
[55]YANG X,ZHOU Y,ZHANG G,et al.The KFIoU Loss for Rotated Object Detection[J].arXiv:2201.12558,2022.
[56]HE W,ZHANG X Y,YIN F,et al.Deep direct regression for multi-oriented scene text detection[C]//Proceedings of the IEEE International Conference on Computer Vision.Venice,Italy:ICCV,2017:745-753.
[57]ZHOU X,YAO C,WEN H,et al.East:an efficient and accurate scene text detector[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Venice,Italy:ICCV,2017:5551-5560.
[58]ZHANG C,LIANG B,HUANG Z,et al.Look more than once:An accurate detector for text of arbitrary shapes[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seoul,Korea(south):ICCV,2019:10552-10561.
[59]FENG P,LIN Y,GUAN J,et al.Toso:Student’s distribution aided one-stage orientation target detection in remote sensing images[C]//2020 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2020).Barcelona,Spain:IEEE,2020:4057-4061.
[60]LIU Y,HE T,CHEN H,et al.Exploring the capacity of an orderless boxdiscretization network for multi-orientation scene text detection[J].International Journal of Computer Vision,2021,129(6):1972-1992.
[61]WEI H,ZHANG Y,CHANG Z,et al.Oriented objects as pairs of middle lines[J].ISPRS Journal of Photogrammetry and Remote Sensing,2020,169:268-279.
[62]ZHOU L,WEI H,LI H,et al.Arbitrary-oriented object detection in remote sensing images based on polar coordinates[J].IEEE Access,2020,8:223373-223384.
[63]GUO Z,ZHANG X,LIU C,et al.Convex-hull Feature Adaptation for Oriented and Densely Packed Object Detection[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,32(8):5252-5265.
[64]LI W,CHEN Y,HU K,et al.Oriented reppoints for aerial object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans,Louisiana:CVPR,2022:1829-1838.
[65]ZHOU Q,YU C.Point RCNN:An Angle-Free Framework forRotated Object Detection[J].Remote Sensing,2022,14(11):2605.
[66]BRADSKI G.The OpenCV library[J].Dr.Dobb’s Journal:Software Tools for the Professional Programmer,2000,25(11):120-123.
[67]HOU L,LU K,YANG X,et al.G-Rep:Gaussian Representation for Arbitrary-Oriented Object Detection[J].arXiv:2205.11796,2022.
[68]CHEN H B,JIANG S,HE G,et al.TEANS:A target enhancement and attenuated nonmaximum suppression object detector for remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2020,18(4):632-636.
[69]HOU L P,LU K,XUE J,et al.Cascade detector with featurefusion for arbitrary-oriented objects in remote sensing images[C]//Proceedings of 2020 IEEE International Conference on Multimedia and Expo.Piscataway:IEEE Press,2020:1-6.
[70]LU X,JI J,XING Z,et al.Attention and feature fusion SSD for remote sensing object detection[J].IEEE Transactions on Instrumentation and Measurement,2021,70:1-9.
[71]FU K,CHANG Z,ZHANG Y,et al.Rotation-aware and multi-scale convolutional neural network for object detection in remote sensing images[J].ISPRS Journal of Photogrammetry and Remote Sensing,2020,161:294-308.
[72]DENG Z,SUN H,ZHOU S,et al.Multi-scale object detection in remote sensing imagery with convolutional neural networks[J].ISPRS Journal of Photogrammetry and Remote Sensing,2018,145:3-22.
[73]DONG Z,WANG M,WANG Y,et al.Object detection in high resolution remote sensing imagery based on convolutional neural networks with suitable object scale features[J].IEEE Transactions on Geoscience and Remote Sensing,2019,58(3):2104-2114.
[74]WANG C,BAI X,WANG S,et al.Multiscale visual attention networks for object detection in VHR remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2018,16(2):310-314.
[75]XIA G S,BAI X,DING J,et al.DOTA:A large-scale dataset for object detection in aerial images[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT,USA:CVPR,2018:3974-3983.
[76]ZHU H,CHEN X,DAI W,et al.Orientation robust object detection in aerial images using deep convolutional neural network[C]//2015 IEEE International Conference on Image Processing (ICIP).Quebec City,QC,Canada:IEEE,2015:3735-3739.
[77]LIU Z,WANG H,WENG L,et al.Ship rotated bounding boxspace for ship extraction from high-resolution optical satellite images with complex backgrounds[J].IEEE Geoscience and Remote Sensing Letters,2016,13(8):1074-1078.
[78]XU C A,SU H,LI J W,et al.RSDD-SAR:SAR ship oblique frame detection dataset [J].Journal of Radar,2022,11(4):581-599.
[79]ZHANG T,ZHANG X,LI J,et al.Sar ship detection dataset(ssdd):Official release and comprehensive data analysis[J].Remote Sensing,2021,13(18):3690.
[80]KARATZAS D,GOMEZ-BIGORDA L,NICOLAOU A,et al.ICDAR 2015 competition on robust reading[C]//2015 13th International Conference on Document Analysis and Recognition(ICDAR).Tunis,Tunisia:IEEE,2015:1156-1160.
[81]YAO C,BAI X,LIU W,et al.Detecting texts of arbitrary orientations in natural images[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition.Providence,RI,USA:IEEE,2012:1083-1090.
[82]VEIT A,MATERA T,NEUMANN L,et al.Coco-text:Dataset and benchmark for text detection and recognition in natural images[J].arXiv:1601.07140,2016.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed