基于规则推理的足球视频任意球射门事件检测

doi:10.11896/jsjkx.220300062

摘要/Abstract

摘要： 足球视频事件检测对视频检索具有重要意义。然而,足球视频中事件较少,且主要发生在远镜头中,难以捕捉关键球员和关键动作,导致足球事件检测困难。近年来,基于深度学习的方法在足球视频事件检测上取得了一定的进展,但对事件的高层语义学习仍不够充分,检测结果有待进一步提高。如何提升足球视频事件检测的准确性是亟待解决的问题。以任意球射门事件为研究对象,提出了足球规则与深度学习相结合的事件检测模型。为了深入了解任意球射门事件的内在特性,人工总结了事件规则并在公共足球数据集上进行了验证,同时提出了规则的应用场景。针对足球视频中事件过少的问题,设计了基于规则的初始定位算法对视频进行预处理。通过多规则组合和应用,从原始视频中初步定位可能发生任意球射门事件的位置,并将其作为深度学习模型的输入进行进一步预测。在公共足球数据集上将所提模型与其他模型进行对比实验。结果表明,该模型取得了最好的效果,其精确率达到78%,召回率达到81.25%。相比其他模型,其精确率的提升尤为明显。可见,足球规则与深度学习相结合的任意球事件检测模型有效提升了任意球射门事件的检测性能,为足球视频中其他事件的检测提供了参考依据。

关键词: 足球视频, 任意球射门, 事件检测, 足球规则, 深度学习

Abstract: Soccer video event detection is of great significance to video retrieval.However,there are fewer events in soccer videos,and these events mainly occur in the far-view shot,which makes it difficult to capture key players and key actions,making soccer event detection more difficult.In recent years,methods based on deep learning have made some progress in soccer video event detection,but the learning ability of the high-level semantic of the event is still insufficient and the detection results need to be further improved.Therefore,how to improve the accuracy of soccer video event detection is an urgent problem to be solved.Taking the shooting event of free kick(free-kick shot event) as the research object,an event detection model combining soccer rules and deep learning is proposed.To have a deeper understanding of the inherent characteristics of the free-kick shot event,the event rules are manually summarized and verified on the public soccer dataset,and the corresponding application scenarios are also proposed.For the problem of too few events in soccer videos,rule-based initial localization algorithm is proposed to preprocess the videos.Through the combination and application of multiple rules,the location where the free-kick shot event may occur is initially located from the original video,which is used as the input of the deep learning model for further prediction.The proposed mo-del is compared with other models on the public soccer dataset.Experimental results show that the proposed model achieves the best results,with the accuracy rate of 78% and the recall rate of 81.25%.Compared with other models,the improvement in accuracy is particularly prominent.It can be seen that the free-kick shot event detection model that combines soccer rules and deep learning effectively improves the performance of free-kick shot event detection and provides a basic reference for further research on the detection of other events in soccer videos.

Key words: Soccer video, Free-kick shot, Event detection, Soccer rules, Deep learning

中图分类号:

TP181

华晓凤, 冯娜, 于俊清, 何云峰. 基于规则推理的足球视频任意球射门事件检测[J]. 计算机科学, 2023, 50(3): 181-190. https://doi.org/10.11896/jsjkx.220300062

HUA Xiaofeng, FENG Na, YU Junqing, HE Yunfeng. Shooting Event Detection of Free Kick in Soccer Video Based on Rule Reasoning[J]. Computer Science, 2023, 50(3): 181-190. https://doi.org/10.11896/jsjkx.220300062

参考文献

[1] YE Q,HUANG Q,GAO W,et al.Exciting event detection in broadcast soccer video with mid-level description and incremental learning[C]//Proceedings of the 13th Annual ACM International Conference on Multimedia.New York:ACM,2005:455-458.
[2] TONG X,LU H,LIU Q.A three-layer event detection framework and its application in soccer video[C]//Proceedings of the International Conference on Multimedia & Expo.Piscataway,NJ:IEEE,2004:1551-1554.
[3] WANG F,MA Y F,ZHANG H,et al.Dynamic bayesian network based event detection for soccer highlight extraction[C]//Proceedings of the 2004 International Conference on Image Processing.Singapore:IEEE,2004:633-636.
[4] XU H,FONG T H,CHUA T S.Fusion of Multiple Asyn-chronous Information Sources for Event Detection in Soccer Video[C]//Proceedings of the 2005 IEEE International Confe-rence on Multimedia and Expo.Piscataway,NJ:IEEE,2005:1242-1245.
[5] XIONG Z,RADHAKRISHNAN R,DIVAKARAN A,et al.Audio events detection based highlights extraction from baseball,golf and soccer games in a unified framework[C]//Proceedings of the 2003 International Conference on Multimedia and Expo.Piscataway,NJ:IEEE,2003:401-404.
[6] JIN G,TAO L,XU G.Hidden Markov Model Based Events Detection in Soccer Video[C]//Proceedings of the International Conference Image Analysis and Recognition.Berlin,German:Springer.2004:605-612.
[7] HAN Y,LIU G,CHOLLET G.Goal event detection in broadcast soccer videos by combining heuristic rules with unsupervised fuzzy c-means algorithm[C]//Proceedings of the 10th International Conference on Control,Automation,Robotics and Vision.Hanoi,Vietnam:IEEE,2008:888-891.
[8] XU M,MADDAGE N C,XU C,et al.Creating audio keywords for event detection in soccer video[C]//Proceedings of the 2003 International Conference on Multimedia and Expo.Piscataway,NJ:IEEE,2003:281-284.
[9] LIU H,HE T.Integrating Multiple Feature Fusion for Semantic Event Detection in Soccer Video[C]//Proceedings of the 2009 International Joint Conference on Artificial Intelligence.Hai-nan,China:IEEE,2009:128-131.
[10] HOSSEINI M S,EFTEKHARI-MOGHADAM A M.Fuzzyrule-based reasoning approach for event detection and annotation of broadcast soccer video [J].Applied Soft Computing,2013,13(2):846-866.
[11] PANDYA D S,ZAVERI M A.Frame based approach for automatic event boundary detection of soccer video using optical flow[C]//Proceedings of the 2017 IEEE International Conference on Signal and Image Processing Applications.Kuching,Malaysia:IEEE,2017:402-406.
[12] FAKHAR B,KANAN H R,BEHRAD A.Learning an event-oriented and discriminative dictionary based on an adaptive label-consistent K-SVD method for event detection in soccer videos [J].Journal of Visual Communication and Image Representation,2018,55:489-503.
[13] MORRA L,MANIGRASSO F,CANTO G,et al.Slicing and di-cing soccer:automatic detection of complex events from spatio-temporal data[C]//Proceedings of the International Conference on Image Analysis and Recognition.Cham:Springer,2020:107-121.
[14] GAO X,LIU X,YANG T,et al.Automatic Key Moment Extraction and Highlights Generation Based on Comprehensive Soccer Video Understanding[C]//Proceedings of the IEEE International Conference on Multimedia & Expo Workshops.London,UK:IEEE,2020:1-6.
[15] TOMEI M,BARALDI L,CALDERARA S,et al.RMS-Net:Regression and Masking for Soccer Event Spotting[C]//Procee-dings of the 25th International Conference on Pattern Recognition.Milan,Italy:IEEE,2021:7699-7706.
[16] CIOPPA A,DELIÈGE A,GIANCOLA S,et al.A Context-Aware Loss Function for Action Spotting in Soccer Videos[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,WA,USA:IEEE,2020:13123-13133.
[17] JIANG H,LU Y,XUE J.Automatic Soccer Video Event Detection Based on a Deep Neural Network Combined CNN and RNN[C]//Proceedings of the 28th International Conference on Tools with Artificial Intelligence.San Jose,CA,USA:IEEE,2016:490-494.
[18] SONG C,RASMUSSEN C.Multi-camera Temporal Groupingfor Play/Break Event Detection in Soccer Games[C]//Procee-dings of the International Symposium on Visual Computing.Cham:Springer,2019:231-243.
[19] FENG N,SONG Z,YU J,et al.SSET:a dataset for shot segmentation,event detection,player tracking in soccer videos [J].Multimedia Tools and Applications,2020,79(39):28971-28992.
[20] WANG T,FENG N,YU J,et al.Shot Boundary DetectionThrough Multi-stage Deep Convolution Neural Network[C]//Proceedings of the International Conference on Multimedia Modeling.Cham:Springer,2019:456-468.
[21] SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition [J].arXiv:1409.1556,2014.
[22] TRAN D,BOURDEV L D,FERGUS R,et al.Learning Spatiotemporal Features with 3D Convolutional Networks[C]//Proceedings of the 2015 IEEE International Conference on Compu-ter Vision.Piscataway,NJ:IEEE,2015:4489-4497.
[23] WU L,YU J,HE Y,et al.Soccer video shot classification and playfield zone detection [J].Journal of Image and Graphics,2017,22(2):265-272.
[24] CHENG P,FENG J,MA H J,et al.Soccer Video Scene Classifi-cation Algorithms Based on C3D [J].Computer System Application,2019,28(12):158-164.
[25] SARKAR S,ALI S,CHAKRABARTI A.Shot Classification and Replay Detection in Broadcast Soccer Video[C]//Procee-dings of the Advanced Computing and Systems for Security.Singapore:Springer,2020:57-66.
[26] YU J,LEI A,HU Y.Soccer Video Event Detection Based on Deep Learning[C]//Proceedings of the International Confe-rence on Multimedia Modeling.Cham:Springer,2019:377-389.
[27] VANDERPLAETSE B,DUPONT S.Improved Soccer Action Spotting using both Audio and Video Streams[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Seattle,WA,USA:IEEE,2020:3921-3931.

相关文章 15

[1]	董永峰, 黄港, 薛婉若, 李林昊. 融合IRT的图注意力深度知识追踪模型 Graph Attention Deep Knowledge Tracing Model Integrated with IRT 计算机科学, 2023, 50(3): 173-180. https://doi.org/10.11896/jsjkx.211200134
[2]	梅鹏程, 杨吉斌, 张强, 黄翔. 一种基于三维卷积的声学事件联合估计方法 Sound Event Joint Estimation Method Based on Three-dimension Convolution 计算机科学, 2023, 50(3): 191-198. https://doi.org/10.11896/jsjkx.220500259
[3]	白雪飞, 马亚楠, 王文剑. 基于特征融合的边缘引导乳腺超声图像分割方法 Segmentation Method of Edge-guided Breast Ultrasound Images Based on Feature Fusion 计算机科学, 2023, 50(3): 199-207. https://doi.org/10.11896/jsjkx.211200294
[4]	刘航, 普园媛, 吕大华, 赵征鹏, 徐丹, 钱文华. 极化自注意力约束颜色溢出的图像自动上色 Polarized Self-attention Constrains Color Overflow in Automatic Coloring of Image 计算机科学, 2023, 50(3): 208-215. https://doi.org/10.11896/jsjkx.220100149
[5]	陈亮, 王璐, 李生春, 刘昌宏. 基于深度学习的可视化仪表板生成技术研究 Study on Visual Dashboard Generation Technology Based on Deep Learning 计算机科学, 2023, 50(3): 238-245. https://doi.org/10.11896/jsjkx.230100064
[6]	张译, 吴秦. 特征增强损失与前景注意力人群计数网络 Crowd Counting Network Based on Feature Enhancement Loss and Foreground Attention 计算机科学, 2023, 50(3): 246-253. https://doi.org/10.11896/jsjkx.220100219
[7]	应宗浩, 吴槟. 深度学习模型的后门攻击研究综述 Backdoor Attack on Deep Learning Models:A Survey 计算机科学, 2023, 50(3): 333-350. https://doi.org/10.11896/jsjkx.220600031
[8]	邹芸竹, 杜圣东, 滕飞, 李天瑞. 一种基于多模态深度特征融合的视觉问答模型 Visual Question Answering Model Based on Multi-modal Deep Feature Fusion 计算机科学, 2023, 50(2): 123-129. https://doi.org/10.11896/jsjkx.211200303
[9]	王鹏宇, 台文鑫, 刘芳, 钟婷, 罗绪成, 周帆. 基于数据增强的自监督飞行航迹预测 Self-supervised Flight Trajectory Prediction Based on Data Augmentation 计算机科学, 2023, 50(2): 130-137. https://doi.org/10.11896/jsjkx.211200016
[10]	郭楠, 李婧源, 任曦. 基于深度学习的刚体位姿估计方法综述 Survey of Rigid Object Pose Estimation Algorithms Based on Deep Learning 计算机科学, 2023, 50(2): 178-189. https://doi.org/10.11896/jsjkx.211200164
[11]	李俊林, 欧阳智, 杜逆索. 基于改进区域候选网络的场景文本检测 Scene Text Detection with Improved Region Proposal Network 计算机科学, 2023, 50(2): 201-208. https://doi.org/10.11896/jsjkx.211000191
[12]	华杰, 刘学亮, 赵烨. 基于特征融合的小样本目标检测 Few-shot Object Detection Based on Feature Fusion 计算机科学, 2023, 50(2): 209-213. https://doi.org/10.11896/jsjkx.220500153
[13]	梁佳利, 华保健, 苏少博. 融合循环划分的张量指令生成优化 Tensor Instruction Generation Optimization Fusing with Loop Partitioning 计算机科学, 2023, 50(2): 374-383. https://doi.org/10.11896/jsjkx.220300147
[14]	蔡肖, 陈志华, 盛斌. 基于移位窗口金字塔Transformer的遥感图像目标检测 SPT:Swin Pyramid Transformer for Object Detection of Remote Sensing 计算机科学, 2023, 50(1): 105-113. https://doi.org/10.11896/jsjkx.211100208
[15]	王斌, 梁宇栋, 刘哲, 张超, 李德玉. 亮度自调节的无监督图像去雾与低光图像增强算法研究 Study on Unsupervised Image Dehazing and Low-light Image Enhancement Algorithms Based on Luminance Adjustment 计算机科学, 2023, 50(1): 123-130. https://doi.org/10.11896/jsjkx.211100058

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed