Computer Science ›› 2023, Vol. 50 ›› Issue (3): 246-253.doi: 10.11896/jsjkx.220100219

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Crowd Counting Network Based on Feature Enhancement Loss and Foreground Attention

ZHANG Yi1, WU Qin1,2   

  1. 1 School of Artificial Intelligence and Computer Science,Jiangnan University,Wuxi,Jiangsu 214122,China
    2 Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence,Jiangnan University,Wuxi,Jiangsu 214122, China
  • Received:2022-01-23 Revised:2022-10-04 Online:2023-03-15 Published:2023-03-15
  • About author:ZHANG Yi,born in 1996,master candidate,is a member of China Computer Federation.Her main research interests include pattern recognition and compu-ter vision.
    WU Qin,born in 1978,Ph.D,associate professor,is a member of China Computer Federation.Her main research interests include computer vision and pattern recognition.
  • Supported by:
    National Natural Science Foundation of China(61972180).

Abstract: Crowd counting aims to estimate the total number of people in an image and present its distribution accurately.The images in the relevant datasets usually involve a variety of scenes and include multiple people.To save labor,most datasets usually annotated each human head by a single point.However,the point labels cannot cover the full human head,which makes it difficult to converge the matching between the crowd feature and the distribution label,and the predicted values cannot be gathered in the foreground region,which seriously affects the density estimation map quality and count accuracy.To solve this problem,count loss is used to constrain the range of predictions on the full map,and a pixel-level distribution consistency loss is used to optimize the density map matching process.In addition,there are many background noises that are easily confused with crowd feature in complex scenes.In order to avoid the interference of false positive predictions on subsequent counting and density map estimation,a foreground segmentation module and feature enhancement loss are proposed to adaptively focus the foreground region and increase the contribution of human head features to the counts,so as to suppress background misjudgments.In addition,in order to make the network adapt to the multi-scale pattern of the human head better,up and down sampling operations are performed on each image to be trained to obtain the multi-scale pattern with the same object.Experiments on several datasets show that the proposed method achieves better or competitive results compared with state-of-the-art methods.

Key words: Crowd counting, Deep learning, Foreground segmentation, Background compensation, Density estimation

CLC Number: 

  • TP391.413
[1]LIN S F,CHEN J Y,CHAO H X.Estimation of number of people in crowded scenes using perspective transformation[J].IEEE Transactions on Systems,Man,and Cybernetics-Part A:Systems and Humans,2001,31(6):645-654.
[2]ZHAO T,NEVATIA R.Bayesian human segmentation incrowded situations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2003.
[3]LI M,ZHANG Z X,HUANG K Q,et al.Estimating the number of people in crowded scenes by MID based foreground segmentation and head-shoulder detection[C]//Proceedings of the IEEE International Conference on Pattern Recognition.IEEE,2008:1-4.
[4]GE W,COLLINS R T.Marked point processes for crowd coun-ting[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2009:2913-2920.
[5]CHAN A B,LIANG Z S J,VASCONCELOS N.Privacy preserving crowd monitoring:Counting people without people models or tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2008:1-7.
[6]RYAN D,DENMAN S,FOOKES C,et al.Crowd counting using multiple local features[C]//Digital Image Computing:Techniques and Applications.2009:81-88.
[7]CHEN K,GONG S,XIANG T,et al.Cumulative attribute space for age and crowd density estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2013:2467-2474.
[8]LIU B,VASCONCELOS N.Bayesian model adaptation forcrowd counts[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2015:4175-4183.
[9]WANG C,ZHANG H,YANG L,et al.Deep people counting in extremely dense crowds[C]//Proceedings of the 2015 ACM Multimedia Conference.ACM,2015:1299-1302.
[10]SHANG C,AI H,BAI B.End-to-end crowd counting via joint learning local and global count[C]//Proceedings of the IEEE International Conference on Image Processing.IEEE,2016:1215-1219.
[11]XIONG F,SHI X,YEUNG D Y.Spatiotemporal modeling for crowd counting in videos[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2017:5161-5169.
[12]SHI Z L,ZHANG L,LIU Y,et al.Crowd counting with deep negative correlation learning[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2018:5382-5390.
[13]LIU X L,WEIJER J,BAGDANOV A D.Leveraging unlabeled data for crowd counting by learning to rank[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:7661-7669.
[14]RANJAN V,LE H,HOAI M.Iterative crowd counting[C]//Proceedings of the European Conference on Computer Vision.Springer,2018:270-285.
[15]CAO X K,WANG Z P,ZHAO Y Y,et al.Scale aggregation network for accurate and efficient crowd counting[C]//Proceedings of the European Conference on Computer Vision.Springer,2018:734-750.
[16]IDREES H,TAYYAB M,ATHREY K,et al.Composition loss for counting,density map estimation and localization in dense crowds[C]//Proceedings of the European Conference on Computer Vision.Springer,2018:532-546.
[17]ZHANG Y Y,ZHOU D,CHEN S Q,et al.Single-image crowd counting via multi-column convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2016:589-597.
[18]WANG B Y,LIU H D,SAMARAS D,et al.Distribution Matching for Crowd Counting[C]//Proceedings of the Advances in Neural Information Processing Systems.MIT Press,2020:1595-1607.
[19]MODOLO D,SHUAI B,VARIOR R R,et al.Understanding the impact of mistakes on background regions in crowd counting[C]//Proceedings of the IEEE Winter Conference on Applications of Computer Vision.IEEE,2021:1649-1658.
[20]IDREES H,SALEEMI I,SEIBERT C,et al.Multi-source multi-scale counting in extremely dense crowd images[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2013:2547-2554.
[21]SINDAGI V,YASARLA R,PATEL V M M.JHU-CROWD++:Large-scale crowd counting dataset and a benchmark method[J].IEEE Transactions on Pattern Analysis and Machine Intelligence.2020:1-17.
[22]ARTETA C,LEMPITSKY V,ZISSERMAN A.Counting in the wild[C]//Proceedings of the European Conference on Computer Vision.Springer,2016:483-498.
[23]SHI Z,METTES P,SNOEK C.Counting with focus for free[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2019:4199-4208.
[24]WAN J,LUO W H,WU B Y,et al.Residual regression with semantic prior for crowd counting[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2019:4031-4040.
[25]LI Y H,ZHANG X F,CHEN D M.Csrnet:Dilated convolu-tional neural networks for understanding the highly congested scenes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2018:1091-1100.
[26]ZHANG C,KANG K,LIH S,et al.Data-driven crowd understanding:A baseline for a large-scale Crowd dataset[J].IEEE Transactions on Multimedia,2016,18(6):1048-1061.
[27]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[EB/OL].(2016-08-22)[2021-04-06].https://arxiv.org/abs/1608.06197.
[28]LIN T Y,GOYAL P,GIRSHICK R,et al.Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2017:2999-3007.
[29]DENG J,DONG W,SOCHER R,et al.Imagenet:A large-scale hierarchical image database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2009:248-255.
[30]KINGMA D P,BA J.Adam:A method for stochastic optimization[EB/OL].(2017-01-30)[2021-11-23].https://arxiv.org/abs/1412.6980v9.
[31]PENG X,PENG Y X,TANG Q.Crowd Counting Based on Single-column Multi-scale Convolutional Neural Network[J].Computer Science,2019,47(4):150-156.
[32]WAN J,CHAN A.Adaptive density map generation for crowd counting[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2019:1130-1139.
[33]MA Z H,WEI X,HONG X P,et al.Bayesian loss for crowd count estimation with point supervision[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2019:6141-6150.
[34]ABOUSAMRA S,HOAI M,SAMARAS D,et al.Localization in the crowd with topological constraints[C]//Proceedings of the AAAI Conference on Artificial Intelligence.AAAI Press,2021:872-881.
[35]XU C F,LIANG D K,XU Y C,et al.Autoscale:learning to scale for crowd counting[J].International Journal of Computer Vision.2022,130:405-434.
[36]THANASUTIVES P,FUKUI K,NUMAO M,et al.Encoder-decoder based convolutional neural networks with multi-scale-aware modules for crowd counting[C]//Proceedings of the IEEE International Conference on Pattern Recognition.IEEE,2021:2382-2389.
[37]TIAN Y K,LEI Y M,ZHANG J P,et al.PaDNet:Pan-density crowd counting[J].IEEE Transactions on Image Processing.2019,29:2714-2727.
[38]SINDAGI V A,PATEL V M.Generating high-quality crowddensity maps using contextual pyramid cnns[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2017:1861-1870
[39]ZHANG A,YUE L,SHEN J,et al.Attentional neural fifields for crowd counting[C]//Proceedings of the IEEE International Conference on Computer Vision.IEEE,2019:5714-5713.
[40]GUO,D,LI,K,ZHA Z J,et al.DADNet:Dilated-Attention-Deformable ConvNet for Crowd Counting[C]//Proceedings of the 27th ACM International Conference on Multimedia.ACM,2019:1823-1832.
[41]LIU N,LONG Y C,ZOU C Q,et al.ADCrowdNet:An attention-injective deformable convolutional network for crowd understanding[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.IEEE,2019:3225-3234.
[42]LIU C C,WENG X Y,MU Y D.Recurrent attentive zooming for joint crowd counting and precise localization[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2019:1217-1226.
[43]LIU W Z,SALZMANN M,FUA P.Context-aware crowdcounting[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2019:5094-5103.
[1] BAI Xuefei, MA Yanan, WANG Wenjian. Segmentation Method of Edge-guided Breast Ultrasound Images Based on Feature Fusion [J]. Computer Science, 2023, 50(3): 199-207.
[2] LIU Hang, PU Yuanyuan, LYU Dahua, ZHAO Zhengpeng, XU Dan, QIAN Wenhua. Polarized Self-attention Constrains Color Overflow in Automatic Coloring of Image [J]. Computer Science, 2023, 50(3): 208-215.
[3] CHEN Liang, WANG Lu, LI Shengchun, LIU Changhong. Study on Visual Dashboard Generation Technology Based on Deep Learning [J]. Computer Science, 2023, 50(3): 238-245.
[4] YING Zonghao, WU Bin. Backdoor Attack on Deep Learning Models:A Survey [J]. Computer Science, 2023, 50(3): 333-350.
[5] DONG Yongfeng, HUANG Gang, XUE Wanruo, LI Linhao. Graph Attention Deep Knowledge Tracing Model Integrated with IRT [J]. Computer Science, 2023, 50(3): 173-180.
[6] HUA Xiaofeng, FENG Na, YU Junqing, HE Yunfeng. Shooting Event Detection of Free Kick in Soccer Video Based on Rule Reasoning [J]. Computer Science, 2023, 50(3): 181-190.
[7] MEI Pengcheng, YANG Jibin, ZHANG Qiang, HUANG Xiang. Sound Event Joint Estimation Method Based on Three-dimension Convolution [J]. Computer Science, 2023, 50(3): 191-198.
[8] LIANG Jiali, HUA Baojian, SU Shaobo. Tensor Instruction Generation Optimization Fusing with Loop Partitioning [J]. Computer Science, 2023, 50(2): 374-383.
[9] ZOU Yunzhu, DU Shengdong, TENG Fei, LI Tianrui. Visual Question Answering Model Based on Multi-modal Deep Feature Fusion [J]. Computer Science, 2023, 50(2): 123-129.
[10] WANG Pengyu, TAI Wenxin, LIU Fang, ZHONG Ting, LUO Xucheng, ZHOU Fan. Self-supervised Flight Trajectory Prediction Based on Data Augmentation [J]. Computer Science, 2023, 50(2): 130-137.
[11] GUO Nan, LI Jingyuan, REN Xi. Survey of Rigid Object Pose Estimation Algorithms Based on Deep Learning [J]. Computer Science, 2023, 50(2): 178-189.
[12] LI Junlin, OUYANG Zhi, DU Nisuo. Scene Text Detection with Improved Region Proposal Network [J]. Computer Science, 2023, 50(2): 201-208.
[13] HUA Jie, LIU Xueliang, ZHAO Ye. Few-shot Object Detection Based on Feature Fusion [J]. Computer Science, 2023, 50(2): 209-213.
[14] CAI Xiao, CEHN Zhihua, SHENG Bin. SPT:Swin Pyramid Transformer for Object Detection of Remote Sensing [J]. Computer Science, 2023, 50(1): 105-113.
[15] WANG Bin, LIANG Yudong, LIU Zhe, ZHANG Chao, LI Deyu. Study on Unsupervised Image Dehazing and Low-light Image Enhancement Algorithms Based on Luminance Adjustment [J]. Computer Science, 2023, 50(1): 123-130.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!