计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (3): 95-103.doi: 10.11896/jsjkx.240500038

• 三维视觉与元宇宙 • 上一篇    下一篇

基于拆分联邦学习的元宇宙视线交互中的隐私主动保护方法研究

骆正权1,2, 王云龙2, 王子磊1, 孙哲南2, 张堃博2   

  1. 1 中国科学技术大学 合肥 230026
    2 中国科学院自动化研究所 北京 100190
  • 收稿日期:2024-05-09 修回日期:2025-01-20 出版日期:2025-03-15 发布日期:2025-03-07
  • 通讯作者: 王云龙(yunlong.wang@ia.ac.cn)
  • 基金资助:
    天津市重点研发计划院市合作项目(24YFYSHZ00290);国家重点研发计划青年科学家项目(2022YFC3310400)

Study on Active Privacy Protection Method in Metaverse Gaze Communication Based on SplitFederated Learning

LUO Zhengquan1,2, WANG Yunlong2, WANG Zilei1, SUN Zhenan2, ZHANG Kunbo2   

  1. 1 University of Science and Technology of China(USTC),Hefei 230026,China
    2 Institute of Automation Chinese Academy of Sciences,Beijing 100190,China
  • Received:2024-05-09 Revised:2025-01-20 Online:2025-03-15 Published:2025-03-07
  • About author:LUO Zhengquan,born in 1995,Ph.D.His main research interests include fe-derated learning and biometrics.
    WANG Yunlong,born in 1990,Ph.D,associate professor.His main research interests include pattern recognition,machine learning,light field photography and biometrics.
  • Supported by:
    Tianjin Key Research and Development Program CAS-Cooperation Project(24YFYSHZ00290)and National Key Research and Development Program of China(2022YFC3310400).

PDF (PC)

200

摘要： 随着元宇宙的迅猛发展,视线交互技术作为元宇宙核心交互方式受到广泛关注,视线隐私问题愈发引起人们的担忧。视线不仅可以表征凝视方向,还能用于个体身份识别,以及识别一系列敏感的软生物特征,如年龄、性别、种族等,甚至可以用于推断个体的情绪、认知状态和决策过程。因此,研究元宇宙视线交互中的隐私保护策略变得极为关键。此外,元宇宙中很多基于视线交互的新功能需要利用特定的个体隐私属性以提供更好的服务,然而目前尚无主动控制视线隐私进行选择性流通的方法。为此,首先围绕视线隐私泄露问题展开了分层次、定量的实证研究;接着创新性地提出了一种融合联邦学习与拆分学习的视线隐私保护框架,有效降低了隐私泄露的风险;进一步地,提出了一种基于对抗训练的主动隐私控制策略,不仅实现了针对性的隐私过滤,而且提高了视线模型的泛化能力;最后进行了严谨的实验验证,所提方法在视线数据的隐私保护和交互性能方面展现出了双重优势,为元宇宙环境中视线交互的隐私保护提供了创新的解决路径和技术方案。

关键词: 视线, 隐私, 安全, 虚拟现实, 联邦学习, 对抗学习

Abstract: In the rapidly evolving metaverse,gaze interaction has emerged as a pivotal mode of communication.However,gaze data encompasses more than mere gaze orientation and ocular mobility.It can also be applied for identification and recognition of soft biometrics,including age,gender,and ethnicity.Furthermore,it has the potential to disclose an individual's emotion,cognitive processes,and decision-making patterns.Given its sensitive nature,the development of robust gaze data privacy protection mechanisms has become imperative,attracting considerable interest.Additionally,numerous gaze-driven applications necessitate specific privacy attributes for functional support,yet active selection and protection of gaze privacy remains unexplored in current research.To this end,this study initially conducts hierarchical and quantitative analyses to uncover the severe state of gaze privacy breaches.Subsequently,it introduces an innovative gaze privacy safeguarding framework that integrates federated learning with split learning,significantly mitigating leakage risks.Moreover,this research proposes an active privacy protection strategy employing adversarial training and information bottleneck technique,which ensures targeted privacy filtration alongside enhancements in model generalization.Comprehensive experiments confirm that the devised APSFGaze approach excels in both privacy protection and performance.This study offers a novel pathway and technological framework for privacy preservation in metaverse gaze interactions.

Key words: Gaze, Privacy, Security, Virtual reality, Federated learning, Adversarial learning

中图分类号: 

  • TP181
[1]ZHANG M,LIU Y,LU F.GazeOnce:Real-time Multi-Person Gaze Estimation[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2022.
[2]ROLFF T,MATTHIAS HARMA H,STEINICKE F,et al.GazeTransformer:Gaze Forecasting for Virtual Reality Using Transformer Networks[C]//DAGM German Conference on Pattern Recognition.Cham:Springer International Publishing,2022.
[3]FATAHIPOUR H,MOSAVI M R,FARIBORZ J.Uncalibrated Eye Gaze Estimation using SE-ResNext with Unconstrained Head Movement and Ambient Light Change[EB/OL].https://www.researchsquare.com/article/rs-2666872/v1.
[4]KOMOGORTSEV O V,JAYARATHNA S,ARAGON C R,et al.Biometric Identification via an Oculomotor Plant Mathematical Model[C]//Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications.2010:57-60.
[5]DUCHOWSKI A T.A Breadth-First Survey of Eye-Tracking Applications[J].Behavior Research Methods,Instruments & Computers,2002,34(4):455-470.
[6]SCHRÖDER C,AL ZAIDAWI S M K,PRINZLER M H,et al.Robustness of Eye Movement Biometrics Against Varying Sti-muli and Varying Trajectory Length[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems.2020:1-7.
[7]LOHR D J,AZIZ S,KOMOGORTSEV O,et al.Eye Movement Biometrics Using a New Dataset Collected in Virtual Reality[C]//ACM Symposium on Eye Tracking Research and Applications.2020:1-3.
[8]ZHANG A T,LE MEUR O.How Old Do You Look? Inferring Your Age from Your Gaze[C]//2018 25th IEEE International Conference on Image Processing(ICIP).IEEE,2018:2660-2664.
[9]KRÖGER J L,LUTZ O H M,MÜLLER F.What Does YourGaze Reveal About You? On the Privacy Implications of Eye Tracking[C]//Privacy and Identity Management.Data for Better Living:AI and Privacy:14th IFIP WG 9.2,9.6/11.7,11.6/SIG 9.2.2 International Summer School,Windisch,Switzerland,August 19-23,2019,Revised Selected Papers 14.2020:226-241.
[10]RENAUD P,ROULEAU J L,GRANGER L,et al.MeasuringSexual Preferences in Virtual Reality:A Pilot Study[J].CyberPsychology & Behavior,2002,5(1):1-9.
[11]BERKOVSKY S,TAIB R,KOPRINSKA I,et al.Detecting Personality Traits Using Eye-Tracking Data[C]//Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems.2019:1-12.
[12]STEIL J,HAGESTEDT I,HUANG X L,et al.Privacy-Aware Eye Tracking Using Differential Privacy[C]//Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications.2019.
[13]DAVID-JOHN B,HOSFELT D,BUTLER K,et al.A Privacy-Preserving Approach to Streaming Eye-Tracking Data[J].ar-Xiv:2102.01770,2021.
[14]BOZKIR E,GÜNLÜ O,FUHL W,et al.Differential privacy for eye tracking with temporal correlations [J].PLoS ONE,2021,16(8):e0255979.
[15]STEIL J,MÜLLER P,SUGANO Y,et al.Privacy-aware eyetracking using differential privacy [J].Optics Express,2019,27(2):A80-A91.
[16]LUAN X X,ZHANG B J,LIU D D,et al.A Lightweight Heatmap-based Eye Tracking System [C]// 2021 International Conference on Computer Communications and Networks(ICCCN).IEEE,2021.
[17]ADHANOM I B,MACNEILAGE P,FOLMER E.Eye Tracking in Virtual Reality:A Broad Review of Applications and Challenges[J].Virtual Reality,2023,27:1481-1505.
[18]HASKINS A J,MENTCH J,BOTCH T L,et al.Active Vision in Immersive,360 Real-World Environments[J].Scientific Reports,2020,10(1):14304.
[19]LAMB M,BRUNDIN M,PEREZ LUQUE E,et al.Eye-Trac-king Beyond Peripersonal Space in Virtual Reality:Validation and Best Practices[J].Frontiers in Virtual Reality,2022,3:864653.
[20]QIAN K,ARICHI T,PRICE A,et al.An Eye Tracking Based Virtual Reality System for Use Inside Magnetic Resonance Imaging Systems[J].Scientific Reports,2021,11(1):16301.
[21]JOO H J,JEONG H Y.A Study on Eye-Tracking-Based Interface for VR/AR Education Platform[J].Multimedia Tools and Applications,2020,79:16719-16730.
[22]MATTHEWS S L,URIBE-QUEVEDO A,THEODOROU A.Rendering Optimizations for Virtual Reality Using Eye-Trac-king[C]//2020 22nd Symposium on Virtual and Augmented Reality(SVR).IEEE,2020:398-405.
[23]KASPROWSKI P,OBER J.Eye Movements in Biometrics[C]//Proceeding of the 2004 International Workshop on Biometric Authentication(BioAW 2004).2004:248-258.
[24]BEDNARIK R,KINNUNEN T,MIHAILA A,et al.Eye Movements as a Biometric[C]//LNCS.Berlin:Springer,2005:780-789.
[25]ZHANG Y,JUHOLA M.On Biometric Verification of a Userby Means of Eye Movement Data Mining[C]//The Second International Conference on Advances in Information Mining and Management.2012:85-90.
[26]KOMOGORTSEV O V,KARPOV A,PRICE L,et al.Biometric authentication via oculomotor plant characteristics.[C]//Proceedings of the IEEE/IARP International Conference on Biometrics(ICB).2012:1-8.
[27]RIGAS I,ECONOMOU G,FOTOPOULOS S.Human eyemovements as a trait for biometrical identification[C]//Biometrics:Theory,Applications and Systems(BTAS),2012 IEEE Fifth International Conference on Biometrics Compendium.IEEE,2012:217-222.
[28]DALRYMPLE K,JIANG M,ZHAO Q,et al.Machine learning accurately classifies age of toddlers based on eye tracking [J].Scientific Reports,2019,9(1):6255.
[29]HARIA R,ZAIDAWI S,MANETH S.Predicting Gender viaEye Movements [J].arXiv:2206.07442,2022.
[30]BLIGNAUT P,WIUM D.Eye-tracking data quality as affected by ethnicity and experimental design [J].Behavior Research Methods,2014,46:67-80.
[31]TARNOWSKI P,KOŁODZIEJ M,MAJKOWSKI A,et al.Eye-tracking analysis for emotion recognition [J/OL].https://onlinelibrary.wiley.com/doi/10.1155/2020/2909267.
[32]ECKSTEIN M K,GUERRA-CARRILLO B,SINGLEY A,et al.Beyond eye gaze:What else can eyetracking reveal about cognition and cognitive development? [J].Developmental Cognitive Neuroscience,2017,25:69-91.
[33]GUAZZINI A,YONEKI E,GRONCHI G.Cognitive dissonance and social influence effects on preference judgments:An eye tracking based system for their automatic assessment [J].International Journal of Human-Computer Studies,2015,73:12-18.
[34]LIU A,XIA L,DUCHOWSKI A,et al.Differential privacy for eye-tracking data [C]//Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications.2019,28:1-10.
[35]GROSS R,AIROLDI E,MALIN B,et al.Integrating utility into face de-identification[C]//International Workshop on Privacy Enhancing Technologies.Springer,2005:227-242.
[36]FUHL W,BOZKIR E,KASNECI E.Reinforcement learning for the privacy preservation and manipulation of eye tracking data[C]//International Conference on Artificial Neural Networks.Springer,2021:595-607.
[37]ELFARES M,HU Z M,REISERT P,et al.Federated Learning for Appearance-based Gaze Estimation in the Wild [C]// An-nual Conference on Neural Information Processing Systems.PMLR,2023.
[38]TISHBY N,PEREIRA F C,BIALEK W.The information bottleneck method[J].arXiv:physics/0004057,2000.
[39]GILAD-BACHRACH R,NAVOT A,TISHBY N.An information theoretic tradeoff between complexity and accuracy[C]//Learning Theory and Kernel Machines.Springer,2003:595-609.
[40]RODRÍGUEZ GÁLVEZ B,THOBABEN R,SKOGLUND M.The convex information bottleneck lagrangian[J].Entropy,2020,22(1):98.
[41]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial networks[J].Communications of the ACM,2020,63(11):139-144.
[42]BRUNYÉ T T,DREW T,WEAVER D L,et al.A review of eye tracking for understanding and improving diagnostic interpretation[J].Cognitive Research:Principles and Implications,2019,4(1):1-16.
[43]LIU W,QIU J L,ZHENG W L,et al.Comparing recognitionperformance and robustness of multimodal deep learning models for multimodal emotion recognition[J].IEEE Transactions on Cognitive and Developmental Systems,2021,14(2):715-729.
[44]MRIDULA T D,FERDAUS A A,PIAS T S.Exploring Emotions in EEG:Deep Learning Approach with Feature Fusion[C]//2023 26th International Conference on Computer and Information Technology(ICCIT).2023.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发