Computer Science ›› 2025, Vol. 52 ›› Issue (3): 95-103.doi: 10.11896/jsjkx.240500038

• 3D Vision and Metaverse • Previous Articles     Next Articles

Study on Active Privacy Protection Method in Metaverse Gaze Communication Based on SplitFederated Learning

LUO Zhengquan1,2, WANG Yunlong2, WANG Zilei1, SUN Zhenan2, ZHANG Kunbo2   

  1. 1 University of Science and Technology of China(USTC),Hefei 230026,China
    2 Institute of Automation Chinese Academy of Sciences,Beijing 100190,China
  • Received:2024-05-09 Revised:2025-01-20 Online:2025-03-15 Published:2025-03-07
  • About author:LUO Zhengquan,born in 1995,Ph.D.His main research interests include fe-derated learning and biometrics.
    WANG Yunlong,born in 1990,Ph.D,associate professor.His main research interests include pattern recognition,machine learning,light field photography and biometrics.
  • Supported by:
    Tianjin Key Research and Development Program CAS-Cooperation Project(24YFYSHZ00290)and National Key Research and Development Program of China(2022YFC3310400).

Abstract: In the rapidly evolving metaverse,gaze interaction has emerged as a pivotal mode of communication.However,gaze data encompasses more than mere gaze orientation and ocular mobility.It can also be applied for identification and recognition of soft biometrics,including age,gender,and ethnicity.Furthermore,it has the potential to disclose an individual's emotion,cognitive processes,and decision-making patterns.Given its sensitive nature,the development of robust gaze data privacy protection mechanisms has become imperative,attracting considerable interest.Additionally,numerous gaze-driven applications necessitate specific privacy attributes for functional support,yet active selection and protection of gaze privacy remains unexplored in current research.To this end,this study initially conducts hierarchical and quantitative analyses to uncover the severe state of gaze privacy breaches.Subsequently,it introduces an innovative gaze privacy safeguarding framework that integrates federated learning with split learning,significantly mitigating leakage risks.Moreover,this research proposes an active privacy protection strategy employing adversarial training and information bottleneck technique,which ensures targeted privacy filtration alongside enhancements in model generalization.Comprehensive experiments confirm that the devised APSFGaze approach excels in both privacy protection and performance.This study offers a novel pathway and technological framework for privacy preservation in metaverse gaze interactions.

Key words: Gaze, Privacy, Security, Virtual reality, Federated learning, Adversarial learning

CLC Number: 

  • TP181
[1]ZHANG M,LIU Y,LU F.GazeOnce:Real-time Multi-Person Gaze Estimation[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2022.
[2]ROLFF T,MATTHIAS HARMA H,STEINICKE F,et al.GazeTransformer:Gaze Forecasting for Virtual Reality Using Transformer Networks[C]//DAGM German Conference on Pattern Recognition.Cham:Springer International Publishing,2022.
[3]FATAHIPOUR H,MOSAVI M R,FARIBORZ J.Uncalibrated Eye Gaze Estimation using SE-ResNext with Unconstrained Head Movement and Ambient Light Change[EB/OL].https://www.researchsquare.com/article/rs-2666872/v1.
[4]KOMOGORTSEV O V,JAYARATHNA S,ARAGON C R,et al.Biometric Identification via an Oculomotor Plant Mathematical Model[C]//Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications.2010:57-60.
[5]DUCHOWSKI A T.A Breadth-First Survey of Eye-Tracking Applications[J].Behavior Research Methods,Instruments & Computers,2002,34(4):455-470.
[6]SCHRÖDER C,AL ZAIDAWI S M K,PRINZLER M H,et al.Robustness of Eye Movement Biometrics Against Varying Sti-muli and Varying Trajectory Length[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems.2020:1-7.
[7]LOHR D J,AZIZ S,KOMOGORTSEV O,et al.Eye Movement Biometrics Using a New Dataset Collected in Virtual Reality[C]//ACM Symposium on Eye Tracking Research and Applications.2020:1-3.
[8]ZHANG A T,LE MEUR O.How Old Do You Look? Inferring Your Age from Your Gaze[C]//2018 25th IEEE International Conference on Image Processing(ICIP).IEEE,2018:2660-2664.
[9]KRÖGER J L,LUTZ O H M,MÜLLER F.What Does YourGaze Reveal About You? On the Privacy Implications of Eye Tracking[C]//Privacy and Identity Management.Data for Better Living:AI and Privacy:14th IFIP WG 9.2,9.6/11.7,11.6/SIG 9.2.2 International Summer School,Windisch,Switzerland,August 19-23,2019,Revised Selected Papers 14.2020:226-241.
[10]RENAUD P,ROULEAU J L,GRANGER L,et al.MeasuringSexual Preferences in Virtual Reality:A Pilot Study[J].CyberPsychology & Behavior,2002,5(1):1-9.
[11]BERKOVSKY S,TAIB R,KOPRINSKA I,et al.Detecting Personality Traits Using Eye-Tracking Data[C]//Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems.2019:1-12.
[12]STEIL J,HAGESTEDT I,HUANG X L,et al.Privacy-Aware Eye Tracking Using Differential Privacy[C]//Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications.2019.
[13]DAVID-JOHN B,HOSFELT D,BUTLER K,et al.A Privacy-Preserving Approach to Streaming Eye-Tracking Data[J].ar-Xiv:2102.01770,2021.
[14]BOZKIR E,GÜNLÜ O,FUHL W,et al.Differential privacy for eye tracking with temporal correlations [J].PLoS ONE,2021,16(8):e0255979.
[15]STEIL J,MÜLLER P,SUGANO Y,et al.Privacy-aware eyetracking using differential privacy [J].Optics Express,2019,27(2):A80-A91.
[16]LUAN X X,ZHANG B J,LIU D D,et al.A Lightweight Heatmap-based Eye Tracking System [C]// 2021 International Conference on Computer Communications and Networks(ICCCN).IEEE,2021.
[17]ADHANOM I B,MACNEILAGE P,FOLMER E.Eye Tracking in Virtual Reality:A Broad Review of Applications and Challenges[J].Virtual Reality,2023,27:1481-1505.
[18]HASKINS A J,MENTCH J,BOTCH T L,et al.Active Vision in Immersive,360 Real-World Environments[J].Scientific Reports,2020,10(1):14304.
[19]LAMB M,BRUNDIN M,PEREZ LUQUE E,et al.Eye-Trac-king Beyond Peripersonal Space in Virtual Reality:Validation and Best Practices[J].Frontiers in Virtual Reality,2022,3:864653.
[20]QIAN K,ARICHI T,PRICE A,et al.An Eye Tracking Based Virtual Reality System for Use Inside Magnetic Resonance Imaging Systems[J].Scientific Reports,2021,11(1):16301.
[21]JOO H J,JEONG H Y.A Study on Eye-Tracking-Based Interface for VR/AR Education Platform[J].Multimedia Tools and Applications,2020,79:16719-16730.
[22]MATTHEWS S L,URIBE-QUEVEDO A,THEODOROU A.Rendering Optimizations for Virtual Reality Using Eye-Trac-king[C]//2020 22nd Symposium on Virtual and Augmented Reality(SVR).IEEE,2020:398-405.
[23]KASPROWSKI P,OBER J.Eye Movements in Biometrics[C]//Proceeding of the 2004 International Workshop on Biometric Authentication(BioAW 2004).2004:248-258.
[24]BEDNARIK R,KINNUNEN T,MIHAILA A,et al.Eye Movements as a Biometric[C]//LNCS.Berlin:Springer,2005:780-789.
[25]ZHANG Y,JUHOLA M.On Biometric Verification of a Userby Means of Eye Movement Data Mining[C]//The Second International Conference on Advances in Information Mining and Management.2012:85-90.
[26]KOMOGORTSEV O V,KARPOV A,PRICE L,et al.Biometric authentication via oculomotor plant characteristics.[C]//Proceedings of the IEEE/IARP International Conference on Biometrics(ICB).2012:1-8.
[27]RIGAS I,ECONOMOU G,FOTOPOULOS S.Human eyemovements as a trait for biometrical identification[C]//Biometrics:Theory,Applications and Systems(BTAS),2012 IEEE Fifth International Conference on Biometrics Compendium.IEEE,2012:217-222.
[28]DALRYMPLE K,JIANG M,ZHAO Q,et al.Machine learning accurately classifies age of toddlers based on eye tracking [J].Scientific Reports,2019,9(1):6255.
[29]HARIA R,ZAIDAWI S,MANETH S.Predicting Gender viaEye Movements [J].arXiv:2206.07442,2022.
[30]BLIGNAUT P,WIUM D.Eye-tracking data quality as affected by ethnicity and experimental design [J].Behavior Research Methods,2014,46:67-80.
[31]TARNOWSKI P,KOŁODZIEJ M,MAJKOWSKI A,et al.Eye-tracking analysis for emotion recognition [J/OL].https://onlinelibrary.wiley.com/doi/10.1155/2020/2909267.
[32]ECKSTEIN M K,GUERRA-CARRILLO B,SINGLEY A,et al.Beyond eye gaze:What else can eyetracking reveal about cognition and cognitive development? [J].Developmental Cognitive Neuroscience,2017,25:69-91.
[33]GUAZZINI A,YONEKI E,GRONCHI G.Cognitive dissonance and social influence effects on preference judgments:An eye tracking based system for their automatic assessment [J].International Journal of Human-Computer Studies,2015,73:12-18.
[34]LIU A,XIA L,DUCHOWSKI A,et al.Differential privacy for eye-tracking data [C]//Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications.2019,28:1-10.
[35]GROSS R,AIROLDI E,MALIN B,et al.Integrating utility into face de-identification[C]//International Workshop on Privacy Enhancing Technologies.Springer,2005:227-242.
[36]FUHL W,BOZKIR E,KASNECI E.Reinforcement learning for the privacy preservation and manipulation of eye tracking data[C]//International Conference on Artificial Neural Networks.Springer,2021:595-607.
[37]ELFARES M,HU Z M,REISERT P,et al.Federated Learning for Appearance-based Gaze Estimation in the Wild [C]// An-nual Conference on Neural Information Processing Systems.PMLR,2023.
[38]TISHBY N,PEREIRA F C,BIALEK W.The information bottleneck method[J].arXiv:physics/0004057,2000.
[39]GILAD-BACHRACH R,NAVOT A,TISHBY N.An information theoretic tradeoff between complexity and accuracy[C]//Learning Theory and Kernel Machines.Springer,2003:595-609.
[40]RODRÍGUEZ GÁLVEZ B,THOBABEN R,SKOGLUND M.The convex information bottleneck lagrangian[J].Entropy,2020,22(1):98.
[41]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial networks[J].Communications of the ACM,2020,63(11):139-144.
[42]BRUNYÉ T T,DREW T,WEAVER D L,et al.A review of eye tracking for understanding and improving diagnostic interpretation[J].Cognitive Research:Principles and Implications,2019,4(1):1-16.
[43]LIU W,QIU J L,ZHENG W L,et al.Comparing recognitionperformance and robustness of multimodal deep learning models for multimodal emotion recognition[J].IEEE Transactions on Cognitive and Developmental Systems,2021,14(2):715-729.
[44]MRIDULA T D,FERDAUS A A,PIAS T S.Exploring Emotions in EEG:Deep Learning Approach with Feature Fusion[C]//2023 26th International Conference on Computer and Information Technology(ICCIT).2023.
[1] HU Kangqi, MA Wubin, DAI Chaofan, WU Yahui, ZHOU Haohao. Federated Learning Evolutionary Multi-objective Optimization Algorithm Based on Improved NSGA-III [J]. Computer Science, 2025, 52(3): 152-160.
[2] WANG Ruicong, BIAN Naizheng, WU Yingjun. FedRCD:A Clustering Federated Learning Algorithm Based on Distribution Extraction andCommunity Detection [J]. Computer Science, 2025, 52(3): 188-196.
[3] CHENG Dawei, WU Jiaxuan, LI Jiangtong, DING Zhijun, JIANG Changjun. Study on Evaluation Framework of Large Language Model’s Financial Scenario Capability [J]. Computer Science, 2025, 52(3): 239-247.
[4] WANG Dongzhi, LIU Yan, GUO Bin, YU Zhiwen. Edge-side Federated Continuous Learning Method Based on Brain-like Spiking Neural Networks [J]. Computer Science, 2025, 52(3): 326-337.
[5] XIE Jiachen, LIU Bo, LIN Weiwei , ZHENG Jianwen. Survey of Federated Incremental Learning [J]. Computer Science, 2025, 52(3): 377-384.
[6] CAO Mingwei, ZHANG Di, PENG Shengjie, LI Ning, ZHAO Haifeng. Survey of Metaverse Technology Development and Applications [J]. Computer Science, 2025, 52(3): 4-16.
[7] WANG Xingguo, SUN Yunxiao, WANG Bailing. Improvement of SSH Transport Layer Protocol Based on Chain of Trust [J]. Computer Science, 2025, 52(2): 353-361.
[8] ZHENG Jianwen, LIU Bo, LIN Weiwei, XIE Jiachen. Survey of Communication Efficiency for Federated Learning [J]. Computer Science, 2025, 52(2): 1-7.
[9] ZHANG Xin, ZHANG Han, NIU Manyu, JI Lixia. Adversarial Sample Detection in Computer Vision:A Survey [J]. Computer Science, 2025, 52(1): 345-361.
[10] WANG Xin, XIONG Shubo, SUN Lingyun. Federated Graph Learning:Problems,Methods and Challenges [J]. Computer Science, 2025, 52(1): 362-373.
[11] SU Chaoran, ZHANG Dalong, HUANG Yong, DONG An. RF Fingerprint Recognition Based on SE Attention Multi-source Domain Adversarial Network [J]. Computer Science, 2025, 52(1): 412-419.
[12] DUN Jingbo, LI Zhuo. Survey on Transmission Optimization Technologies for Federated Large Language Model Training [J]. Computer Science, 2025, 52(1): 42-55.
[13] LIU Yuming, DAI Yu, CHEN Gongping. Review of Federated Learning in Medical Image Processing [J]. Computer Science, 2025, 52(1): 183-193.
[14] CHEN Liang, SUN Cong. Deep-learning Based DKOM Attack Detection for Linux System [J]. Computer Science, 2024, 51(9): 383-392.
[15] LI Yongjun, ZHU Yuefei, BAI Lifang. Enhanced Location K-anonymity Privacy Protection Scheme Based on Geohash [J]. Computer Science, 2024, 51(9): 393-400.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!