Computer Science

Computer Science ›› 2021, Vol. 48 ›› Issue (2): 167-174.doi: 10.11896/jsjkx.200100065

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Event-based User Experience Evaluation Method for Virtual Reality Applications

MA Si-qi, CHE Xiao-ping, YU Qi, YUE Chen-feng   

  1. School of Software Engineering,Beijing Jiaotong University,Beijing 100044,China
  • Received:2020-01-09 Revised:2020-06-04 Online:2021-02-15 Published:2021-02-04
  • About author:MA Si-qi,born in 1995,postgraduate.Her main research interests include data analysis,machine learning,testing and user experience.
    CHE Xiao-ping,born in 1986,Ph.D,associate professor,is a member of China Computer Federation,the ACM and the IEEE.His main research interests include user experience,passive testing,crowdsensing and Internet of Things.

PDF (PC)

1229

Abstract: With the rapid development of virtual reality(VR) technology in recent years,the VR industry is currently growing fast.Although VR provides many new possibilities for content form,due to the immaturity of VR application design,designers in the VR industry still face severe challenges.Design of VR applications still needs to be verified by users.Therefore,user expe-rience(UX) analysis is really critical to the success of VR software development.At the same time,with the increase of users' requirements for VR content quality,whether the content is attractive enough will greatly affects users' experience and determine the service life of VR applications.Consequently,searching for interaction events that will affect VR UX is critical to improve user stickiness.At present,the researches on VR mainly focus on the improvement of hardware and software,and pay less attention to the content event design.Moreover,there is no unified and clear standard for VR UX evaluation.This paper attempts to find out the relationship between user traits,VR interaction events and user experience through realistic experiments.This paper firstly defines four types of VR interaction events,and designs a questionnaire for collecting the tester's traits and their subjective evaluation.During the experiment,objective physiological data of the testers and their participation process are recorded.The 80 testers are divided into two groups to experience two types of VR games with constant time.The statistical method and improved Prism algorithm are used to find out the correlation among user traits,type of game interaction events and user experience.The experiment results can provide references for VR designers and developers,and at the same time provide the preliminary study to standardized VR user experience evaluation.

Key words: Prism algorithm, User experience, User traits, Virtual reality, VR interaction

CLC Number: 

  • TP311.5
[1] BERG L P,VANCE J M.Industry use of virtual reality in product design and manufacturing:a survey[J].Virtual Reality,2017,21(1):1-17.
[2] HAMARI J,ALHA K,JÄRVELÄ S,et al.Why do players buyin-game content? An empirical study on concrete purchase motivations[J].Computers in Human Behavior,2017,68:538-546.
[3] FROMMEL J,SONNTAG S,WEBER M.Effects of controller-based locomotion on player experience in a virtual reality exploration game[C]//Proceedings of the 12th International Conference on the Foundations of Digital Games.ACM,2017:30.
[4] SHIN D.Empathy and embodied experience in virtual environment:To what extent can virtual reality stimulate empathy and embodied experience?[J].Computers in Human Behavior,2018,78:64-73.
[5] KNIBBE J,SCHJERLUND J,PETRAEUS M,et al.The dream is collapsing:the experience of exiting VR[C]//Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems.ACM,2018:483.
[6] OH J,KIM G J.Effect of accompanying onomatopoeia withsound feedback toward presence and user experience in virtual reality[C]//Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology.ACM,2018:81.
[7] TRINDADE Y,REBELO F,NORIEGA P.Tourism and virtual reality:user experience evaluation of a virtual environment prototype[C]//International Conference of Design,User Experience,and Usability.Springer,Cham,2018:730-742.
[8] KELLING C,KAUHANEN O,VÄÄTÄJÄ H,et al.Implica-tions of Audio and Narration in the User Experience Design of Virtual Reality[C]//Proceedings of the 22nd International Academic Mindtrek Conference.ACM,2018:258-261.
[9] ROTH D,KLEINBECK C,FEIGL T,et al.[poster] social augmentations in multi-user virtual reality:A virtual museum experience[C]//2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).IEEE,2017:42-43.
[10] SENNO B,BARCHA P.Customizing User Experience with Adaptive Virtual Reality[C]//Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion.ACM,2018:42.
[11] CHITTARO L,SIONI R,CRESCENTINI C,et al.Mortality salience in virtual reality experiences and its effects on users' attitudes towards risk[J].International Journal of Human-Computer Studies,2017,101:10-22.
[12] ANDERSEN T,ANISIMOVAITE G,CHRISTIANSEN A,et al.A preliminary study of users' experiences of meditation in virtual reality[C]//2017 IEEE Virtual Reality (VR).IEEE,2017:343-344.
[13] CHO D,HAM J,OH J,et al.Detection of stress levels from biosignals measured in virtual reality environments using a kernel-based extreme learning machine[J].Sensors,2017,17(10):2435.
[14] BAKKER J,PECHENIZKIY M,SIDOROVA N.What's your current stress level? Detection of stress patterns from GSR sensor data[C]//2011 IEEE 11th International Conference on Data Mining Workshops.IEEE,2011:573-580.
[15] SARKAR P,ROSS K,RUBERTO A J,et al.Classification ofcognitive load and expertise for adaptive simulation using deep multitask learning[C]//2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII).IEEE,2019:1-7.
[16] HINKLE L,KHOSHHAL K,METSIS V.Physiological Measurement for Emotion Recognition in Virtual Reality[C]//2019 2nd International Conference on Data Intelligence and Security (ICDIS).IEEE,2019:136-143.
[17] WARREN L E,BOWMAN D A.User experience with semi-natural locomotion techniques in virtual reality:the case of the Virtuix Omni[C]//Proceedings of the 5th Symposium on Spatial User Interaction.ACM,2017:163-163.
[18] CALANDRA D,BILLI M,LAMBERTI F,et al.Arm swinging vs treadmill:a comparison between two techniques for locomotion in virtual reality[C]//Proceedings of the 39th Annual European Association for Computer Graphics Conference:Short Papers.Eurographics Association,2018:53-56.
[19] GUGENHEIMER J,STEMASOV E,FROMMEL J,et al.Sharevr:Enabling co-located experiences for virtual reality between hmd and non-hmd users[C]//Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems.ACM,2017:4021-4033.
[20] BOZGEYIKLI L,RAIJ A,KATKOORI S,et al.Effects of Instruction Methods on User Experience in Virtual Reality Serious Games[C]//International Conference on Virtual,Augmented and Mixed Reality.Springer,Cham,2017:215-226.
[21] MEN L,BRYAN-KINNS N,HASSARD A S,et al.The impact of transitions on user experience in virtual reality[C]//2017 IEEE Virtual Reality (VR).IEEE,2017:285-286.
[22] JERALD J.The VR book:Human-centered design for virtualreality[M].Morgan & Claypool,2015.
[23] LEE J,BANG J,SUH H.Identifying Affordance Features in Virtual Reality:How Do Virtual Reality Games Reinforce User Experience?[C]//International Conference on Augmented Cognition.Springer,Cham,2018:383-394.
[24] KIM M,LEE J,KIM C,et al.Tpvr:User interaction of third person virtual reality for new presence and experience[J].Symmetry,2018,10(4):109.
[25] HARMS J,SEITZ D,WIMMER C,et al.Low-cost gamification of online surveys:Improving the user experience through achievement badges[C]//Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play.ACM,2015:109-113.
[26] SCHREPP M,COTA M P,GONÇALVES R,et al.Adaption of user experience questionnaires for different user groups[J].Universal Access in the Information Society,2017,16(3):629-640.
[27] CHERTOFF D B,GOLDIEZ B,LAVIOLA J J.Virtual Experience Test:A virtual environment evaluation questionnaire[C]//2010 IEEE virtual reality conference (VR).IEEE,2010:103-110.
[28] KIM H K,PARK J,CHOI Y,et al.Virtual reality sicknessquestionnaire (VRSQ):Motion sickness measurement index in a virtual reality environment[J].Applied ergonomics,2018,69:66-73.
[29] HEALEY J A,PICARD R W.Detecting stress during real-world driving tasks using physiological sensors[J].IEEE Transactions on Intelligent Transportation Systems,2005,6(2):156-166.
[30] DENNISON M S,WISTI A Z,D'ZMURA M.Use of physiological signals to predict cybersickness[J].Displays,2016,44:42-52.
[31] WANG Y,CHARDONNET J R,MERIENNE F.VR sickness prediction for navigation in immersive virtual environments using a deep long short term memory model[C]//2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE,2019:1874-1881.
[32] YAO L,LIU Y,LI W,et al.Usingphysiological measures toevaluate user experience of mobile applications[C]//International Conference on Engineering Psychology and Cognitive Ergonomics.Springer,Cham,2014:301-310.
[33] TCHA-TOKEY K,LOUP-ESCANDE E,CHRISTMANN O,et al.A questionnaire to measure the user experience in immersive virtual environments[C]//Proceedings of the 2016 Virtual Reality International Conference.ACM,2016:19.
[34] YU Q,CHE X,MA S,et al.A Hybrid User Experience Evaluation Method for Mobile Games[J].IEEE Access,2018,6:49067-49079.
[35] PARSONS T D,LARSON P,KRATZ K,et al.Sex differences in mental rotation and spatial rotation in a virtual environment[J].Neuropsychologia,2004,42(4):555-562.
[36] LIKERT R.A technique for the measurement of attitudes[J].Archives of Psychology,1932,140:54-55.
[37] YUDA E,YAMAMOTO K,YOSHIDA Y,et al.Discrimination of Emotional Type by Heartbeat Signal Information[C]//2018 IEEE 7th Global Conference on Consumer Electronics (GCCE).IEEE,2018:531-532.
[1] QU Qian-wen, CHE Xiao-ping, QU Chen-xin, LI Jin-ru. Study on Information Perception Based User Presence in Virtual Reality [J]. Computer Science, 2022, 49(9): 146-154.
[2] LIU Yue-hong, NIU Shao-hua, SHEN Xian-hao. Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network [J]. Computer Science, 2022, 49(7): 127-131.
[3] ZHANG Ji-kai, LI Qi, WANG Yue-ming, LYU Xiao-qi. Survey of 3D Gesture Tracking Algorithms Based on Monocular RGB Images [J]. Computer Science, 2022, 49(4): 174-187.
[4] ZHU Chen-shuang, CHENG Shi-wei. Prediction and Assistance of Navigation Demand Based on Eye Tracking in Virtual Reality Environment [J]. Computer Science, 2021, 48(8): 315-321.
[5] ZHANG Yu-xiang, REN Shuang. Overview of Application of Positioning Technology in Virtual Reality [J]. Computer Science, 2021, 48(1): 308-318.
[6] ZHANG Zheng-lin,ZHANG Li-wei,WANG Wen-juan,XIA Li,FU Hao,WANG Hong-zhi,YANG Li-zhuang and
LI Hai. Survey on Computerized Neurocognitive Assessment System [J]. Computer Science, 2020, 47(2): 150-156.
[7] LI Meng-jun, LI Wen-wei. Adaptive Adjustment Algorithm of Mobile Phone Screen Brightness Based on Visual Characteristics [J]. Computer Science, 2019, 46(2): 255-260.
[8] JI Li-xia, LIU Cheng-ming. Fuzzy Static Image Target Reproduction Method Based on Virtual Reality Technology [J]. Computer Science, 2018, 45(7): 248-251.
[9] HAN Li , LIU Zheng-jie. CAUXT:A Tool to Help User Experience Researchers Capture Users’ Experience Data in Context of Interest [J]. Computer Science, 2018, 45(7): 278-285.
[10] ZHANG Ning, LIU Ying-chun, SHEN Zhi-peng and GUO Chen. Review of Virtual Reality Technology Based Application Research in English Teaching for Special Purposes [J]. Computer Science, 2017, 44(Z6): 43-47.
[11] WANG Wei, HU Chang-wu, GUO Dong, ZHANG Jing-xuan, CHANG Jin-da and ZHANG Li-qing. Cloud Operating System for Cloudware [J]. Computer Science, 2017, 44(11): 33-40.
[12] PAN Ren-yu, SUN Chang-le, XIONG Wei and WANG Hai-tao. Survey and Prospect of Collision Detection Based on Virtual Assembly Environment [J]. Computer Science, 2016, 43(Z11): 136-139.
[13] LI Juan-ni,HUA Qing-yi and JI Xiang. Task Analysis and Task Modeling Method in Mobile Environment [J]. Computer Science, 2014, 41(10): 210-215.
[14] HAN Li,LIU Zheng-jie,ZHANG Jun and CHEN Yuan-yuan. Remote User Experience Evaluation Review:Tools,Methods and Challenges [J]. Computer Science, 2014, 41(10): 196-203.
[15] LI Dai-wei and SUN Jian-shun. Design and Implementation of Explosive Devices Exclude Virtual Simulation Training System [J]. Computer Science, 2013, 40(9): 302-306.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.