计算机科学 ›› 2014, Vol. 41 ›› Issue (10): 196-203.doi: 10.11896/j.issn.1002-137X.2014.10.043
韩立,刘正捷,张军,陈媛嫄
HAN Li,LIU Zheng-jie,ZHANG Jun and CHEN Yuan-yuan
摘要： 远程用户体验评估允许评估人员和用户在时间和(或)空间上分离,它具有经济、便捷、对用户打扰少,能在真实环境中对大量多样的用户进行研究等特点,被认为是解决传统用户体验评估问题的一个理想方案,并逐渐成为人机交互领域中一个热门的研究课题。首先介绍了远程用户体验评估的概念和分类；然后描述了远程评估的研究概况；接着提出了远程评估工具的系统架构,并依据系统架构,详细描述了数据采集、分析和呈现的各种方法；最后,总结了远程评估所面对的挑战以及未来的研究方向。
| [1] ISO DIS．Ergonomics of human system interaction-Part 210:Human-centred design for interactive systems[S]．Switzerland:International Standardization Organization (ISO),2010:9241-210 [2] Law E,Roto V,et al．Understanding,scoping and defining user experience:a survey approach[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems,2009．Boston,USA,2009:719-728 [3] Hassenzahl M,Tractinsky N．User experience-a research agenda [J]．Behaviour and Information Technology,2006,25:91-97 [4] Law E,Roto V,et al．User experience evaluation methods:current state and development needs[C]∥Proceedings of Nordic Conference on Human-Computer Interaction．Reykjavik,Iceland,2010:521-530 [5] Agarwal A,Meyer A．Beyond usability:evaluating emotional response as an integral part of the user experience[C]∥Procee-dings of the SIGCHI Conference on Human Factors in Computing Systems．Boston,USA,2009:2919-2930 [6] Johnson P．Usability and mobility:interactions on the move[C]∥Proceedings of Human-Computer Interaction with Mobile Devices．Glasgow,Scotland,1998(2):1-4 [7] Kallio T,Kaikkonen A,et al．Usability testing of mobile applications:a comparison between laboratory and field testing [J]．Usability studies,2005,1:4-16 [8] Elzakker V,Delikostidis I,et al．Field-based usability evaluation methodology for mobile geo-applications [J]．The Cartographic Journal,2008,45:139-149 [9] Christensen P,Romero M,et al．Children,mobility,and space:using GPS and mobile phone technologies in ethnographic research [J]．Mixed Methods Research,2011,5:227-246 [10] Coleman E．Ethnographic approaches to digital media [J]．Annual Review of Anthropology,2010,39:487-505 [11] Duh H,Tan G．Usability evaluation for mobile device:a compari-son of laboratory and field tests[C]∥Proceedings of International Confe-rence on Human-Computer Interaction with Mobile Devices and Services．Helsinki,Finland,2006:181-186 [12] Monahan K,Lahteenmaki M,et al．An investigation into the use of field methods in the design and evaluation of interactive systems[C]∥Proceedings of British HCI Group Annual Confe-rence．Swinton,UK,2008:99-108 [13] Xu Sun,Andrew M．A comparison of field-based and lab-based experiments to evaluate user experience of personalised mobile devices [J]．Advances in Human-Computer Interaction,2013,2013:64-72 [14] Raptis D,Tselios N,et al．Context-based design of mobile applications for museums:a survey of existing practices[C]∥Proceedings of International Conference on Human Computer Interaction with Mobile Devices and Services．Salzburg,Austria,2005:153-160 [15] Kjeldskov J,Skov M,et al．Is it worth the hassle? Exploring the added value of evaluating the usability of context-aware mobile systems in the field[C]∥Proceedings of International Mobile Conference．Glasgow,UK,2004:61-73 [16] Kjeldskov J,Stage J．New techniques for usability evaluation of mobile systems [J]．Human Computer Studies,2004,60:599-620 [17] Bruun A,Gull P,et al．Let your users do the testing:a comparison of three remote asynchronous usability testing methods,2009[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Boston,USA,2009:1619-1628 [18] Hartson H,Castillo J,et al．Remote evaluation:the network as an extension of the usability laboratory[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Vancouver,Canada,1996:228-235 [19] Susan D,David S．Remote possibilities? International usability testing at a distance [J]．Interactions,2004,11:10-17 [20] Kaasila J,Ferreira D,et al．Testdroid:automated remote UI testing on Android[C]∥Proceedings of International Conference on Mobile and Ubiquitous Multimedia．Ulm,Germany,2012:1-4 [21] Crabtree A,Chamberlain A,et al．Introduction to the Special Issue of “The Turn to The Wild” [J]．ACM Transactions on Computer-Human Interaction,2013,3(3):1-4 [22] Andreasen M,Nielsen H,et al．What happened to remote usabi-lity testing? An empirical study of three methods[C]∥Procee-dings of the SIGCHI Conference on Human Factors in Computing Systems．San Jose,USA,2007:1405-1414 [23] Thompson K,Rozanski E,et al．Here,there,anywhere:remote usability testing that works[C]∥Proceedings of Conference on Information Technology Education．Salt Lake,USA,2004:132-137 [24] Carta T,Paternò F,et al．Web usability probe:a tool for supporting remote usability evaluation of web sites[C]∥Procee-dings of IFIP TC International Conference Human-Computer Interaction．Lisbon,Portugal,2011:349-357 [25] Carta T,Paternò F,et al．Support for remote usability evaluation of web mobile applications[C]∥Proceeding of ACM International Conference on Design of Communications．Pisa,Italy,2011:129-136 [26] DeShazo J,Harris L,et al．Designing and remotely testing mobile diabetes video games [J]．Telemedicine and Telecare,2010,16:378-382 [27] Scholtz J．Adaptation of traditional usability testing methods for remote testing[C]∥Proceedings of Annual Hawaii International Conference on System Sciences．Maui,HI,2001:8-15 [28] Kasik D,George H．Towards automatic generation of novice usertest scripts[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Vancouver,Canada,1996:224-251 [29] Brush A,Ames M,et al．A comparison of synchronous remote and local usability studies for an expert interface[C]∥Procee-dings of the SIGCHI Conference on Human Factors in Computing Systems．Vienna,Austria,2004:1179-1182 [30] Pierce J,Strengers Y．Introduction to the Special Issue on Practice-Oriented Approaches to Sustainable HCI [J]．ACM Transactions on Computer-Human Interaction,2013,4(20):1-8 [31] Carlos A,Francisco A．Hands-on experiences of undergraduate students in Automatics and Robotics using a virtual and remote laboratory [J]．Computers & Education,2011,57:2451-2461 [32] 贡宏印．情境感知的远程用户研究工具研究与设计 [D]．大连:大连海事大学信息科学技术学院,2013 [33] Obendorf H,Weinreich H,et al．Automatic support for web user studies with SCONE and TEA,2004[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Vienna,Austria,2004:1135-1138 [34] Atterer R．Logging usage of AJAX applications with the "UsaProxy" HTTP proxy[C]∥Proceedings of International World Wide Web Conference．Edinburgh,Scotland,2006:1-4 [35] Ferreira D,Dey A,et al．Understanding human-smartphone concerns:a study of battery life[C]∥Proceedings of International Conference on Pervasive Computing．San Francisco,USA,2011:19-33 [36] Ma X,Yan B,et al．A toolkit for usability testing of mobile applications[C]∥Proceedings of International Conference on Mobile Computing,Applications and Services computer Sciences．Los Angeles,USA,2011:226-245 [37] Paganelli L,Paterno F．Intelligent analysis of user interactionswith web applications[C]∥Proceedings of International Confe-rence on Intelligent User Interfaces．San Francisco,USA,2002:111-118 [38] Tiedtke T,Mrtin C,et al．AWUSA - A tool for automated website analysis,2002[C]∥Proceedings of Design,Verification and Specification of Interactive Systems conference．Rostock,Germany,2002:251-266 [39] Drachen A,Thurau C,et al．Game Analytics [M]．London:Springer,2013:205-253 [40] Santana V,Baranauskas M．A Prospect of Websites Evaluation Tools Based on Event Logs[C]∥Proceedings of TC 13 Human-Computer Interaction Symposium.Milano,Italy,2008:99-104 [41] Hong J,Heer J,et al．WebQuilt:A proxy-based approach to remote web usability testing [J]．ACM Transactions on Information Systems,2001,19:263-285 [42] Nebeling M,Speicher M,et al．W3Touch:metrics-based Webcontent adaptation for touch[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Paris,France,2013:2311-2320 [43] Porat T,Schclar A．MATE:a mobile analysis tool for usability experts[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Paris,France,2013:265-270 [44] Huang Jin-feng．Remote mobile test system:a mobile phonecloud for application testing[C]∥Proceedings of IEEE International Conference Cloud Computing Technology and Science．Taipei,Taiwan,2012:1-4 [45] Lecerof A,Paternò F．Automatic Support for Usability Evaluation [J]．IEEE Transactions on Software Engineering,1998,24:863-888 [46] Maly I,Slavík P．Towards Visual Analysis of Usability TestLogs Using Task Models[C]∥Proceedings of International Workshop on Task Models and Diagrams for User Interface Design．Hasselt,Belgium,2006:24-38 [47] Propp S,Forbrig P．ViSE:A Virtual Smart Environment for Usability Evaluation[C]∥Proceedings of Human-Centred Software Engineering conference．Reykjavik,Iceland,2010:38-45 [48] Levy M,Dayan A,et al．Task modeling infrastructure for analyzing smart phone usage[C]∥Proceedings of Mobile Business and Global Mobility Roundtable Conference．Athens,Greece,2010:264-271 [49] Hay B,Wets G,et al．Mining navigation patterns using a sequence alignment method [J]．Knowledge and Information Systems,2004,6:150-163 [50] Charfi S,Ezzedine H．Towards an automatic analysis of interaction data for HCI evaluation application to a transport network supervision system[C]∥Proceedings of International Confe-rence on Human-Computer Interaction．Orlando,USA,2011,6761:175-184 [51] Au R W F,Baker S,et al．Automated Usability Testing Framework[C]∥Proceedings of conference on Australasian user interface．Wollongong,NSW,Victoria,Australia,2008:55-64 [52] Chyna P,Sobecki J．Remote Usability Evaluation Using EyeTracking Enhanced with Intelligent Data Analysis[C]∥Proceedings of Design,User Experience,and Usability Conference．Las Vegas,USA,2013:212-221 [53] Google．Google Analytics．http://www.google.com/analytics/index.html [54] Baidu．Baidu mobile statistics．http://mtj.baidu.com/web/welcome [55] Liang H,Song H．A remote usability testing platform for mobile phones[C]∥Proceeding of IEEE International Conference Computer Science and Automation Engineering.Shanghai,China,2011:312-316 [56] Pfeiffer S,Fischer S．Automatic audio content analysis[C]∥Proceedings of the fourth ACM international conference on Multimedia．Rostock,Germany,1996:21-30 [57] Patton R．Software Testing(2nd edition)[M]．USA:Sams Publishing,2006:387-419 [58] Vargas A,Weffers H．A method for remote and semi-automatic usability Evaluation of web-based applications through user behavior analysis[C]∥Proceedings of International Conference on Methods and Techniques in Behavioral Research．Utrecht,Netherlands,2010(19):1-4 [59] Horng D．Data Mining Meets HCI:Making Sense of LargeGraphs [D]．Pittsburgh:Machine Learning Department Carnegie Mellon University,2012 [60] Hammontree M,Weiler P,et al．Remote usability testing [J]．Interactions,1994,1:21-25 [61] Arhippainen L,Ahti M．Empirical evaluation of user experience in two adaptive mobile application prototype[C]∥Proceedings of International Conference on Mobile and Ubiquitous Multimedia．Lulea,Sweden,2003:27-34 [62] Pantic M,Rothkrantz L．Automatic analysis of facial expres-sions:The state of the art [J]．IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22:1424-1445 [63] Johnson P,Wilson S．ADEPT:advanced design environment for prototyping with task models[C]∥Proceedings of the INTERACT and CHI Conference on Human Factors in Computing Systems．Amsterdam,Netherlands,1993:56 [64] Salton G,Buckley C．Term-weighting approaches in automatictext retrieval [J]．Information Processing & Management,1988,24:513-523 [65] Tamminen S,Oulasvirta A,et al．Understanding mobile contexts [J]．Personal and Ubiquitous Computing,2004,8:135-143 [66] Thad S．Wearable computing:through the looking glass[C]∥Proceedings of the 2013 International Symposium on Wearable Computers．Zurich Switzerland,2013:125-126 [67] Kjeldskov J,Skov M．Instant data analysis:conducting usability evaluations in a day[C]∥Proceedings of Nordic conference on Human-computer interaction．Tampere,Finland,2004:233-240 [68] Nielsen J．Usability engineering [M]．Boston:Academic Press,1993:23-43 [69] Souza C,Bentolila D．Automatic evaluation of API usabilityusing complexity metrics and visualizations[C]∥Proceedings of International Software Engineering Conference．Vancouver,Canada,2009:299-302 [70] Liu D,Bias R．Crowdsourcing for usability testing[C]∥Pro-ceedings of the American Society for Information Science and Technology．Baltimore,USA,2012:1-10 [71] Mele M,Federici S．Gaze and eye-tracking solutions for psychological research [J]．Cognitive Processing,2012,13:261-265 [72] Virzi R,Sorce J．A comparison of three usability evaluationmethods:heuristic,think-aloud,and performance testing[C]∥Proceedings of Human Factors and Ergonomics Society Annual Meeting．Washington,USA,1993:309-313 [73] Schnepp J,Shiver B．Improving deaf accessibility in remote usability testing[C]∥Proceedings of international ACM SIGACCESS conference on Computers and accessibility．Dundee,Scotland,2011:255-256 [74] Alcantud F,Coret J．Usability remote evaluation:METBA system[C]∥Proceedings of International Conference on Interactive Collaborative Learning．Villach,Austria,2012:1-8 [75] Nielsen J,Molich R．Heuristic evaluation of user interfaces[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Seattle,USA,1990:249-256 [76] Sohn T,Li K．A diary study of mobile information needs[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Florence,Italy,2008:433-442 [77] Steves M,et al．A comparison of usage evaluation and inspection methods for assessing groupware usability[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Seattle,Washington,USA,2001:125-134 [78] Satoh I．A testing framework for mobile computing software[J]．IEEE Transactions on Software Engineering,2003,29:1112-1121 [79] Engelbrecht K,Quade M,et al．Analysis of a new simulation approach to dialog system evaluation [J]．Speech Communication,2009,51:1234-1252 [80] Paganelli L,Paternò F．Tools for remote usability evaluation of Web applications through browser logs and task models [J]．Behavior Research Methods,Instruments,& Computers,2003,35:369-378 [81] Paterno F,Mancini C．ConcurTaskTrees:a diagrammatic nota-tion for specifying task models[C]∥Proceedings of InternationalConference on Human-Computer Interaction．Sydney,Australia,1997:362-369 [82] Chalil K,Greenstein JS．Synchronous remote usability testing - a new approach facilitated by virtual worlds[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Vancouver,Canada,2011:2225-2234 [83] Vreede G,Fruhling A,et al．A repeatable collaboration process for usability testing[C]∥Proceedings of Hawaii International Conference on System Sciences．Big Island,USA,2005:6 [84] Schusteritsch R,WeiTowards C,et al．Towords the perfect infrastructure for usability testing on mobile devices[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．San Jose,USA,2007:1839-1844 [85] Petrie H,Hamilton F,et al．Remote usability evaluation withdisabled people[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Montréal,Canada,2006:1133-1141 [86] Fourney A,Mann R．Characterizing the usability of interactive applications through query log analysis[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Vancouver,Canada,2011:1817-1826 [87] Song C,Ranganath S．Automatic sign language analysis:a survey and the future beyond lexical meaning [J]．IEEE Transactions on Pattern Analysis and Machine Intelligence,2005,27:873-891 [88] Castillo J,Hartson H,et al．Remote usability evaluation:Can users report their own critical incidents?[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems．Los Angeles,USA,1998:253-254 [89] Paterson M,Dancik V．Longest Common Subsequences[C]∥Proceedings of Mathematical Foundations of Computer Science．Kosice,Slovakia,1994:127-142 [90] Mandryk R,Inkpen K．Using psychophysiological techniques to measure user experience with entertainment technologies [J]．Behaviour & Information Technology,2006,25:141-158 [91] Dirican A,Gktürk M．Psychophysiological measures of human cognitive states applied in human computer interaction[C]∥Proceedings of World Conference on Information Technology．Istanbul,Turkey,2011:1361-1367 [92] Feuerstack S,Blumendorf M,et al．Automated usability evaluation during model-based interactive system development[C]∥Proceedings of Second Conference on Human-Centered Software Engineering and International Workshop on Task Models and Diagrams．Pisa,Italy,2008:134-141 |
| No related articles found! |
|
||
首页