计算机科学 ›› 2017, Vol. 44 ›› Issue (Z6): 1-6.doi: 10.11896/j.issn.1002-137X.2017.6A.001
• 综述研究 • 下一篇
王海鹏,黄天彪,任崇帅,姚五一
WANG Hai-peng, HUANG Tian-biao, REN Chong-shuai and YAO Wu-yi
摘要: 多通道遥交互(Multimodal Tele-interaction,MMTI)旨在通过使用多种交互设备和协作方式,并利用多交互通道间的互补特性,以便有效传达和理解用户交互信息,提高交互效率,增进交互自然性,最终使用户能够以“预期的想法”完成遥交互任务。近年来,随着多通道遥交互的发展,人们对深空、深海和远程医疗的探索和开发不断增强,由于通信时延的约束,多通道遥交互面临着交互异步和通道缺失等问题,对用户行为、心理和认知等人素特性产生了根本影响,切断和阻碍了交互通道的连续性、实时性和自然性,降低了交互的用户体验,并进一步影响了系统的有效性,因此迫切需要对大时延约束下的多通道遥交互技术进行研究。分析了国内外研究现状,给出了遥交互的一个定义,讨论了遥交互研究问题和关键技术(包括时延、异步和缺失问题),讨论了其关键应用领域,最后展望了遥交互的发展趋势和研究挑战。
[1] 董士海,王坚,等.人机交互和多通道用户界面[M].北京:科学出版社,1999. [2] 周芝庭,帅立国,等.互联网应用中触觉通信时延问题分析[J].测控技术,2008,27(4):53-55. [3] VITTORIAS I,KAMMERL J,HIRCHE S,et al.Perceptual co-ding of haptic data in time-delayed teleoperation[C]∥EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.2009:208-213. [4] HIRCHE S,BUSS M.Transparent data reduction in networked telepresence and teleaction systems.part ii:Time-delayed communication[J].Presence:Teleoperators and Virtual Environments,2007,16(5):532-542. [5] HINTERSEER P,STEIBACH E,CHAUDHURI S.Model baseddata compression for 3D virtual haptic teleinteraction[C]∥Digest of Technical Papers.International Conference on Consumer Electronics.2006:23-24. [6] SUOMELA J.Tele-presence Aided Teleoperation of Semi-au-tonomous Work Vehicles[D].Helsinki University of Technology,2001. [7] 朱士俊.远程医学的建立和发展[J].解放军医院管理,1998,5(3):297. [8] 张珩,李庚田.遥科学的概念、 应用与发展[J].中国航天,1997(11):15-20. [9] SATO K,HISATAKA T.Unmanned system for heavy con-struction equipment[C]∥Proc.of the 5th Int.Symp.on Robo-tics in Construction.Tokyo,Japan,1988:449-458. [10] FERRELL W R.Remote manipulation with transmission delay[J].IEEE Transactions on Human Factors in Electronics,1965 (1):24-32. [11] PAVLOVYCH,ANDRIY,STUERZLINGER W.Target following formance in the presence of latency,jitter,and signal dropouts[C]∥Proceedings of Graphics Interface Canadian Human-Computer Communications Society.2011:33-40. [12] PANTEL L,WOLF L C.On the impact of delay on real-timemultiplayer games[C]∥Proceedings of the 12th International Workshop on Network and Operating Systems Support for Di-gital Audio and Video.ACM,2002:23-29. [13] KAHNEMAN D.Attention and effort[M].Englewood Cliffs,NJ:Prentice-Hall,1973. [14] 黄攀峰,刘正雄.空间遥操作技术[M].长沙:国防工业出版社,2015. [15] YORIZAWA I,TAKIKAWA K,OKUBO S,et al.Subjective ef-fects of talker echo and transmission delay in video conferencing via communication satellite[J].Trans.IECE Japan,1981,64(11):1281-1288. [16] LIU Z,HEER J.The effects of interactive latency on exploratory visual analysis[J].IEEE Transactions on Visualization and Computer Graphics,2014,20(12):2122-2131. [17] KIM T,ZIMMERMAN P M,WADE M J,et al.The effect of delayed visual feedback on telerobotic surgery[J].Surgical Endoscopy And Other Interventional Techniques,2005,19(5):683-686. [18] Smith K U,WARGO L,JONES R,et al.Delayed and space-displaced sensory feedback and learning[J].Perceptual and Motor Skills,1963,16(3):781-796. [19] FERRELL W R,SHERIDAN T B.Remote manipulative control with transmission delay[J].IEEE Transactions on Human Factors in Electronics,1963,HFE-4(1):25-29. [20] SMITH W M,MCCRAY J W,SMITH K U.Delayed visual feedback and behavior[J].Science,1960,132(3433):1013-1014. [21] BRUTLAG J.Speed matters for Google web search[R].Google.June,2009. [22] JOTA R,NG A,DIETZ P,et al.How fast is fast enough?:a study of the effects of latency in direct-touch pointing tasks[C]∥Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.ACM,2013:2291-2300. [23] MACKENZIE IS,WARE C.Lag as a determinant of human performance in interactive systems[C]∥Proceedings of the INTERACT’93 and CHI’93 Conference on Human Factors in Computing Systems.ACM,1993:488-493. [24] MIYASATO T,NAKATSU R.Allowable delay between images and tactile information in a haptic interface[C]∥International Conference on Virtual Systems and MultiMedia,1997.IEEE,1997:84-89. [25] JAY C,GLENCROSS M,HUBBOLD R.Modeling the effects of delayed haptic and visual feedback in a collaborative virtual environment[J].ACM Transactions on Computer-Human Interaction (TOCHI),2007,14(2):8. [26] KAARESOJA T,ANTTILA E,HOGGAN E.The effect oftactile feedback latency in touchscreen interaction[C]∥2011 IEEE World Haptics Conference (WHC).IEEE,2011:65-70. [27] HIRZINGER G,BRUNNER B,DIETRICH J,et al.ROTEX-the first remotely controlled robot in space[C]∥1994 IEEE International Conference on Robotics and Automation,1994.IEEE,1994:2604-2611. [28] ANVARI M,BRODERICK T,STEIN H,et al.The impact of latency on surgical precision and task completion during robotic-assisted remote telepresence surgery[J].Computer Aided Surgery,2005,10(2):93-99. [29] LEWIS C H,GRIFFIN M J.Human factors consideration inclinical applications of virtual reality[J].Studies in Health Technology and Informatics,1997:35-58. [30] LAWTHER A,GRIFFIN M J.The motion of a ship at sea and the consequent motion sickness amongst passengers[J].Ergonomics,1986,29(4):535-552. [31] HETTINGER L J,RICCIO GE.Visually induced motion sickness in virtual environments[J].Presence:Teleoperators & Virtual Environments,1992,1(3):306-310. [32] WLOKA M M.Lag in multiprocessor virtual reality[J].Pre-sence:Teleoperators & Virtual Environments,1995,4(1):50-63. [33] LIU A,THARP G,FRENCH L,et al.Some of what one needs to know about using head-mounted displays to improve tele-operator performance[J].IEEE Transactions on Robotics and Automation,1993,9(5):638-648. [34] THARP G K,LIU AM,FRENCH L,et al.Timing considerations of helmet-mounted display performance[C]∥SPIE/IS&T 1992 Symposium on Electronic Imaging:Science and Technology.International Society for Optics and Photonics,1992:570-576. [35] LIU A,THARP G,HIROSE M,et al.Visual factors affectinghuman operator performance with a helmet-mounted display[R].SAE Technical Paper,1991. [36] HELD R,DURLACH N.Telepresence,time delay and adapta-tion[J].Pictorial Communication in Virtual and Real Environments,1991:232-246. [37] DWYER T F.Telepsychiatry:psychiatric consultation by interactive television[J].American Journal of Psychiatry,1973,130(8):865-869. [38] NORDAHL R.Self-induced footsteps sounds in virtual reality:Latency,recognition,quality and presence[C]∥The 8th Annual International Workshop on Presence(PRESENCE 2005).2005:353-355. [39] MOUTOUSSIS K.Functional segregation and temporal hierarchy of the visual perceptive systems[J].Proceedings of the RoyalSociety of London B:Biological Sciences,1997,264(1387):1407-1414. [40] FERRELL W R.Remote manipulation with transmission delay[J].IEEE Transactions on Human Factors in Electronics,1965,(1):24-32. [41] DIXON N F,SPITZ L.The detection of auditory visual desynchrony[J].Perception,1980,9(6):719-721. [42] SHI Z,ZOU H,MULLER H J.Temporal perception of visual-haptic events in multimodal telepresence system[M].In Tech,2010. [43] VATAKIS A,SPENCE C.Evaluating the influence of framerate on the temporal aspects of audiovisual speech perception[J].Neuroscience Letters,2006,405(1):132-136. [44] VAN ERP J B F,WERKHOVEN P J .Vibro-tactile and visual asynchronies:Sensitivity and consistency[J].Perception,2004,33(1):103-111. [45] FJISKAI W,SHIMOJO S,KASHINO M,et al.Recalibration of audiovisual simultaneity[J].Nature Neuroscience,2004,7(7):773-778. [46] STONE J V,HUNKIN N M,PORRILL J,et al.When is now? Perception of simultaneity[J].Proceedings of the Royal Society of London B:Biological Sciences,2001,268(1462):31-38. [47] SPENCE C,SHORE D I,KLEIN R M.Multisensory prior entry[J].Journal of Experimental Psychology:General,2001,130(4):799. [48] DIXON N F,SPITZ L.The detection of auditory visual desynchrony[J].Perception,1980,9(6):719-721. [49] LEVITIN D J,MACLEAN K,MATHEWS M,et al.The perception of cross-modal simultaneity[J].Proc.of International Journal of Computing Auticipatory Systems,2008,517(1):323-329. [50] VOGELS I M L C.Detection of temporal delays in visual-haptic interfaces[J].Human Factors:The Journal of the Human Factors and Ergonomics Society,2004,46(1):118-134. [51] THOMPSON JM,OTTENSMEYER M P.Human factors in telesurgery:effects of time delay and asynchrony in video and control feedback with local manipulative assistance[J].Telemed J,1999,5(2):29-37. [52] SPENCE C,SQUIRE S.Multisensory integration:maintaining the perception of synchrony[J].Curr Biol,2003,13(13):R519-R521. [53] ANDERSON R J,SPONG MW.Bilateral control of teleoperators with time delay[J].IEEE Transactions on Automatic Control,1989,34(5):494-501. [54] KIM T,ZIMMERMAN P M,WADE M J,et al.The effect of delayed visual feedback on telerobotic surgery[J].Surgical Endoscopy And Other Interventional Techniques,2005,19(5):683-686. [55] ZAMPINI M,SHORE D I,SPENCE C.Audiovisual temporalorder judgments[J].Experimental Brain Research,2003,152(2):198-210. [56] SPENCE C,SHORE D I,KLEIN R M.Multisensory prior entry[J].Journal of Experimental Psychology:General,2001,130(4):799-832. [57] ZAMPINI M,BROWN T.Audiotactile temporal order judg-ments[J].Acta Psychologica,2005,118(3):277-291. [58] BRANDI F,KAMMERL J,STEINBACH E.Error-resilient perceptual coding for networked haptic interaction[C]∥Procee-dings of the 18th ACM International Conference on Multimedia.ACM,2010:351-360. [59] SHI Z,ZOU H,RANK M,et al.Effects of packet loss and latency on the temporal discrimination of visual-haptic events[J].IEEE Transactions on Haptics,2010,3(1):28-36. [60] FJISAKI W,SHIMOJO S,KASHNO M,et al.Recalibration of audiovisual simultaneity[J].Nature Neuroscience,2004,7(7):773-778. [61] TAM J,CARTER E,KIESLER S,et al.Video increases the perception of naturalness during remote interactions with latency[C]∥Extended Abstracts on Human Factors in Computing Systems(CHI’12).ACM,2012:2045-2050. [62] VITTORIALS I,KAMMERL J,HIRCHE S,et al.Perceptualcoding of haptic data in time-delayed teleoperation[C]∥EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems(World Haptics 2009).Third Joint,IEEE,2009:208-213. [63] HIRCHE S,HINTERSEER P,STEINBACH E,et al.Transpa-rent data reduction in networked telepresence and teleaction systems.part i:Communication without time delay[J].Presence:Teleoperators and Virtual Environments,2007,16(5):523-531. [64] 俸文.多通道人机交互技术的研究[D].南京:南京理工大学,2004. [65] TAKAHASHI Y,MASUDA I.A visual interface for securityrobots[C]∥IEEE International Workshop on Robot and Human Communication,1992.IEEE,1992:123-128. [66] LIN Q,KUO C.Virtual tele-operation of underwater robots[C]∥1997 IEEE International Conference on Robotics and Automation,1997.IEEE,1997,2:1022-1027. [67] 徐旭明,叶榛.基于视觉临场感的机器人遥操作系统[J].高技术通讯,2000(3):57-61. [68] 艾海舟,张朋飞.室外移动机器人的视觉临场感系统[J].机器人,2000,22(1):28-32. [69] IMAIDA T,YOKOKOHJI Y,DOI T,et al.Ground-space bila-teral teleoperation of ETS-VII robot arm by direct bilateral coupling under 7-s time delay condition[J].IEEE Transactions on Robotics and Automation,2004,20(3):499-511. [70] HELD R,EFSTATHIOU A,GREENE M,et al.Adaptation to Displaced and Delayed Visual Feedback from the Hand[J].Journal of Experimental Psychology:General,1966,2(6):887-891. [71] BUSS M,PEER A,SCHAUSS T,et al.Multi-modal multi-user telepresence and teleaction system[C]∥IEEE/RSJ International Conference on Intelligent Robots and Systems,2008(IROS 2008).IEEE,2008:4137-4138. [72] USS M,PEER A,SCHAUSS T,et al.Development of a Multi-modal Multi-user Telepresence and Teleaction System[J].The International Journal of Robotics Research,2010,29(10):1298-1316. [73] FERRARI E,CAVALLO E.Operator-machine interaction:ergonomic evaluation of a vineyard teleoperated machine[C]∥International Conference of Agricultural Engineering-CIGR-AgEng 2012:Agriculture and Engineering for a Healthier Life.Valencia,Spain,CIGR-EurAgEng,2012. [74] STASSEN H G,SMETS G J F.Telemanipulation and Telepre-sence[J].Control Eng.Practice,1997,5(3):363-374. [75] 徐旭明,叶棒.基于视觉临场感的机器人遥操作系统[J].高技术通讯,2000,10(3):57-60. [76] 戴炬.异机构遥控主从手的双向力反映[J].控制理论和应用,1996(13):259-261. [77] 游松.基于国际互联网的遥操作机器人系统的研究[D].北京:北京航空航天大学,2000. [78] 吴国钊,傅雪冬.基于Intemet的机器人实时跟踪系统[J].机器人技术与应用,2000(1):24-26. [79] 马宏鹏,安刚.基于互联网的机器人遥操作系统平台[J].自动化与仪表,2000,15(6):3-7. [80] 韦庆,李杰.基于事件反馈的机器人监控技术及其在大时延遥操作中的应用[C]∥空间机器人及遥科学技术研讨会论文集.1999:307-313. [81] 陈辉,宋爱国.带有力觉和触觉临场感的灵巧手主从系统的设计[J].机器人,1998,20(6):437-441. [82] 宋爱国,曾庆军.FTP技术中多传感器信号的无源传输算法[J].电子学报,1996,24(11):108-112. [83] 刘准.国外深海技术发展研究[J].船艇,2006(10):6-8. [84] SAGER W W,SANO T,GELDMACHER J.IODP Expedition324:Ocean Drilling at Shatsky Rise Gives Clues about Oceanic Plateau Formation[J].Scientific Drilling,2011(12):24-31. [85] GUPTA H,NAYAK S,COMMITTEE K W.Deep scientificdrilling to study reservoir-triggered Earthquakes in Koyna,Western India[J].Scientific Drilling,2011,12:53-54. [86] 莫杰,肖菲.深海探测技术的发展[J].科学,2012,4(5):11-15. [87] 徐文,钱洪宝,张杰,等.我国海洋监测高技术发展的回顾与思考[J].海洋技术学报,2015,34(3):59-61. [88] LABONTE D,BOISSY P,MICHAUD F.Comparative analysis of 3-D robot teleoperation interfaces with novice users[J].IEEE Transactions on Systems,Man,and Cybernetics(Part B),2010,40(5):1331-1342. [89] HUBER D,HERMAN H,KELLY A,et al.Real-time photo-rea-listic visualization of 3D environments for enhanced tele-operation of vehicles[C]∥IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops),2009.IEEE,2009:1518-1525. [90] KIM S,KANG J,CHUNG M J.Probabilistic voxel mappingusing an adaptive confidence measure of stereo matching[J].Intelligent Service Robotics,2013,6(2):89-99. [91] 马厦飞,连琏.“海马”号 4500 米级 ROV 系统研发历程[J].船舶与海洋工程,2015,31(1):9-12. [92] 高艳波,李慧青.深海高技术发展现状及趋势[J].海洋技术,2010,29(3):119-124. [93] DWYER T F.Telepsychiatry:psychiatric consultation by interactive television[J].American Journal of Psychiatry,1973,130(8):865-869. [94] SINGH R,JEONG B,RENAMBOTET L.TeraVision:a distributed,scalable,high resolution graphics streaming system[C]∥2004 IEEE International Conference on Cluster Computing.IEEE,2004:391-400. [95] JEONG B.Collaborative Visualization Architeeturein Scalable A-daptive Graphics Environment[C]∥IBM Visualization and Graphics Student Symposium,TJ Watson Research Hawthorne and Yorktown.2007. [96] FRIESEN J A,TARMAN T D.Remote high-performance visua-lization and collaboration[J].IEEE Computer Graphics and Applications,2000,20(4):45-49. [97] YHAKUR ,BAX J S,HOLDSWORTH D W,et al.Design and performance evaluation of a remote catheter navigation system[J].IEEE Transactions on Biomedical Engineering,2009,56(7):1901-1908. [98] 刘浩.导管机器人系统的建立及其关键技术研究[D].哈尔滨:哈尔滨工业大学,2010. [99] 李显凌.用于微创介入手术的导管导向机器人研究[D].哈尔滨:哈尔滨工业大学,2009. [100] RAHMAN S M M,IKEURA R,NOBE M,et al.Control of a power assist robot for lifting objects based on human operator’s perception of object weight[C]∥The 18th IEEE International Symposium on Robot and Human Interactive Communication,2009(RO-MAN 2009).IEEE,2009:84-90. [101] WATANABE T,YOSHIKAWA T.Grasping optimization using a required external force set[J].IEEE Transactions on Automation Science and Engineering,2007,4(1):52-66. [102] SHELTON J N,CHIU G T C,PIZLO Z.Exponentially segmented positioning of a single link mechanism:A control algorithm that satisfies Fitts’ Law[C]∥American Control Conference,2007(ACC’07).IEEE,2007:5983-5988. [103] 熊友军.基于增强现实的遥操作关键技术研究[D].武汉:华中科技大学,2005. |
No related articles found! |
|