Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (12): 2-11.doi: 10.11896/jsjkx.240400166

• Integration of Digital Twin Network and Artificial Intelligence • Previous Articles     Next Articles

Review of Key Technologies,Research Progress and Applications of Metaverse

WANG Wentong1,2, ZHANG Zhijun2, ZHANG Mingyang2   

  1. 1 School of Design, South China University of Technology, Guangzhou 510006, China
    2 School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, China
  • Received:2024-04-23 Revised:2024-08-08 Online:2024-12-15 Published:2024-12-10
  • About author:WANG Wentong,born in 1985,Ph.D candidate.His main research interests include human-computer interaction and virtual reality.
    ZHANG Zhijun,born in 1982,Ph.D,professor,Ph.D supervisor.His main research interests include the theoretical research and application of dynamic self-learning neural network and robot motion planning.
  • Supported by:
    National Natural Science Foundation(62373157),National High-Level Talents Special Support Program(Youth Talent of Technological Innovation of Ten-Thousands Talents Program)(A4220130),International Scientific Research Cooperation Project of Guangdong Science and Technology Plan(2023A0505050083),National Key Research and Development Program of China(2017YFB1002505) and Soft Science Research Program of Guangdong Province(2024A1010030001).

PDF (PC)

404

Abstract: With the rapid development of digital technology,metaverse has become one of the focuses of people’s attention.As a new kind of virtual world,the metaverse will redefine the way people live and work.In this paper,the concept and significance of the metaverse are introduced,and the key technologies of the metaverse are studied in depth from the technical characteristics of the metaverse.Specifically,this paper analyzes the six technologies of blockchain,interaction technology,artificial intelligence,Internet of things,computing power and operations,and digital twin,summarizes the research progress,problems and challenges of the key technologies of the metaverse,and prospects the research direction,development trend,and application prospect of the future metaverse.

Key words: Metaverse, Blockchain, Expanding reality, Artificial intelligence, Digital twin

CLC Number: 

  • TP393
[1]DUAN H,LI J,FAN S,et al.Metaverse for social good:A university campus prototype[C]//Proceedings of the 29th ACM International Conference on Multimedia.2021:153-161.
[2]FANG L,SHEN H.Changes in technology and civilization－Aconceptual study of the Metaverse[J].Metaverse,2021,2(2):18.
[3]NG,DAVY TSZ KIT.What is the metaverse? Definitions,technologies and the community of inquiry[J].Australasian Journal of Educational Technology,38.4(2022):190-205.
[4]CHEN Y,CHENG H.The economics of the metaverse:A comparison with the real economy[J].Metaverse,2022,3(1):19.
[5]PARK S M,KIM Y G.A Metaverse:Taxonomy,components,applications,and open challenges[J].IEEE Access,2022,10:4209-4251.
[6]HUYNH-THE T,GADEKALLU T R,WANG W Z,et al.Blockchain for the metaverse:A Review[J].Future Generation Computer Systems,2023(143):401-419.
[7]JEON H J,YOUN H C,KO S M,et al.Blockchain and AI Meet in the Metaverse[M]//Advances in the Convergence of Blockchain and Artificial Intelligence.2022:73-82.
[8]ALI M,VECCHIO M,PINCHEIRA M,et al.Applications of blockchains in the Internet of things:a comprehensive survey[J].IEEE Communications Surveys & Tutorials,2019,21:1676-1717.
[9]MOHURLE S,PATIL M.A brief study of wannacry threat:Ransomware attack 2017[J].International Journal of Advanced Research in Computer Science,2017,8(5):1938-1940.
[10]MURTHY G R S,JADON R S.A review of vision based hand gestures recognition[J].International Journal of Information Technology and Knowledge Management,2009,2(2):405-410.
[11]GRIMES G J.Digital data entry glove interface device[P].US4414537A[2024-10-08].DOI:US4414537A.
[12]Leap Motion Inc.Leap Motion[EB/OL].https://www.leap-motion.com.
[13]SAKSHI P A,WAQAR M N.Efficacy of Leap Motion tracking device in Rehabilitation[J].Journal of medical pharmaceutical and allied sciences,2022,11(4):5214-5216.
[14]POTTER L E,ARAULLO J,CARTER L.The leap motion controller:a view on sign language[C]//Proceedings of the 25th Australian Computer-Human Interaction Conference:Augmentation,Application,Innovation,Collaboration.New York:ACM Press,2013:175-178.
[15]QI J,CUI Z J,YU Y S,et al.Computer vision-based hand gesture recognition for human-robot interaction:a review[J].Complex & Intelligent Systems,2024(10):1581-1606.
[16]GRACIA-IBÁÑEZ V,VERGARA M,SANCHO-BRU J L,et al.Functional range of motion of the hand joints in activities of the International Classification of Functioning,Disability and Health[J].Journal of Hand Therapy,2017,30(3):337-347.
[17]NOY L,FRIEDMAN J,FLASH T.Evaluating the reconstruction of virtual hand movements from real hand movements measured using a data-glove[J].2011.https://www.researchgate.net/publication/235631077_Evaluating_the_reconstruction_of_virtual_hand_movements_from_real_hand_movements_mea-sured_using_a_data-glove.
[18]REN Z,YUAN J,MENG J,et al.Robust part-based hand gesture recognition using kinect sensor[J].IEEE transactions on multimedia,2013,15(5):1110-1120.
[19]BILMES J A.What HMMs can do[J].IEICETRANSACTIONS on Information and Systems,2006,89(3):869-891.
[20]LI J,YU D,HUANG J T,et al.Improving wideband speech re-cognition using mixed-bandwidth training data in CD-DNN-HMM[C]//2012 IEEE Spoken LanguageTechnology Workshop(SLT).IEEE,2012:131-136.
[21]SHAHIN M,AHMED B,MCKECHNIE J,et al.A comparison of GMM-HMM and DNN-HMM based pronunciation verification techniques for use in the assessment of childhood apraxia of speech[C]//Fifteenth Annual Conference of the International Speech Communication Association.2014:1583-1587.
[22]HANNUN A,CASE C,CASPER J,et al.Deep speech:Scaling-up end-to-end speech recognition[J].arXiv:1412.5567,2014.
[23]GRAVES A,JAITLY N.Towards end-to-end speech recogni-tion with recurrent neural networks[C]//International Conference on Machine Learning.PMLR,2014:1764-1772.
[24]WATANABE S,HORI T,KIM S,et al.Hybrid CTC/atten-tion architecture for end-to-end speech recognition[J].IEEEJournal of Selected Topics in Signal Processing,2017,11(8):1240-1253.
[25]CHAN W,JAITLY N,LE Q V,et al.Listen,attend and spell[J].arXiv:1508.01211,2015.
[26]LI J,ZHAO R,MENG Z,et al.Developing RNN-T models surpassing high-performance hybrid models with customization capability[J].arXiv:2007.15188,2020.
[27]SAINATH T N,HE Y,LI B,et al.A streaming on-device end-to-end model surpassing server-side conventional model quality and latency[C]//ICASSP 2020－2020 IEEE International Conference on Acoustics,Speech andSignal Processing(ICASSP).IEEE,2020:6059-6063.
[28]LIU G P.Implementation of Dielectric Free Floating Projection Interactive System with Haptic Feedback[D].Hangzhou:Zhejiang University of Science and Technology,2021.
[29]YANG P.Air Haptic Feedback Technology Based on UltrasonicRadiation Pressure[D].Changchun:Jilin University,2019.
[30]LONG B,SEAH S A,CARTER T,et al.Rendering volumetric haptic shapes in mid-air using ultrasound[J].ACM Transactions on Graphics(TOG),2014,33(6):181.
[31]CARTER T,SEAH S A,LONG B,et al.Ultra Haptics:multi-point mid-air haptic feedback fortouch surfaces[C]//Procee-dings of the 26th annual ACM symposium on User interface software and technology.ACM,2013:505-514.
[32]MONNAI Y,HASEGAWA K,FUJIWARA M,et al.HaptoMime:mid-air haptic interaction with afloating virtual screen[C]//ACM.2014:663-668.
[33]LIU Z,HU X N,BO R H,et al.A three-dimensionally architected electronic skin mimicking human mechanosensation[J].Science,2024(384):987-994.
[34]KERRUISH E.Arranging sensations:smell and taste in augmented and virtual reality[J].The Senses and Society,2019,14(1):31-45.
[35]MATSUKURA H,ISHIDA H.Review on development of devices for amplifying human olfaction:Approaches using real and virtual concentration method[J].Electronics & Communications in Japan,2019,102(2):55-60.
[36]CHEN Y B.Research on olfactory presentation technology of biological virtual experiment[D].Guangzhou:Guangdong University of Technology,2020.
[37]WANG C,CHEN Z S,CHAN C L J,et al.Biomimetic olfactory chips based on large-scale monolithically integrated nanotube sensor arrays[J].Nature Electronics,2024(7):157-167.
[38]BERA T K.Brain Computer Interfaces:Current Trends and Applications[M].Berlin:Springer,2015.
[39]MASON S G,BIRCH G E.A general framework for brain-computer interface design[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2003,11(1):70-85.
[40]XU W Q,CHEN L,SUI X F,et al.Brain Computer interface:Brain information reading and Brain activity regulation technology[J].Chinese Science Bulletin,2023(8):927-943.
[41]WU Z,ZHENG N,ZHANG S,et al.Maze learning by a hybrid brain-computer system[J].Scientific Reports,2016,6(1):1-12.
[42]FLESHER S N,DOWNEY J E,WEISS J M,et al.A brain-computer interface that evokes tactile sensations improves robotic arm control[J].Science,2021,372(6544):831-836.
[43]JOHNSON W G,OURNAS L M.Why Elon Musk’s newbrain-computer Interface?[J].English Abstract,2020(11):6.
[44]LEE K,LIU D,PERROUD L,et al.A brain-controlled exoske-leton with cascaded event-related desynchronization classifiers[J].Robotics and Autonomous Systems,2017,90:15-23.
[45]ZHANG Z,HUANG Y,CHEN S,et al.An intention-drivensemi-autonomous intelligent robotic system for drinking[J].Frontiers in Neurorobotics,2017,11:48.
[46]KAWALA-STERNIUK A,BROWARSKA N,AL-BAKRI A,et al.Summary of over fifty years with brain-computer interfaces－a review[J].Brain Sciences,2021,11(1):43.
[47]RABAEY J M.Brain-machine interfaces as the new frontier in extreme miniaturization[C]//2011 Proceedings of the European Solid-State Device Research Conference(ESSDERC).IEEE,2011:19-24.
[48]ZOU L,TIAN H,GUAN S,et al.Self-assembled multifunc-tional neural probes for precise integration of optogenetics and electrophysiology[J].Nature communications,2021,12(1):1-9.
[49]GUO Q Z.The origin of the universe,the present situation and future[J].News,2022(1):26-31.
[50]KRIZHEVSKY A,SUTSKEVER I,HINTON G E.Imagenetclassification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[51]BENGIO Y,DUCHARME R,VINCENT P.A neural probabilistic language model[J].The Journal of Machine Learning Research,2000,3:1137-1151.
[52]XU B,LIU C,ZENG Y.Research status and developments ofbrain-inspired intelligence[J].Journal of the Chinese Academy of Sciences,2016,31(6):793-802.
[53]LYU J C,YE Q,TIAN Y X,et al.Brain-inspired Large-scale Deep Neural Network System[J].Ruan Jian Xue Bao/Journal of Software,2022,33(4):1412-1429.
[54]ZHANG M M,TANG Y,HE Y,et al.Review of Application Progress of Digital Twin in CNC Machining[J].Digital Technology and Application,2012,40(10):125-127.
[55]YANG J,ZHANG A S,PANG B,et al.A Review on the Deve-lopment of Metaverse Technology and Its Application Prospect in the field of Architecture[J].Chinese Journal of Civil and Environmental Engineering,2024(1):33-45.
[56]LIU Z S,ZHANG A S,WANG W S,et al.Fire safety dynamic evacuation method for Winter Olympic Stadium driven by digital twin[J].Journal of Tongji University(Natural Science Edition),2020,48(7):962-971.
[57]ZHANG X W,WANG L.Research on digital twin Technology and its Application in Library:Taking the Library Construction of Xiongan New Area as an example[J].Library and Information Work,2020,64(17):64-73.
[58]LU Q,XIE X,PARLIKAD A K,et al.Digital twin-enabledanomaly detection for built asset monitoring in operation and maintenance[J].Automation in Construction,2020,118:103277.
[59]GUO F Y,LIU J H,ZOU F,et al.Research on Assembly Process Design Status and Key Implementation Technology of Digital Twin Driven[J].Journal of Mechanical Engineering,2019,55(17):110-132.
[60]LIU D T,GUO K,WANG B K,et al.Review and prospect of digital twin technology[J].Chinese Journal of Scientific Instrument,2018,39(11):1-10.
[61]SHREYAS K,MAURYA S,NERMA R,et al.Green CloudComputing for the Metaverse:Powering the Future of the Internet with Renewable Energy[C]//2024 11th International Conference on Reliability,Infocom Technologies and Optimization(Trends and Future Directions)(ICRITO).IEEE,2024.
[62]KUMAER K K,RAMARAJE,INDIRADNVSLS.Data Fusion Method and Internet of Things(IoT) for Smart City Application[C]//2021 Third International Conference on Intelligent Communication Technologies and Virtual Mobile Networks(ICICV).IEEE,2021.
[63]ZHOU Y J,HUANG H,YUAN S H,et al.Metafi++:Wifi-enabled transformer-based human pose estimation for metaverse avatar simulation[J].IEEE Internet of Things Journal,2023,10(16):14128-14136.
[64]KHAN M A,SALAH K.IoT security:Review,blockchain solutions,and open challenges[J].Future Generation Computer Systems,2018,82:395-411.
[65]ZHONG J P,ZHENG Y X.Empowering future education:Learning in the Edu-Metaverse[C]//Proceedings of the 2022 International Symposium on Educational Technology(ISET’22).IEEE,2022:292-295.
[66]CHEN F,SUN C H,XING B,et al.Agricultural Metaverse:Key Technologies,Application Scenarios,Challenges and Prospects[J].Smart Agriculture,2022,4(4):126-137.
[67]Super code.Super code Hard core released the first metaverse wisdom tea garden in Zhejiang,and the high play of digital agriculture came[EB/OL].[2022-06-10].https://xw.qq.com/cmsid/20220313A01C9300.
[68]NICOLA K.Whatisthefuture of the Metaverse for in-door agriculture?[EB/OL].[2022-05-18].https://www.hortidaily.com/article/9406592/what-is-the-future-ofthe-metaverse-for-indoor-agriculture/.
[69]LEE J,KUNDU P.Integrated cyber-physical systems and industrial metaverse for remote manufacturing[J].Manufacturing Letters,2022,34:12-15.
[70]YAO X,MA N,ZHANG J,et al.Enhancing wisdom manufacturing as industrial metaverse for industry and society 5.0[J].Journal of Intelligent Manufacturing,2022,35:235-255.
[71]ÇÖLTEKIN A,LOCHHEAD I,MADDEN M,et al.Extendedreality in spatial sciences:A review of research challenges and future directions[J].ISPRS International Journal of Geo-Information,2020,9(7):439.
[72]SHARMAS,SINGH J,GUPTA A,et al.User Safety and Secu-rity in the Metaverse:A Critical Review[J].IEEE Open Journal of the Communications Society,2024,5:5467-5487.
[73]OTOUM Y,GOTTIMUKKALA N,KUMAR N,et al.Machine Learning in Metaverse Security:Current Solutions and Future Challenges[J].ACM Computing Surveys,2024,56(8):1-36.
[74]KAUR R,DUŠAN G,TOMAŽ K.Artificial intelligence for cybersecurity:Literature review and future research directions[J].Information Fusion,2023,97:101804.
[75] DIPANKAR D,AKHTAR Z,SEN S.Machine learning in cybersecurity:a comprehensive survey[J].The Journal of Defense Modeling and Simulation,2022,19:57-106.
[76]JOHAN S.Metaverse and its implication in law and business[J].Jurnal Hukum Progresif,2022,10(2):153-166.
[77]KURU K.MetaOmniCity:Towards immersive urban metaverse cyberspaces using smart city digital twins[J].IEEE Access,2023,11:43844-43868.
[78]Bitcoin Energy Consumption Index.[EB/OL].https://digiconomist.net/bitcoin-energyconsumption.
[79]BUYYA R,ILAGER S,ARROBA P.Energy-efficiency and sustainability in new generation cloud computing:A vision and directions for integrated management of data centre resources and workloads[J].Software:Practice and Experience,2024,54(1):24-38.
[80]ONIK M M H,KIM C S,LEE L Y,et al.Privacy-aware blockchain for personal data sharing and tracking[J].Open Computer Science,2019,9(1):80-91.
[81]WANG J Q,WU Z H,LI Y W,et al.Large language models for robotics:Opportunities,challenges,and perspectives[J].arxiv:2401.04334,2024.
[82]HUYNH-THE T,PHAM Q V,PHAM X Q,et al.Artificial intelligence for the metaverse:A survey[J].Engineering Applications of Artificial Intelligence,2023,117(A):105581.
[83]CHEN C Y,ZHANG K Z K,CHU Z F,et al.Augmented reality in the metaverse market:the role of multimodal sensory interaction[J].Internet Research,2024,34(1):9-38.
[1] WANG Dong, LI Xiaoruo, ZHU Bingnan. Transaction Granularity Modifiable Consortium Blockchain Scheme Based on Dual Merkel Trees Block Structure [J]. Computer Science, 2024, 51(9): 408-415.
[2] ZANG Wenyang, LYU Jinlai. Study on Time Rotation Notary Group Model Based on Threshold Signature [J]. Computer Science, 2024, 51(8): 403-411.
[3] XIANG Yanjie, HUANG Xiaofang, XIANG Kefeng, ZHENG Ji’nan. Blockchain Certificateless Encryption Mechanism Based on National Secret Algorithm [J]. Computer Science, 2024, 51(8): 440-446.
[4] SUN Li. Application,Challenge and New Strategy of Block Chain Technology in Metaverse [J]. Computer Science, 2024, 51(7): 373-379.
[5] LI Zhiyuan, XU Binglei, ZHOU Yingyi. Blockchain Anonymous Transaction Tracking Method Based on Node Influence [J]. Computer Science, 2024, 51(7): 422-429.
[6] ZANG Hongrui, YANG Tingting, LIU Hongbo, MA Kai. Study on Cryptographic Verification of Distributed Federated Learning for Internet of Things [J]. Computer Science, 2024, 51(6A): 230700217-5.
[7] TANG Xin, SUN Yufei, WANG Yujue, SHI Min, ZHU Dengming. Three Layer Knowledge Graph Architecture for Industrial Digital Twins [J]. Computer Science, 2024, 51(6A): 230400153-6.
[8] ZHU Jun, ZHANG Guoyin, WAN Jingjing. Study on Data Security Framework Based on Identity and Blockchain Integration [J]. Computer Science, 2024, 51(6A): 230400056-5.
[9] LAN Yajie, MA Ziqiang, CHEN Jiali, MIAO Li, XU Xin. Survey on Application of Searchable Attribute-based Encryption Technology Based on Blockchain [J]. Computer Science, 2024, 51(6A): 230800016-14.
[10] TAN Jingqi, XUE Lingyan, HUANG Haiping, CHEN Long, LI Yixuan. Data Security Management Scheme Based on Editable Medical Consortium Chain [J]. Computer Science, 2024, 51(6A): 240400056-8.
[11] KANG Zhong, WANG Maoning, MA Xiaowen, DUAN Meijiao. New Design of Redactable Consortium Blockchain Scheme Based on Multi-user Chameleon Hash [J]. Computer Science, 2024, 51(6A): 230600004-6.
[12] GENG Qian, CHUAI Ziang, JIN Jian. Operational Consistency Model Based on Consortium Blockchain for Inter-organizational Data Exchange [J]. Computer Science, 2024, 51(6A): 230800145-9.
[13] TIAN Hongliang, XIAN Mingjie, GE Ping. Fine Grained Security Access Control Mechanism Based on Blockchain [J]. Computer Science, 2024, 51(6A): 230400080-7.
[14] JIANG Rui, YANG Kaihui, WANG Xiaoming, LI Dapeng, XU Youyun. Attentional Interaction-based Deep Learning Model for Chinese Question Answering [J]. Computer Science, 2024, 51(6): 325-330.
[15] ZHANG Ruirong, NIU Baoning, FAN Xing. Multi-attribute Blockchain Decentralization Degree Measurement Model [J]. Computer Science, 2024, 51(5): 382-389.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.