计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (11A): 240300063-5.doi: 10.11896/jsjkx.240300063

• 交叉&应用 • 上一篇    下一篇

基于模仿学习的机器人演奏示教系统

赵雨飞1, 靳聪2, 刘潇雨3, 王洁2, 朱永贵1, 李波4   

  1. 1 中国传媒大学数据科学与智能媒体学院 北京 100000
    2 中国传媒大学信息与通信工程学院 北京 100000
    3 中国爱乐乐团 北京 100000
    4 西北工业大学电子信息学院 西安 710000
  • 出版日期:2024-11-16 发布日期:2024-11-13
  • 通讯作者: 靳聪(jincong0623@cuc.edu.cn)
  • 作者简介:(Zhaoyf@cuc.edu.cn)

Robot Performance Teaching Demonstration System Based on Imitation Learning

ZHAO Yufei1, JIN Cong2, LIU Xiaoyu3, WANG Jie2, ZHU Yonggui1, LI Bo4   

  1. 1 School of Data Science and Media Intelligence,Communication University of China,Beijing 100000,China
    2 School of Information and Communication Engineering,Communication University of China,Beijing 100000,China
    3 China Philharmonic Orchestra,Beijing 100000,China
    4 School of Electronics and Information,Northwestern Polytechnical University,Xi'an 710000,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:ZHAO Yufei,born in 2000,postgra-duate.Her main research interests include computer vision and deep lear-ning.
    JIN Cong,born in 1986,Ph.D,is a member of CCF(No.C4376M).Her main research interests include reinforcement learning,music AI and audio digital twins.

PDF (PC)

167

摘要： 近年来,模仿学习被广泛应用于机器人领域,并展示出巨大的潜力。同时关注到智能系统在教育领域的应用越来越多样化,将机器人合理地应用到教学中可以提升教学效果,如果机器人可以教授一些专业技巧,如演奏乐器,可以为学生和人类老师都提供很大的便利。模仿学习特别适用于高度专业和技术性强的小提琴演奏,但在将专家演示引入动态运动原语(Dynamic Movement Primitive,DMP)的过程中,模糊性问题尤为突出,例如换弦角度的不确定性。传统的换弦角度测量方法如物理测量会有很大的误差且无法泛化,为了解决这一问题,提出了一种名为基于模糊和PCA的动态运动原语(Fuzzy Dynamic Movement Primitive for Teaching,T-FDMP)的新模型。该模型基于二型模糊模型和主成分分析(Principal Component Analysis,PCA)进行构建,使用主成分分析法(PCA)得到的特征变量(运弓角度)作为隶属度函数(琴弦角度)的输入进行学习,同时构建了一个专业级的音乐演奏行为数据库。仿生实验结果证明,所提出的T-FDMP模型能够以高精度控制机器人进行小提琴演奏,还为模仿学习在其他高度专业和技术性强的领域的应用提供了新的研究方向。

关键词: 模仿学习, 机器人控制, 二型模糊模型, 智慧教育, 动态运动原语

Abstract: In recent years,imitation learning has been widely applied in the field of robotics,demonstrating significant potential.At the same time,the application of intelligent systems in the field of education is becoming more and more diversified,and the reasonable application of robots in teaching can improve the teaching effect.If robots can instruct certain professional skills,such as playing musical instruments,it could offer significant convenience for both students and human teachers.Imitation learning is particularly suitable for highly specialized and technically demanding tasks,such as violin performance.However,the introduction of expert demonstrations into the process of dynamic movement primitives(DMP),especially regarding the ambiguity issues like uncertainties in string-changing angles,poses a prominent challenge.Traditional methods of measuring string-changing angles,such as physical measurements,exhibit substantial errors and lack generalization.To address this issue,a new model named fuzzy dynamic movement primitive for teaching(T-FDMP) is proposed.The model is constructed based on Type-2 fuzzy model and principal component analysis(PCA).It utilizes the features obtained from principal component analysis(PCA),specifically the bowing angle,as input for the membership functions(string angles) and simultaneously builds a professional-level music perfor-mance behavior database.Bionic experimental results demonstrate that our T-FDMP model can precisely control the robot for violin performance.Furthermore,it opens up new research directions for imitation learning in other highly specialized and technical domains.

Key words: Imitation learning, Robot control, Type-2 fuzzy model, Smart education, Dynamic movement primitives

中图分类号: 

  • TP242
[1]HUA J,ZENG L C,LI G F,et al.Learning for a robot:Deep reinforcement learning,imitation learning,transfer learning[J].Sensors,2021,21(4):1278.
[2]JAIN A,SHARMA A,PARHI D R.A survey on imitationlearning techniques for end-to-end autonomous vehicles[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(9):14128-14147.
[3]GHASEMIPOUR K S,ZEMEL R,GU S.A Divergence Minimi-zation Perspective on Imitation Learning Methods[J].arXiv:1911.02256,2019.
[4]HU A,CORRADO G,GRIFFITHS N,et al.Model-based imitation learning for urban driving[J].Advances in Neural Information Processing Systems,2022,35:20703-20716.
[5]RAJARAMAN N,YANG L,JIAO J,et al.Toward the fundamental limits of imitation learning[J].Advances in Neural Information Processing Systems,2020,33:2914-2924.
[6]RASHIDINEJAD P,ZHU B,MA C,et al.Bridging offline reinforcement learning and imitation learning:A tale of pessimism[J].Advances in Neural Information Processing Systems,2021,34:11702-11716.
[7]ORSINI M,RAICHUK A,HUSSENOT L,et al.What matters for adversarial imitation learning?[J].Advances in Neural Information Processing Systems,2021,34:14656-14668.
[8]CELEMIN C,PÉREZ-DATTARI R,CHISARI E,et al.Interactive imitation learning in robotics:A survey[J].Foundations and Trends© in Robotics,2022,10(1/2):1-197.
[9]BEHRENS G A,GREEN S B.The ability to identify emotional content of solo improvisations performed vocally and on three different instruments[J].Psychology of Music,1993,21(1):20-33.
[10]WOODY R H.Emotion,imagery and metaphor in the acquisi-tion of musical performance skill[J].Music Education Research,2002,4(2):213-224.
[11]WALTHAM C.The science of string instruments[M].NewYork:Springer,2010.
[12]YOUNG D.The hyperbow controller:Real-time dynamics measurement of violin performance[C]//Proceedings of the 2002 Conference on New Interfaces for Musical Expression.2002:1-6.
[13]SCHOONDERWALDT E,DEMOUCRON M.Extraction ofbowing parameters from violin performance combining motion capture and sensors[J].The Journal of the Acoustical Society of America,2009,126(5):2695-2708.
[14]BEVILACQUA F,RASAMIMANANA N H,FLÉTY E,et al.The augmented violin project:research,composition and performance report[C]//6th International Conference on New Interfaces for Musical Expression(NIME 6).2006:402-406.
[15]CAMPO A,MICHAŁKO A,VAN KERREBROECK B,et al.The assessment of presence and performance in an AR environment for motor imitation learning:A case-study on violinists[J].Computers in Human Behavior,2023,146:107810.
[16]ALAM A.Employing adaptive learning and intelligent tutoring robots for virtual classrooms and smart campuses:reforming education in the age of artificial intelligence[M]//Advanced Computing and Intelligent Technologies:Proceedings of ICACIT 2022.Singapore:Springer Nature Singapore,2022:395-406.
[17]AL HAKIM V G,YANG S H,LIYANAWATTA M,et al.Robots in situated learning classrooms with immediate feedback mechanisms to improve students' learning performance[J].Computers & Education,2022,182:104483.
[18]VALAGKOUTI I A,TROUSSAS C,KROUSKA A,et al.Emotion recognition in human-robot interaction using the NAO robot[J].Computers,2022,11(5):72.
[19]ZHANG Y,ZHANG C,CHENG L,et al.The use of deep lear-ning-based gesture interactive robot in the treatment of autistic children under music perception education[J].Frontiers in Psychology,2022,13:762701.
[20]ZHU J,GIENGER M,KOBER J.Learning task-parameterized skills from few demonstrations[J].IEEE Robotics and Automation Letters,2022,7(2):4063-4070.
[21]IJSPEERT A J,NAKANISHI J,HOFFMANN H,et al.Dynamical movement primitives:learning attractor models for motor behaviors[J].Neural Computer,2013,25(2):328-373.
[22]ZHANG C Z.Study on Dynamic Vibration and Acoustic Behaviorof Violin[D].Guangzhou:South China University of Technology,2014.
[23]TAKAGI T,SUGENO M.Fuzzy identification of systems and its applications to modeling and control[J].IEEE Trans.Syst.Man Cybern,1985(1):116-132.
[24]SUN D,LIAO Q,LOUTFI A.Type-2 fuzzy model-based movement primitives for imitation learning[J].IEEE Transactions on Robotics,2022,38(4):2462-2480.
[25]SHLENS J.A tutorial on principal component analysis[J].ar-Xiv:1404.1100,2014.
[26]GUO D L.Theory and Application of Fuzzy Systems [M].Beijing:Science Press,2021.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发