Computer Science ›› 2023, Vol. 50 ›› Issue (12): 192-202.doi: 10.11896/jsjkx.221000188

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Following Method of Mobile Robot Based on Fusion of Stereo Camera and UWB

FU Yong, WU Wei, WAN Zeqing   

  1. School of Internet of Things Engineering,Jiangnan University,Wuxi,Jiangsu 214122,China
  • Received:2022-10-23 Revised:2023-03-19 Online:2023-12-15 Published:2023-12-07
  • About author:FU Yong,born in 1996,postgraduate.His main research interests include robot systems and path planning.
    WU Wei,born in 1985,associate professor.His main research interests include distributed parameter systems,computational intelligence and robot systems.
  • Supported by:
    Natural Science Foundation of Jiangsu Province(BK20201340).

Abstract: This paper studies the autonomous following random robots in a human-machine blending environment.Especially,a stable and effective method is presented for the robot to determine the desired following target and the recognition after the target is lost,that is,to achieve the visual tracking and positioning of pedestrians based on the image of stereo camera and point cloud data.Then,the location information of UWB is introduced to determine the target pedestrian,and a filter algorithm is used to fuse the sensor data to get the coordinate information under the camera coordinate system.Finally,the coordinate transformation is used to convert the location under the robot coordinate system.An improved dynamic window algorithm(MDWA) is also proposed to improve the following tasks performed by the robot.In addition,based on sensor data,a behaviour decision module including following behaviour,recovery behaviour and transition behaviour is proposed.Through the switching between behaviours,the robot can also retrieve the target when it is lost due to the turning of the target or the change of ambient lighting conditions which make the camera invalid.Experimental results show that the proposed following system can automatically determine the desired following target at starting up,and the robot can achieve good obstacle avoidance following in the scene with static obstacles or in the dynamic scene with other non-target pedestrian disturbances in the view.In particular,the robot can independently retrieve the following target in a turning scene or in a scene with varying lighting conditions,and the success rate of the robot in a turning scene is 81%.

Key words: Human-robot integration, Human following, Dynamic window, Navigation, Obstacle avoidance

CLC Number: 

  • TP242
[1]YAO H C,PENG J W,DAI H D,et al.A Compliant human following method for mobile robot based on an improved spring model[J].Robot,2021,43(6):684-693.
[2]SUN Y,LIU J T.RGB-D sensor based human comfortable following behavior for service robots in indoor environments[J].Robot,2019,41(6):823-833.
[3]GROSS H M,DEBES K,EINHORN E,et al.Mobile robotic rehabilitation assistnt for walking and orientation training of stroke patients:A report on working progress[C]//IEEE International Conference on Systems,Man,and Cybernetics.IEEE,2014:1880-1887.
[4]CHUNG W,KIM H,YOO Y,et al.The detection and following of human legs through inductive approaches for a mobile robot with a single laser range finder[J].IEEE Transactions on Industrial Electronics,2011,59(8):3156-3166.
[5]KOIDE K,MIURA J,MENEGATTI E.A portable three-di-mensional LiDAR-based system for long-term and wide-area people behavior measurement[J].International Journal of Advanced Robotic Systems,2019,16(2):1729881419841532-1-1729881419841532-16.
[6]SONG C,ZHAO J J,WANG K,et al.A survey of few shot learning based on intelligent perception[J].Acta Aeronauticaet Astronautica Sinica,2020,41(S1):723756.
[7]HONIG S S,ORON-GILAD T,ZAICHYK H,et al.Toward socially aware person-following robots[J].IEEE Transactions on Congitive and Developmental Systems,2018,10(4):936-954.
[8]BAO X C,RAGHAVENDER S,JOHN K T.Integrating stereo vision with a CNN tracker for a person-following robot[C]//International Conference on Computer Vision Systems.Shenzhen:2017:300-313.
[9]BAO X C,RAGHAVENDER S,JOHN K T.Person followingrobot using selected online ada-boosting with stereo camera[C]//2017 14th Conference on Computer and Robot Vision(CRV).Edmonton,AB:IEEE,2017:48-55.
[10]AYEY Y,THIHA K,MYINT PYU M M,et al.A Deep Neural Network Based Human Following Robot with Fuzzy Control[C]//2019 IEEE International Conference on Robotics and Biomimetics(ROBIO).Dali:IEEE,2019:720-725.
[11]DOISY G,JEVTIC A,LUCET E,et al.Adaptive person-following algorithm based on depth images and mapping[C]//Proceedings of the IROS Workshop on Robot Motion Planning.Vi-lamoura,Algarve,Portugal:IEEE,2012:7-12.
[12]NIKDEL P,SHRESTHA R,VAUGHAN R.The hands-freepush-cart:Autonomous following in front by predicting user trajectory around obstacles[C]//IEEE International Conference on Robotics and Automation.Brisbane.QLD:IEEE,2018:4548-4554.
[13]TASAKI R,SAKURAI H,TERASHIMA K.Moving target localization method using foot mounted acceleration sensor for autonomous following robot[C]//IEEE Conference on Control Technology and Applications.Maui,HI:IEEE,2017:827-833.
[14]PANG L,CAO Z Q,YU J Z.A pedestrian-aware collision-free following approach for mobile robots based on A* and TEB[J].Acta Aeronauticaet Astronautica Sinica,2021,42(4):524909.
[15]YOON Y,YOON H,KIM J.Depth assisted person following robots[C]//IEEE RO-MAN.Gyeongju:IEEE,2013:330-331.
[16]GU C.The research and design of an autonomous following robot[D].Nanjing:Southeast University,2016.
[17]YANG C,SONG K.Control Design for Robotic Human-Following and Obstacle Avoidance Using an RGB-D Camera[C]//2019 19th International Conference on Control,Automation and Systems(ICCAS).JEJU:IEEE,2019:934-939.
[18]JUNG E J,LEE J H,YI B J,et al.Development of a laser-range-finder-based human tracking and control algorithm for a marathoner service robot[J].IEEE/ASME transactions on mechatronics,2014,19(6):1963-1976.
[19]ZHANG Y,WANG C,WANG X,et al.Fairmot:On the fairness of detection and re-identification in multiple object tracking[J].International Journal of Computer Vision,2021,129(11):3069-3087.
[20]CEN M F,HUANG Y L,ZHONG X Y,et al.Real-time obstacle avoidance and person following based on adaptive window approach[C]//2019 IEEE International Conference on Mechatronics and Automation(ICMA).Tianjin:IEEE,2019:64-69.
[21]XU Y W,YAN W X,WU W.Improvement of LiDAR SLAM Front-end Algorithm Based on Local Mapping Similar Scenes[J].Robot,2022,44(2):176-185.
[22]CHIEN V D,HEUNGJU A,JONG W K,et al.Collision-Free Navigation in Human-Following Task Using a Cognitive Robo-tic System on Differential Drive Vehicles[J].IEEE 25 Transactions on Cognitive and Developmental Systems,2023,15(1):78-87.
[1] XING Linquan, XIAO Yingmin, YANG Zhibin, WEI Zhengmin, ZHOU Yong, GAO Saijun. Spacecraft Rendezvous Guidance Method Based on Safe Reinforcement Learning [J]. Computer Science, 2023, 50(8): 271-279.
[2] LIU Jiawei, DU Xin, FAN Fangzhao, XIE Chengbi. Design of Indoor Mapping and Navigation System Based on Multi-sensor [J]. Computer Science, 2023, 50(6A): 220300218-8.
[3] HUANG Yuzhou, WANG Lisong, QIN Xiaolin. Bi-level Path Planning Method for Unmanned Vehicle Based on Deep Reinforcement Learning [J]. Computer Science, 2023, 50(1): 194-204.
[4] CHEN Bo-chen, TANG Wen-bing, HUANG Hong-yun, DING Zuo-hua. Pop-up Obstacles Avoidance for UAV Formation Based on Improved Artificial Potential Field [J]. Computer Science, 2022, 49(6A): 686-693.
[5] YANG Han, WAN You, CAI Jie-xuan, FANG Ming-yu, WU Zhuo-chao, JIN Yang, QIAN Wei-xing. Pedestrian Navigation Method Based on Virtual Inertial Measurement Unit Assisted by GaitClassification [J]. Computer Science, 2022, 49(6A): 759-763.
[6] LYU Run, LI Guan-yu, QI Pei, QIAN Wei-xing, WANG Lan-ze, FENG Tai-ping. New SLAM Method of Multi-layer Lidar Assisted by Rotational Strapdown Inertial NavigationSystem [J]. Computer Science, 2022, 49(11A): 211200088-5.
[7] ZHU Chen-shuang, CHENG Shi-wei. Prediction and Assistance of Navigation Demand Based on Eye Tracking in Virtual Reality Environment [J]. Computer Science, 2021, 48(8): 315-321.
[8] DU Wan-ru, WANG Xiao-yin, TIAN Tao, ZHANG Yue. Artificial Potential Field Path Planning Algorithm for Unknown Environment and Dynamic Obstacles [J]. Computer Science, 2021, 48(2): 250-256.
[9] ZHOU Jun and WANG Tian-qi. Single Departure and Arrival Procedure Optimization in Airport Terminal Area Based on Branch and Bound Method [J]. Computer Science, 2020, 47(6A): 552-555.
[10] CHEN Yu-tao, XU Wen-chao, ZHAO Zhao-na, LIU Hong-en, WANG Hao. Optimization of Scheduling and Maintenance Strategy for Navigation Aircraft Operation [J]. Computer Science, 2020, 47(11A): 632-637.
[11] WANG Wei-guang, YIN Jian, QIAN Xiang-li, ZHOU Zi-hang. Realtime Multi-obstacle Avoidance Algorithm Based on Dynamic System [J]. Computer Science, 2020, 47(11A): 111-115.
[12] ZHAI Shu-ying, LI Ru, LI Bo, HAO Shao-yang. Survey on Applications of Visual Crowdsensing [J]. Computer Science, 2019, 46(6A): 11-15.
[13] CHENG Hao-hao, YANG Sen, QI Xiao-hui. Online Obstacle Avoidance and Path Planning of Quadrotor Oriented to Urban Environment [J]. Computer Science, 2019, 46(4): 241-246.
[14] CHEN Jin-yin, HU Ke-ke,LI Yu-wei. Research on UAV Multi-point Navigation Algorithm Based on MB-RRT* [J]. Computer Science, 2018, 45(6A): 85-90.
[15] CHENG Jing, ZHANG Tao, WANG Tao, DONG Zhan-wei. Graphic Complexity-based Prioritizing Technique for Regression Testing of Mobile Navigation Service [J]. Computer Science, 2018, 45(6): 141-144.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!