计算机科学 ›› 2022, Vol. 49 ›› Issue (8): 225-229.doi: 10.11896/jsjkx.211100046

• 人工智能 • 上一篇    下一篇

一种基于UIA接口的RPA系统设计方法

王岩松, 秦云川, 蔡宇辉, 李肯立   

  1. 湖南大学计算机科学与工程学院 长沙 410082
  • 收稿日期:2021-11-03 修回日期:2022-03-14 发布日期:2022-08-02
  • 通讯作者: 秦云川(qinyunchuan@hnu.edu.cn)
  • 作者简介:(wangys1997@hnu.edu.cn)
  • 基金资助:
    国家重点研发计划(2020YFB2104002)

Design and Implementation of RPA System Based on UIA Interface

WANG Yan-song, QIN Yun-chuan, CAI Yu-hui, LI Ken-li   

  1. School of Computer Science and Engineering,Hunan University,Changsha 410082,China
  • Received:2021-11-03 Revised:2022-03-14 Published:2022-08-02
  • About author:WANG Yan-song,born in 1997,postgraduate.His main research interests include artificial intelligence and big data processing.
    QIN Yun-chuan,born in 1983,Ph.D,is a member of China Computer Federation.His main research interests include autonomous unmanned systems and high-performance embedded computing.
  • Supported by:
    National Key Research and Development Program of China(2020YFB2104002).

摘要: 机器人流程自动化(Robotic Process Automation,RPA)是当今的研究热点之一,RPA主要使用Win32接口对窗体进行自动化操作,该方法需要对API进行封装,开发成本高,且只能识别基于Win32 API设计的UI元素,同时基于Win32接口实现自动化需要模拟键盘和鼠标操作,由于该模拟操作基于一种广播消息机制,导致操作的响应时间较长,因此提出了一种基于微软的 UIA(UI Automation)技术,用于构建 RPA流程自动化应用的方案。该方案采用UIA方法进行UI元素的自动化操作,适应的UI程序框架范围广,包括Win32,WPF,QT,Silverlight等;而且开发成本低。同时,该方案将消息与UI元素绑定,避开了低效的广播消息机制,提高了RPA的执行效率。实验结果显示,与模拟键盘鼠标方法相比,所提方法的执行时间平均可缩短55.67%。

关键词: RPA, UIA技术, 自动化

Abstract: Robotic process automation(RPA)is one of the current research hotspots.RPA mainly uses Win32 interface to automate the operation of windows.This method requires to encapsulate the API,which is expensive to develop and can only identify UI elements designed based on Win32 API.At the same time,automation based on Win32 interface needs to simulate keyboard and mouse operations.Because these operations are based on a broadcast message mechanism,the response time is long.This paper proposes a solution for building RPA process automation application based on Microsoft's UIA technology.The solution uses UIA methods to automate UI elements.It can adapt to a wide range of UI program frameworks,including Win32,WPF,QT,Silverlight,etc.,and the development cost is low.At the same time,this method binds the message to the UI element,avoids the inefficient broadcast message mechanism,and improves the execution efficiency of RPA.Experimental results show that the execution time can be shortened by 55.67% on average compared with the keyboard and mouse method.

Key words: Automation, RPA, UIA technology

中图分类号: 

  • TP312
[1]GRABOWSKI R,SIEGEL B,JOHNSON C,et al.Finance in a Digital World:Technology as a partner,not a threat[J].The Journal of Government Financial Management,2019,67(4):18-23.
[2]MCCANN D.Special report:The new digital workforce[R].CFO.Com.Ireland,2018,51(2):24.
[3]SUTIPITAKWONG S,JAMSRI P.The Effectiveness of RPA in Fine-tuning Tedious Tasks[C]//2020 6th International Confe-rence on Engineering,Applied Sciences and Technology(ICEAST).2020:1-4.
[4]YOKOYAMA H,FUKUI M,MATSUMOTO T,et al.UseGameplay RPA as Programming Materials[C]//2020 Nicograph International(NicoInt).2020:92-92.
[5]CHEN D,XIAO D,LI Y,et al.Automatic approval method for financial reimbursement of electric power enterprises based on RPA robot[C]//2021 International Conference on Computer Technology and Media Convergence Design(CTMCD).2021:231-234.
[6]WEWERKA J,MICUS C,REICHERT M.Seven Guidelines for Designing the User Interface in Robotic Process Automation[C]//2021 IEEE 25th International Enterprise Distributed Object Computing Workshop(EDOCW).Gold Coast,Australia,2021:157-165.
[7]WEWERKA J,DAX S,REICHERT M.A User AcceptanceModel for Robotic Process Automation[C]//2020 IEEE 24th International Enterprise Distributed ObjectComputing Confe-rence(EDOC).Eindhoven,Netherlands,2020:97-106.
[8]BAIDYA A.Document Analysis and Classification:A RoboticProcess Automation(RPA)and Machine Learning Approach[C]//2021 4th International Conference on Information and Computer Technologies(ICICT).HI,USA,2021:33-37.
[9]LE V C,PHAN D H,VU T D.Robotic Process Automation and Opportunities for Vietnamese Market[C]//Proceedings of the 2019 7th International Conference on Computer and Communications Management(ICCCM 2019).2019:86-90.
[10]MAALLA A.Development Prospect and Application Feasibility Analysis of Robotic Process Automation[C]//2019 IEEE 4th Advanced Information Technology,Electronic and Automation Control Conference(IAEAC).2019:2714-2717.
[11]KETKAR Y,GAWADE S.Effectiveness of Robotic ProcessAutomation for data mining using UiPath[C]//2021 International Conference on Artificial Intelligence and Smart Systems(ICAIS).2021:864-867.
[12]ISSAC R,MUNI R,DESAI K.Delineated Analysis of Robotic Process Automation Tools[C]//2018 Second International Conference on Advances in Electronics,Computers and Communications(ICAECC).2018:1-5.
[13]MARTINS P,SÁ F,MORGADO F,et al.Using machine lear-ning for cognitive Robotic Process Automation(RPA)[C]//2020 15th Iberian Conference on Information Systems and Technologies(CISTI).2020:1-6.
[14]YATSKIV N,YATSKIV S,VASYLYK A.Method of Robotic Process Automation in Software Testing Using Artificial Intelligence[C]//2020 10th International Conference on Advanced Computer Information Technologies(ACIT).2020:501-504.
[15]GUPTA S,RANI S,DIXIT A.Recent Trends in Automation-A study of RPA Development Tools[C]//2019 3rd International Conference on Recent Developments in Control,Automation &Power Engineering(RDCAPE).2019:159-163.
[16]YATSKIV S,VOYTYUK I,YATSKIV N,et al.ImprovedMethod of Software Automation Testing Based on the Robotic Process Automation Technology[C]//2019 9th International Conference on Advanced Computer Information Technologies(ACIT).2019:293-296.
[17]FENG K,GAO J H.GUI Event Interaction Graph Generation Method Based on Model[J].Computer Science,2013,40(Z6):184-187,203.
[18]JASON G,MASAHIKO K.Engineering Software for Accessibility[M]//Microsoft Press.2009:1-7.
[19]MORGADOI C,PAIVAAC R.The iMPAcT Tool:Testing UIPatterns on Mobile Applications[C]//2015 30th IEEE/ACM International Conference on Automated Software Engineering(ASE).2015:876-881.
[20]COSTA P,PAIVAAC R,NABUCO M.Pattern Based GUI Testing for Mobile Applications[C]//2014 9th International Conference on the Quality of Information and Communications Technology(QUATIC 2014).2014:66-74.
[21]HUANG S Q,LIU Y B,HUANG X S.Research on Automatic Testing Technology of Model Driven Development Tools[J].Computer Science,2021,48(6A):568-571.
[1] 冷典典, 杜鹏, 陈建廷, 向阳.
面向自动化集装箱码头的AGV行驶时间估计
Automated Container Terminal Oriented Travel Time Estimation of AGV
计算机科学, 2022, 49(9): 208-214. https://doi.org/10.11896/jsjkx.210700028
[2] 高文龙, 周天阳, 朱俊虎, 赵子恒.
基于双向蚁群算法的网络攻击路径发现方法
Network Attack Path Discovery Method Based on Bidirectional Ant Colony Algorithm
计算机科学, 2022, 49(6A): 516-522. https://doi.org/10.11896/jsjkx.210500072
[3] 黄双芹, 刘英博, 黄向生.
模型驱动开发工具的自动化测试技术研究
Research on Automatic Testing Technology of Model Driven Development Tools
计算机科学, 2021, 48(6A): 568-571. https://doi.org/10.11896/jsjkx.201000139
[4] 曹浩, 郭绍忠, 刘聃, 许瑾晨.
面向64位RISC-V的基础数学库自动化移植
Automatic Porting of Basic Mathematics Library for 64-bit RISC-V
计算机科学, 2021, 48(6): 41-47. https://doi.org/10.11896/jsjkx.201200058
[5] 周天阳, 曾子懿, 臧艺超, 王清贤.
基于多Agent联合决策的队组协同攻击规划
Team Cooperative Attack Planning Based on Multi-agent Joint Decision
计算机科学, 2021, 48(5): 301-307. https://doi.org/10.11896/jsjkx.200800174
[6] 刘芳, 洪玫, 王潇, 郭丹, 杨正卉, 黄小丹.
面向Java的Randoop自动化单元测试生成工具性能分析
Performance Analysis of Randoop Automated Unit Test Generation Tool for Java
计算机科学, 2020, 47(9): 24-30. https://doi.org/10.11896/jsjkx.200200116
[7] 罗云芳, 唐承娥, 韦军.
基于粗糙规则的脉冲神经膜系统计算能力的研究
Computing Ability of Spiking Neural P System Based on Rough Rules
计算机科学, 2020, 47(6A): 626-630. https://doi.org/10.11896/JsJkx.190500120
[8] 孟繁祎, 王莹, 于海, 朱志良.
复杂软件系统的重构技术:现状、问题与展望
Refactoring of Complex Software Systems Research:PresentProblem and Prospect
计算机科学, 2020, 47(12): 1-10. https://doi.org/10.11896/jsjkx.200800067
[9] 柴锐, 薛凡, 曾建潮, 秦品乐.
一种医学肾动态显像自动化定量评估方法
Automatic Quantitative Evaluation Approach for Medical Renal Dynamic Imaging
计算机科学, 2019, 46(8): 321-326. https://doi.org/10.11896/j.issn.1002-137X.2019.08.053
[10] 仲美稣, 杨勇生, 周亚民.
基于速度控制的自动化码头AGV无冲突路径规划
Free-conflict AGV Path Planning in Automated Terminals Based on Speed Control
计算机科学, 2019, 46(7): 308-314. https://doi.org/10.11896/j.issn.1002-137X.2019.07.047
[11] 童泽平, 吴应强, 任亮, 李巍.
SP-AS/RS基于新型出入口结构的行程时间分析
Travel Time Analysis of SP-AS/RS with New Configuration for I/O Point
计算机科学, 2019, 46(4): 315-320. https://doi.org/10.11896/j.issn.1002-137X.2019.04.049
[12] 张英杰, 朱雪峰.
模式驱动的软件架构设计研究综述
Review of Pattern Driven Software Architecture Design
计算机科学, 2018, 45(11A): 48-52.
[13] 梁浩然,周宽久,崔凯,潘杰,侯刚.
基于PEFSM行为模型的黑盒测试用例生成方法
Black-box Test Case Generation Method Based on PEFSM Behavioral Model
计算机科学, 2017, 44(4): 234-240. https://doi.org/10.11896/j.issn.1002-137X.2017.04.050
[14] 冯谷,李尼格.
模型驱动的移动应用测试方法
Model-driven Testing for Mobile Applications
计算机科学, 2017, 44(11): 232-239. https://doi.org/10.11896/j.issn.1002-137X.2017.11.035
[15] 马文卓,张杰.
基于立体定位技术的自动化精密加工系统设计
Design Robot System Based on Stereo Perception Technology Computer Engineering and Applications
计算机科学, 2016, 43(Z6): 563-567. https://doi.org/10.11896/j.issn.1002-137X.2016.6A.135
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!