计算机科学 ›› 2020, Vol. 47 ›› Issue (12): 42-49.doi: 10.11896/jsjkx.201200021

所属专题: 复杂系统的软件工程和需求工程

• 复杂系统的软件工程和需求工程* • 上一篇    下一篇

面向机器学习系统的需求建模与决策选择

杨立1, 马佳佳1, 江华禧1, 马肖肖1, 梁赓1, 左春1,2   

  1. 1 中国科学院软件研究所 北京 100190
    2 中科软科技股份有限公司 北京 100190
  • 收稿日期:2020-09-02 修回日期:2020-10-29 出版日期:2020-12-15 发布日期:2020-12-17
  • 通讯作者: 左春(zuochun@sinosoft.com.cn)
  • 作者简介:yangli2017@iscas.ac.cn
  • 基金资助:
    中国科学院战略性先导A类专项(XDA20080200);国家重点研发计划项目(2018YFB1005002)

Requirements Modeling and Decision-making for Machine Learning Systems

YANG Li1, MA Jia-jia1, JIANG Hua-xi1, MA Xiao-xiao1, LIANG Geng1, ZUO Chun1,2   

  1. 1 Institute of Software Chinese Academy of Sciences Beijing 100190,China
    2 Sinosoft.CoLtd Beijing 100190,China
  • Received:2020-09-02 Revised:2020-10-29 Online:2020-12-15 Published:2020-12-17
  • About author:YANG Li,born in 1978Ph.Dassociate professoris a member of China Computer Federation.His main research interests include intelligent software engineering and blockchain.
    ZUO Chun,born in 1959professorPh.D supervisor.His main research interests include software engineering and so on.
  • Supported by:
    Strategy Priority Research Program of Chinese Academy of Sciences(XDA20080200) and National Key Research and Development Program of China(2018YFB1005002).

摘要: 机器学习支撑的系统应用越来越普遍但是此类系统的需求通常难以表达完整且可能存在一些难以检测的冲突使得这些系统通常无法在生产环境中高效满足用户的综合需求.此外对于在实际场景中使用的机器学习系统用户信任通常取决于包含可解释性、公平性等非功能需求在内的综合需求的满足程度且在不同领域内应用机器学习通常有特定的需求为保证需求描述的质量及实施过程的决策带来了挑战.为解决以上问题文中提出了一个机器学习系统的需求建模和决策选择框架包括一个MLS(Machine LearningSystems)需求概念模型和机器学习管道过程元模型以及对训练数据集、算法等组件的决策选择方法旨在规范实际场景中机器学习系统的需求设计、开发和评估.实例研究表明提出的MLS需求描述和实现方法是可行且有效的.

关键词: 非功能需求, 机器学习系统, 决策选择, 需求建模, 元模型

Abstract: The application of systems supported by machine learning is becoming more and more common.Howeverbecause the requirements of such systems are often difficult to express completely and there may be some conflicts which are hard to detectthese systems usually cannot efficiently meet the comprehensive needs of users in a real application environment.In additionfor Machine Learning Systems (MLS) used in actual scenariosuser trust usually depends on the satisfaction of comprehensive requirements including non-functional requirements such as interpretability and fairnessand application of machine learning in different fields usually has specific needswhich brings challenges to ensure the quality of requirement description and decision-making for implementation process.To solve above-mentioned problemsthis paper presents a machine learning system requirements and decision-making framework which includes a concept MLS requirements model and a Meta-Model of MLS pipeline processas well as decision making method for training datasets and algorithms selection.The purpose is to standardize the designdevelopment and evaluation of requirements for machine learning used in actual scenarios.The case study shows that the proposed MLS requirement description and implementation method is feasible and effective.

Key words: Decision making, Machine learning systems, Meta-model, Non-functional requirements, Requirements modeling

中图分类号: 

  • TP391
[1] ALPAYDIN E.Introduction to machine learning[M].MITPress,2020.
[2] TOMSETT R,BRAINES D,HARBORNE D,et al.Interpretable to whom?A role-based model for analyzing interpretable machine learning systems[J].arXiv:1806.07552,2018.
[3] AGARWAL S.Trade-Offs between Fairness,Interpretability,and Privacy in Machine Learning[D].Waterloo:University of Waterloo,2020.
[4] PERINI A,SUSI A,AVESANI P.A machine learning approach to software requirements prioritization[J].IEEE Transactions on Software Engineering,2012,39(4):445-461.
[5] ARORA C,SABETZADEH M,NEJATI S,et al.An activelearning approach for improving the accuracy of automated domain model extraction[J].ACM Transactions on Software Engineering and Methodology (TOSEM),2019,28(1):1-34.
[6] PUDLITZ F,BROKHAUSEN F,VOGELSANG A.Extractionof system states from natural language requirements[C]//2019 IEEE 27th International Requirements Engineering Conference (RE).IEEE,2019:211-222.
[7] WINKLER J,VOGELSANG A.Automatic classification of requirements based on convolutional neural networks[C]//2016 IEEE 24th International Requirements Engineering Conference Workshops (REW).IEEE,2016:39-45.
[8] MAIMON O,ROKACH L.Introduction to knowledge discovery and data mining[M]//Data Mining and Knowledge Discovery Handbook.Springer,Boston,MA,2009:1-15.
[9] CHAPMAN P,CLINTON J,KERBER R,et al.CRISP-DM 1.0 Step-by-step Data Mining Guide[OL].https://www.the-modeling-agency.com/crisp-dm.pdf (accessed on 4 August 2020).
[10] ISHIKAWA F,YOSHIOKA N.How do engineers perceive difficulties in engineering of machine-learning systems?-Questionnaire survey[C]//2019 IEEE/ACM Joint 7th International Workshop on Conducting Empirical Studies in Industry (CESI) and 6th International Workshop on Software Engineering Research and Industrial Practice (SER&IP).IEEE,2019:2-9.
[11] HORKOFF J.Non-functional requirements for machine learn-ing:Challenges and new directions[C]//2019 IEEE 27th International Requirements Engineering Conference (RE).IEEE,2019:386-391.
[12] HIRT R,KOEHL N J,SATZGER G.An end-to-end processmodel for supervised machine learning classification:from problem to deployment in information systems[C]//Designing the Digital Transformation:DESRIST 2017 Research in Progress Proceedings of the 12th International Conference on Design Science Research in Information Systems and Technology .Karlsruhe,Germany.Karlsruher Institut für Technologie (KIT),2017:55-63.
[13] ARDAGNA C A,BELLANDI V,BEZZI M,et al.Model-based big dataanalytics-as-a-service:take big data to the next level[J/OL].IEEE Transactions on Services Computing.https://ieeexplore.ieee.org/abstract/document/8319508.
[14] ZHANG X,LI T,WANG X,et al.Formal analysis to non-functional requirements of trustworthy software[J].Ruan Jian Xue Bao,2015,26(10):2545-2566.
[15] JIA Y D,LIU L.Recognition and classification of non-functional requirements in Chinese[J].Ruan Jian Xue Bao,2019,30(10):3115-3126.
[16] YANG L,LI M,REN J,et al.A Human-in-the-Loop Method for Developing Machine Learning Applications[C]//2019 6th International Conference on Systems and Informatics (ICSAI).IEEE,2019:492-498.
[17] YANG L,ZUO C,WANG Y G.Research and Implementation of Service Oriented Architecture for Knowledge Discovery[J].Chinese Journal of Computers,2005,28(4):445-457.
[18] ZHANG J M,HARMAN M,MA L,et al.Machine learning testing:Survey,landscapes and horizons[J/OL].IEEE Transactions on Software Engineering.https://ieeexplore.ieee.org/abstract/document/9000651.
[19] MAAS A,DALY R E,PHAM P T,et al.Learning word vectors for sentiment analysis[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies.2011:142-150.
[20] TANG D,QIN B,LIU T.Document modeling with gated recurrent neural network for sentiment classification[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:1422-1432.
[21] ZHANG X,ZHAO J,LECUN Y.Character-level convolutional networks for text classification[C]//Advances in Neural Information Processing Systems.2015:649-657.
[22] Sentiment140.For Academics[EB/OL].http://help.sentiment140.com/for-students/?spm=a2c4e.11153940.blogcont576274.32.1b956c9czbnj9g.
[23] BAZIOTIS C,ATHANASIOU N,CHRONOPOULOU A,et al.Ntua-slp at semeval-2018 task 1:Predicting affective content in tweets with deep attentive rnns and transfer learning[J].arXiv:1804.06658,2018.
[24] MPQA.MPQA Resources[EB/OL].http://mpqa.cs.pitt.edu/.
[25] 19Joey85.Sentiment-annotated-news-corpus-and-sentiment-lexicon-in-Slovene[EB/OL].https://github.com/19Joey85/Sentiment-annotated-news-corpus-and-sentiment-lexicon-in-Slovene.
[1] 张子良, 庄毅, 叶彤.
基于元模型的协同建模模型组装与更新方法
Cooperative Modeling Model Combination and Update Method Based on Meta-model
计算机科学, 2021, 48(12): 67-74. https://doi.org/10.11896/jsjkx.201100024
[2] 王铁鑫, 李文心, 曹静雯, 杨志斌, 黄志球, 王飞.
知识驱动的企业间协作参与者动态推荐方法
Knowledge-driven Method Towards Dynamic Partners Recommendation in Inter-enterprise Collaboration
计算机科学, 2020, 47(6): 210-218. https://doi.org/10.11896/jsjkx.190700194
[3] 钱晔,李彤,郁涌,孙吉红,于倩,彭琳.
一种面向同步交互的软件演化过程建模方法
Approach to Modeling Software Evolution Process for Synchronous Interaction
计算机科学, 2016, 43(8): 154-158. https://doi.org/10.11896/j.issn.1002-137X.2016.08.032
[4] 郭鹏,李亚晖,孙磊,蔡晓乐.
面向嵌入式软件开发的UML到Simulink模型转换方法
UML Model to Simulink Model Transformation Method in Design of Embedded Software
计算机科学, 2016, 43(2): 192-198. https://doi.org/10.11896/j.issn.1002-137X.2016.02.042
[5] 马 丽,毋国庆,黄 勃,程 铭,崔梦天.
BDL模型到UML状态图的可视化方法研究
Visualization Method of BDL Model to UML State Diagram
计算机科学, 2015, 42(7): 38-43. https://doi.org/10.11896/j.issn.1002-137X.2015.07.009
[6] 刘欢欢,麻志毅,陈泓婕.
基于PaaS的云应用软件部署环境的元模型
Meta-model of PaaS-based Cloud Application’s Deployment Environment
计算机科学, 2015, 42(10): 45-49.
[7] 朱锐,李彤,莫启,张璇,王一荃,林雷蕾,代飞.
基于EPMM的软件过程行为偏离诊断研究
Research on Deviation Diagnostic of Software Process Behavior Based on EPMM Modelling
计算机科学, 2014, 41(11): 56-62. https://doi.org/10.11896/j.issn.1002-137X.2014.11.012
[8] 曾一,黄兴砚,李函逾,王翠钦.
基于MDA的需求捕获工具的设计与实现
Design and Implementation of Requirements Capture Tools Based on MDA
计算机科学, 2014, 41(10): 204-209. https://doi.org/10.11896/j.issn.1002-137X.2014.10.044
[9] 代飞,李彤,谢仲文,莫启,金运志.
基于EPMM的软件过程结构合理性研究
Research on Structure Soundness of Software Processes Based on EPMM
计算机科学, 2013, 40(8): 186-190.
[10] 刘攀峰,邱听夕,陈祥献,黄 海,段会龙.
基于UML的列车自动防护系统需求建模研究
UMI. Based Software Requirement Modeling for Automatic Train Protection
计算机科学, 2012, 39(Z6): 478-481.
[11] 栾文静,杜玉越.
一种基于服务簇网元模型的Web服务发现方法
Web Service Discovery Method Based on Net Unit Model of Service Cluster
计算机科学, 2012, 39(8): 147-152.
[12] 金瑛浩,孙立镌.
语义特征建模系统中布尔算法的研究
Research on Boolean Operation in Semantic Feature Modeling System
计算机科学, 2012, 39(7): 276-279.
[13] 孙为军,李师贤,严玉清.
模型驱动环境下模型演化的形式化研究
Study on the Formalization of Model Evolution with Model Driven Architecture
计算机科学, 2012, 39(6): 116-119.
[14] 刘彬,米东,杜晓明,高鲁.
基于OMCM的仿真系统概念建模方法
Conceptual Modeling Method of Simulation System Based on
计算机科学, 2012, 39(5): 137-140.
[15] 金瑛浩,孙立镌.
协同语义特征建模技术研究
Research on Collaborative Semantic Feature Modeling System
计算机科学, 2012, 39(2): 280-282.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!