仿真模型可信度评估研究综述及难点分析

摘要/Abstract

摘要： 近年来,随着计算机和建模与仿真技术的不断发展,仿真模型在军事、社会、经济等领域得到了广泛应用。同时,仿真模型的功能和性能大大增强,仿真系统越来越复杂,仿真模型的可信度评估面临着新的挑战,已经成为建模与仿真领域的关键难题。鉴于此,文中对国内外仿真模型可信度研究的主要工作进行了系统的回顾与总结,特别地分析了近年来的最新研究成果。首先,明确了仿真模型可信度的概念及其与校核、验证与确认(Verification,Validation and Accreditation,VV&A)之间的关系;然后,总结了仿真模型可信度研究的发展历程、范式和生命周期模型,对仿真模型可信度评估的方法与技术进行了总结与分类;最后,提出了仿真模型可信度评估面临的8项挑战。通过综述与难点分析,文章为制定仿真模型可信度评估的研究框架、创新可信度评估理论和方法提供了参考。

关键词: VV&A, 仿真模型, 可信度评估, 难点分析, 研究综述

Abstract: Along with the developing of computing and modeling and simulation techniques,simulation models have been applied in military,social and economic areas widely in these years.Meanwhile,the capabilities of simulation mo-dels become more and more strong,and simulation systems become more and more complex.Thus,the credibility assessment of simulation models is confronted with many challenges and becomes the crucial difficult problem nowadays.This paper reviewed principal works,especially current researches on credibility assessment of simulation models.Firstly,the concept of credibility and its relationship with Verification,Validation and Accreditation(VV&A) were introduced.Secondly,the research history,paradigm and life cycle of credibility assessment of simulation models were summarized,and important methods and techniques were analyzed and classified.Finally,eight challenges of credibility assessment of simulation models were presented.The overview and difficulties analysis provide significant references for constructing the framework and innovating theories and methods for the credibility assessment of simulation models.

Key words: Credibility assessment, Difficulties analysis, Overview, Simulation models, VV&A

中图分类号:

TP391.9

杨小军, 徐忠富, 张星, 孙丹辉. 仿真模型可信度评估研究综述及难点分析[J]. 计算机科学, 2019, 46(6A): 23-29. https://doi.org/

YANG Xiao-jun, XU Zhong-fu, ZHANG Xing, SUN Dan-hui. Overview and Difficulties Analysis on Credibility Assessment of Simulation Models[J]. Computer Science, 2019, 46(6A): 23-29. https://doi.org/

参考文献

[1]黄柯棣,查亚兵.系统仿真可信性研究综述[J].系统仿真学报,1997,9(1):4-9.
[2]王子才,张冰,杨明.仿真系统的校核、验证与验收(VV&A):现状与未来[J].系统仿真学报,1999,11(5):321-325,340.
[3]李鹏波,张金槐.仿真可信性的研究综述[J].计算机仿真,2000,17(4):12-14.
[4]张伟,王行仁.仿真可信度[J].系统仿真学报,2001,13(3):312-314.
[5]刘庆鸿,陈德源,王子才.建模与仿真校核、验证与确认综述[J].系统仿真学报,2003,15(7):925-930.
[6]唐见兵,查亚兵,李革.仿真VV＆A研究综述[J].计算机仿真,2006,23(11):82-85,98.
[7]贾仁耀,刘湘伟.建模与仿真的校核与验证技术综述[J].计算机仿真,2007,24(4):49-52.
[8]伊文英,李波.基于Agent仿真模型的校核与验证研究[J].计算机科学,2015,42(Z11):460-463,472.
[9]李伟,林圣琳,周玉臣,等.复杂仿真系统可信度评估研究进展[J].中国科学:信息科学,2018,48:767-782.
[10]左彩霞.复杂全数字仿真系统的可信性评估方法的研究与应用[D].成都:电子科技大学,2010.
[11]顾闯,刘彬,张星,等.装备保障仿真概念模型的语义验证方法研究[J].计算机科学,2014,41(9):24-27,37.
[12]唐见兵,査亚兵.作战仿真系统校核、验证与确认及可信度评估[M].北京:国防工业出版社,2013.
[13]CONWAY R W,JOHNSON B M,MAXWELL W L.Some Problems of Digital Systems Simulation [J].Management Scie-nce,1959,6(1):92-110.
[14]BIGGS A G,CAWTHORNE A R.Bloodhood Missile Evaluation [J].Journal of the Royal Aeronautical Society,1962,66(621):571-587.
[15]NAYLOR THOMAS H,FINGER J M.Verification of Compu-ter Simulation Models [J].Management Science,1967,14(2):B92-B101.
[16]MCKENNY J L.Critique of:Verification of Computer Simulation Models[J].Management Science,1967,14(2):B-102-B-103.
[17]SCHRANK W E,HOLT CC.Critique of:Verification of Computer Simulation Models[J].Management Science,1967,14(2):B-104-B-106.
[18]FISHMAN G S,KIVIAT P J.The Statisties of Diserete Event Simulation[J].Simulation,1968,10:185-195.
[19]VAN HORN R L.Validation of Simulation Results [J].Management Science,1971,17(5):247-258.
[20]SHANNON R E.Systems Simulation:The Art and Science [M].Prentice-Hall,1975.
[21]KHEIR N A,HOLMES W M.On Validating Simulation Mo-dels of Missile Systems [J].Simulation,1978,30(4):117-128.
[22]周威,张国忠.建模与仿真VV&A基本问题研究——VV&A的系统观[M].北京:机械工业出版社,2017.
[23]SARGENT R G.Verification and Validation of Simulation Mo-dels [J].Journal of Simulation,2013,7:12-24.
[24]BALCI O.Principles of Simulation Model Validation,Verification and Testing [J].Transaction of the Society for Computer Simulation International,1997,14(1):3-12.
[25]BALCI O.Golden Rules of Verification,Validation,Testing,and Certification of Modeling and Simulation Applications [J].SCS M&S Magazine,2010,1-7.
[26]BALCI O.Verification,Validation,and Certification of Modeling and Simulation Applications [C]∥Proceedings of the 2003 Winter Simulation Conference,2003,150-158.
[27]BALCI O.A Life Cycle for Modeling and Simulation [J].Simulation,2012,88(7):870-883.
[28]SARGENT R G,GOLDSMAN D M,YAACOUB T.A Tutorial on the Operational Validation of Simulation Models [C]∥Proceedings of the 2016 Winter Simulation Conference,2016,163-177.
[29]BALCI O.Verification,Validation and Accreditation of Simulation Models [C]∥Proceedings of the 1997 Winter Simulation Conference.1997:135-141.
[30]AHN J,DE WECK O L,STEELE M.Credibility Assessment of Models and Simulations Based on NASA’s Models and Simulation Standard Using the Delphi Method [J].Systems Enginee-ring,2014,17(2):237-248.
[31] JEBEILE J,BARBEROUSSE A.Empirical Agreement in Model Validation [J].Studies in History and Philosophy of Science,2016,56:168-174.
[32]DAMBORG M J.An Example of Error Analysis in Dynamic Model Validation [J].Simulation,1985,44(6):301-305.
[33]BARLAS Y.An Autocorrelation Function Test for Output Validation [J].Simulation,1990,55(1):7-16.
[34]刘慧博,钱永杰.基于粗糙集和灰色关联分析的仿真可信度评估[J].系统仿真学报,2018,30(2):459-464.
[35]宁小磊,吴颖霞,于天朋,等.基于改进灰色关联分析的仿真模型综合验证方法[J].兵工学报,2016,37(2):338-347.
[36]李玉宁.基于灰色关联分析的仿真模型验证及辅助工具研究[D].哈尔滨:哈尔滨工业大学,2015.
[37]LEGATES D R,MCCABE J G J.Evaluating the Use of Goodness-of-fit Measures in Hydrologic and Hydroclimatic Model Validation [J].Water Resources Research,1999,35(1):233-241.
[38]FISHMAN G S,KIVIAT P J.The Analysis of Simulation-Generated Time Series [J].Management Science,1967,13(7):525-557.
[39]MONTGOMERY D C,CONARD R G.Comparison of Simulation and Flight-test Data for Missile Systems [J].Simulation,1980,34(2):63-72.
[40]WANG X Y,MUEEN A,DING H,et al.Experimental Compa-rison of Representation Methods and Distance Measures for Time Series Data [J].Data Min Knowl Disc,2013,26:275-309.
[41]王斐,梁晓庚,王艳奎,等.基于曲率距离的制导系统动态仿真可信度评估[J].系统仿真学报,2013,25(8):1896-1900.
[42]李鹏波,谢红卫.频谱分析方法在仿真可信性研究中的应用[J].系统仿真学报,1998,10(3):18-21,54.
[43]JIANG X M,MAHADEVAN S.Wavelet Spectrum Analysis Approach to Model Validation of Dynamic Systems [J].Mechanical Systems and Signal Processing,2011,25:575-590.
[44]张鸿喜,张洁.小波变换在舷外有源诱饵仿真模型验证中的应用[J].系统仿真学报,2018,30(1):318-324.
[45]HAUDUC H,NEUMANN M B,MUSCHALLA D,et al.Efficiency Criteria for Environmental Model Quality Assessment:A Review and Its Application to Wastewater Treatment [J].Environmental Modelling & Software,2015,68:196-204.
[46]CROCHEMORE L,PERRIN C,ANDRÉASSIAN V,et al. Comparing Expert Judgement and Numerical Criteria for Hydrograph Evaluation [J].Hydrological Sciences Journal,2015,60(3):402-423.
[47]FACHADA N,LOPES V V,MARTINS R C,et al.Model-independent Comparison of Simulation Output [J].Simulation Mo-delling Practice and Theory,2017,72:131-149.
[48]LI L Y,LU Z Z.A New Method for Model Validation with Multivariate Output [J].Reliability Engineering and System Safety,2018,169:579-592.
[49]ZHAO L F,LU Z Z,YUN W Y,et al.Validation Metric Based on Mahalanobis Distance for Models with Multiple Correlated Responses [J].Reliability Engineering and System Safety,2017,159:80-89.
[50]LIAO W C,ZHANG J,ZHENG X P,et al.A Generalized Validation Procedure for Pedestrian Models [J].Simulation Modelling Practice and Theory,2017,77:20-31.
[51]OLSEN M M,RAUNAK M,SETTEDUCATI M.Enabling Quantified Validation for Model Credibility [C]∥Proceedings of the 50th Computer Simulation Conference.2018:1-10.
[52]KWAG S,GUPTA A,DINH N.Probabilistic Risk Assessment Based Model Validation Method Using Bayesian Network [J].Reliability Engineering and System Safety,2018,169:380-393.
[53]LI Y L,WANG X J,WANG C,et al.Non-probabilistic Bayesian Update Method for Model Validation [J].Applied Mathematical Modelling,2018,58:388-403.
[54]LAMPERTI F.An Information Theoretic Criterion for Empirical Validation of Simulation Models [J].Econometrics and Statistics,2018,5:83-106.
[55]ATKINSON A D,HILL R R,PIGNATIELLO J J J,et al. Wavelet ANOVA Approach to Model Validation [J].Simulation Modelling Practice and Theory,2017,78:18-27.
[56]ATKINSON A D,HILL R R,PIGNATIELLO J J J,et al. Wavelet ANOVA Bisection Method for Identifying Simulation Model Bias [J].Simulation Modelling Practice and Theory,2018,80:66-74.
[57]SONG H F,LIU J Y,SCHNIEDER E.Validation,Verification and Evaluation of a Train to Train Distance Measurement System by Means of Colored Petri Nets [J].Reliability Engineering and System Safety,2017,164:10-23.
[58]SONG H F,SCHNIEDER E.Development and Validation of a Distance Measurement System in Metro Lines [C]∥IEEE Transactions on Intelligent Transportation Systems.2018:1-16.
[59]GUARRO S B,YAU M K,OZGUNER U,et al.Formal Framework and Models for Validation and Verification of Software-Intensive Aerospace Systems [C]∥Proceedings of AIAA SciTech Forum.2017:1-10.
[60]刘荷明,张树平,王小锋,等.层次分析法在仿真系统可信度评估中的应用[J].航天电子对抗,2016,32(1):35-38.
[61]张忠,方可,杨明.基于群组AHP的复杂仿真系统可信度评估方法[J].系统工程与电子技术,2011,33(11):2569-2572.
[62]王雨潇,王松艳,晁涛,等.制导控制系统的多准则变权综合评估方法[J].系统仿真学报,2018,30(2):435-442.
[63]吴静,吴晓燕,高忠长.基于模糊聚类和粗糙集的仿真可信性模糊综合评估[J].系统工程与电子技术,2010,32(4):770-773.
[64]徐迪.基于相似理论的系统仿真可信性分析[J].系统工程理论与实践,2001,4:49-52.
[65]王石,伍丁红,戴金海.基于 Bayes 网的武器装备仿真可信度评估[J].系统仿真学报,2011,23(6):1082-1085,1103.
[66]BALCI O,ADAMS R J,MYERS D S,et al.A collaborative eva-luation environment for credibility assessment of modeling and simulation applications [C]∥Proceedings of the 2002 Winter Simulation Conference.214-220.
[67]查亚兵,唐见兵.基于主观综合评判的作战仿真可信度评估方法研究[J].国防科技大学学报,2010,32(6):153-157.
[68]李莉红.船舶电力推进系统仿真可信度评估研究[D].哈尔滨:哈尔滨工程大学,2012.
[69]张忠.仿真系统可信度评估方法研究[D].哈尔滨:哈尔滨工业大学,2014.
[70]YANG Y N,KUMARASWAMY M M,PAM H J,et al.Inte-grated Qualitative and Quantitative Methodology to Assess Validity and Credibility of Models for Bridge Maintenance Management System Development [J].Journal of Management in Engineering,2011,27(3):149-158.
[71]PADILLA J J,DIALLO S Y,LYNCH C J,et al.Observations on the Practice and Profession of Modeling and Simulation:A Survey Approach [J].Simulation,2018,94(6):493-506.
[72]SARGENT R G,BALCI O.History of Verification and Validation of Simulation Models[C]∥Proceedings of the 2017 Winter Simulation Conference.2017:292-307.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed