Computer Science ›› 2023, Vol. 50 ›› Issue (10): 230-238.doi: 10.11896/jsjkx.221000046

• Artificial Intelligence • Previous Articles     Next Articles

Hybrid Bayesian Network Structure Learning via Evolutionary Order Search

LI Mingjia, QIAN Hong, ZHOU Aimin   

  1. School of Computer Science and Technology,East China Normal University,Shanghai 200062,China
    Shanghai Institute of AI for Education,East China Normal University,Shanghai 200062,China
  • Received:2022-10-09 Revised:2023-03-12 Online:2023-10-10 Published:2023-10-10
  • About author:LI Mingjia,born in 1998,postgraduate.His main research interests include evolutionary machine learning,causal infe-rence and intelligent education.QIAN Hong,born in 1991,Ph.D,asso-ciate researcher.His main researchin-terests include derivative-free optimization,evolutionary machine learning,and intelligent education.
  • Supported by:
    National Natural Science Foundation of China(62106076),Science and Technology Commission of Shanghai Municipality(19511120601),Natural Science Foundation of Shanghai(21ZR1420300),“Chenguang Program” Sponsored by Shanghai Education Development Foundation and Shanghai Municipal Education Commission(21CGA32) and Fundamental Research Funds for the Central Universities.

Abstract: Bayesian network is an effective tool for uncertainty knowledge representation and reasoning.Learning and discovering its structure is the basis of reasoning via this tool.Existing Bayesian network structure learning algorithms often encounter the dilemma of balancing effectiveness and efficiency in real-world applications such as intelligent education.On the one hand,score-and-search methods can find out the high-quality solutions,but they suffer from the high algorithmic complexity.On the other hand,hybrid methods are highly efficient but the quality of the found solutions is not satisfactory.To address the above dilemma,this paper proposes an evolutionary order search based hybrid Bayesian network structure learning method called EvOS.First,the proposed EvOS constructs an undirected graph skeleton through a constraint algorithm,and then applies an evolutionary algorithm to search for the optimal node order,and finally uses the found node order to guide the greedy search so as to obtain the Bayesian network structure.This paper conducts the empirical study to verify the effectiveness and efficiency of the proposed EvOS in the commonly-used benchmark datasets as well as the real-world task of educational knowledge structure discovery.Experimental results show that,compared with the score-and-search methods,EvOS is able to achieve up to 100 times speedup while maintaining the similar accuracy,and its effectiveness is significantly better than that of the hybrid methods.

Key words: Bayesian network, Structure learning, Order search, Evolutionary optimization, Knowledge structure discovery

CLC Number: 

  • TP183
[1]TING C,CHEAH W,CHIUNG C.Student engagement mode-ling using bayesian networks[C]//Proceedings of IEEE International Conference on Systems,Man,and Cybernetics.New York:IEEE,2013:2939-2944.
[2]LI J R,LING X B,LI C X,et al.Dynamic Network SecurityAnalysis Based on Bayesian Attack Graphs [J].Computer Science,2022,49(3):62-69.
[3]WANG S F,HE S,WU Y,et al.Fusion of visible and thermalimages for facial expression recognition [J].Frontiers of Computer Science,2014 8(2):232-242.
[4]XU Y Y,CHAI Y M,WANG L M,et al.Emotional Sentence Classification Method Based on OCC Model and Bayesian Network [J].Computer Science,2020,47(3):222-230.
[5]SPIRTES P,CLARK G,RICHARD S.Causation,prediction,and search [M].Massachusetts:MIT Press,2000:221-234.
[6]MARGARITIS D.Learning Bayesian network model structure from data [D].Pittsburgh,Pennsylvania:Carnegie-Mellon Univ,Pittsburgh Pa School of Computer Science,2003.
[7]SCHWARZ G.Estimating the dimension of a model [J].The Annals of Statistics,1978,6(2):461-464.
[8]AKAIKE H.A new look at the statistical model identification [J].IEEE Transactions on Automatic Control,1974,19(6):716-723.
[9]BUNTINE W.Theory refinement on Bayesian networks[C]//Proceedings of the 7th Conference on Uncertainty in Artificial Intelligence.San Francisco:Morgan Kaufmann,1991:52-60.
[10]HECKERMAN D,DAN G,DAVID M C.Learning Bayesiannetworks:The combination of knowledge and statistical data [J].Machine Learning,1995,20(3):197-243.
[11]COOPER G F,HERSKOVITS E.A Bayesian method for the induction of probabilistic networks from data [J].Machine Lear-ning,1992,9(4):309-347.
[12]LARRANAGA P,KUIJPERS C M H,MURGA R H,et al.Learning Bayesian network structures by searching for the best ordering with genetic algorithms [J].IEEE Transactions on Systems,Man and Cybernetics,Part A:Systems and Humans,1996,26:487-493.
[13]CHICKERING D M.Optimal structure identification withgreedy search [J].Journal of Machine Learning Research,2002,3:507-554.
[14]COOPER G F.The computational complexity of probabilistic inference using Bayesian belief networks [J].Artificial Intelligence,1990,42(2/3):393-405.
[15]TSAMARDINOS I,BROWN L E,ALIFERIS C F.The max-min hill-climbing Bayesian network structure learning algorithm [J].Machine Learning,2006,65(1):31-78.
[16]GASSE M,ALEX A,HAYTHAM E.A hybrid algorithm forBayesian network structure learning with application to multi-label learning [J].Expert Systems with Applications,2014,41(15):6755-6772.
[17]HOLLAND J.Adaptation in Natural and Artificial Systems[M].Ann Arbor:University of Michigan Press,1975:174-186.
[18]JIN Y C,SENDHOFF B.A systems approach to evolutionary multiobjective structural optimization and beyond [J].IEEE Computational Intelligence Magazine,2009,4(3):62-76.
[19]QIAN H,YU Y.Derivative-free reinforcement learning:a re-view [J].Frontiers of Computer Science,2021,15(6):156336.
[20]LARRANAGA P,POZA M,YURRAMENDI Y,et al.Structure learning of Bayesian networks by genetic algorithms:A perfor-mance analysis of control parameters [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1996,18(9):912-926.
[21]ETXEBERRIA R,LARRANAGA P,JUAN M P.Analysis ofthe behaviour of genetic algorithms when learning Bayesian network structure from data [J].Pattern Recognition Letters,1997,18(11/12/13):1269-1273.
[22]MATZKEVICH I,BRUCE A.The topological fusion of Bayes nets[C]//Proceedings of the 8th Annual Conference on Uncertainty in Artificial Intelligence.San Francisco:Morgan Kaufmann,1992:191-198.
[23]LV Y L,MIAO J Z,LIANG J Y,et al.BIC-based node order learning for improving Bayesian network structure learning [J].Frontiers of Computer Science,2021 15(6):156337.
[24]AOUAY S,JAMOUSSI S,AYED Y B.Particle swarm optimization based method for Bayesian network structure learning[C]//Proceedings of 5th International Conference on Modeling,Simulation and Applied Optimization.New York:IEEE,2013:1-6.
[25]WU Y H,JOHN M,DAVID C.Two novel ant colony optimization approaches for Bayesian network structure learning[C]//Proceedings of the 2010 IEEE Congress on Evolutionary Computation.New York:IEEE,2010:1-7.
[26]SPIRTES P,ZHANG K.Causal discovery and inference:con-cepts and recent methodological advances[C]//Proceedings of Applied Informatics.Springer Open,2016:1-28.
[27]PEARL J.Causality:models,reasoning and inference(SecondEdition)[M].Cambridge:Cambridge University Press,2009:123-145.
[28]SPIRTES P,MEEK C.Learning Bayesian networks with dis-crete variables from data[C]//Proceedings of the 1st Annual Conference on Knowledge Discovery and Data Mining,Montreal.Menlo Park:Morgan Kaufmann,1995:294-299.
[29]LAURITZEN S L,DAVID J S.Local computations with probabilities on graphical structures and their application to expert systems [J].Journal of the Royal Statistical Society:Series B(Methodological),1988,50(2):157-194.
[30]SACHS K,PEREZ O,PE'ER D,et al.Causal protein-signaling networks derived from multi parameter single-cell data[J].Science,2005,308(5721):523-529.
[31]SPIEGELHALTER D J,DAWID A P,LAURITZENS L,et al.Bayesian analysis in expert systems [J].Statistical Science,1993,8(3):219-247.
[32]BINDER J,KOLLER D,RUSSELL S,et al.Adaptive probabilistic networks with hidden variables [J].Machine Learning,1997,29(2):213-244.
[33]BEINLICH I A,SUERMONDTH J,CHAVEZ R M,et al.The ALARM monitoring system:A case study with two probabilistic inference techniques for belief networks[C]//LNMI 38:Proceedings of the 2nd European Conference on Artificial Intelligence in Medicine.Berlin,Heidelberg:Springer,1989:247-256.
[34]ABRAMSON B,BROWN J,EDWARDS W,et al.Hailfinder:A Bayesian system for forecasting severe weather [J].Interna-tional Journal of Forecasting,1996,12(1):57-71.
[35]COLOMBO D,MATHUIS M H.Order-Independent Constraint-Based Causal Structure Lear-ning [J].Journal of Machine Learning Research,2014,15:3921-3962.
[36]WU R Z,LIU Q,LIU Y P,et al.Cognitive modelling for predicting examinee performance[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence.Menlo Park:AAAI,2015:1017-1024.
[37]TORRE J.DINA model and parameter estimation:A didactic[J].Journal of Educational and Behavioral Statistics,2009,34(1):115-130.
[1] XU Miao, WANG Huiling, LIANG Yi, QI Xiaolong, GAO Yang. Improved K2 Algorithm Based on Two-step Search Strategy [J]. Computer Science, 2023, 50(9): 303-310.
[2] LI Jia-rui, LING Xiao-bo, LI Chen-xi, LI Zi-mu, YANG Jia-hai, ZHANG Lei, WU Cheng-nan. Dynamic Network Security Analysis Based on Bayesian Attack Graphs [J]. Computer Science, 2022, 49(3): 62-69.
[3] ZHONG Kun-hua, CHEN Yu-wen, QIN Xiao-lin. Sub-BN-Merge Based Bayesian Network Structure Learning Algorithm [J]. Computer Science, 2022, 49(11A): 210800172-7.
[4] LI Chao, QIN Biao. Efficient Computation of Intervention in Causal Bayesian Networks [J]. Computer Science, 2022, 49(1): 279-284.
[5] LI Chao, QIN Biao. Efficient Computation of MPE in Causal Bayesian Networks [J]. Computer Science, 2021, 48(4): 14-19.
[6] XU Yuan-yin,CHAI Yu-mei,WANG Li-ming,LIU Zhen. Emotional Sentence Classification Method Based on OCC Model and Bayesian Network [J]. Computer Science, 2020, 47(3): 222-230.
[7] ZHANG Cheng-wei, LUO Feng-e, DAI Yi. Prediction Method of Flight Delay in Designated Flight Plan Based on Data Mining [J]. Computer Science, 2020, 47(11A): 464-470.
[8] LIN Lang, ZHANG Zi-li. Bayesian Structure Learning Based on Physarum Polycephalum [J]. Computer Science, 2019, 46(9): 206-210.
[9] CHAI Hui-min, FANG Min, LV Shao-nan. Local Path Planning of Mobile Robot Based on Situation Assessment Technology [J]. Computer Science, 2019, 46(4): 210-215.
[10] LIU Hui-qing, GUO Yan-bu, LI Hong-ling, LI Wei-hua. Short Text Feature Extension Method Based on Bayesian Networks [J]. Computer Science, 2019, 46(11A): 66-71.
[11] XING Zhi-wei, ZHU Hui, LI Biao, LUO Qian. Dynamic Estimation of Flight Departure Time Based on Bayesian Network [J]. Computer Science, 2019, 46(10): 329-335.
[12] ZHANG Zhi-dong, WANG Zhi-hai, LIU Hai-yang and SUN Yan-ge. Ensemble Multi-label Classification Algorithm Based on Tree-Bayesian Network [J]. Computer Science, 2018, 45(3): 189-195.
[13] XU Jian-rui, LI Zhan-wu and XU An. KDE-CGA Algorithm of Structure Learning for Small Sample Data Bayesian Network [J]. Computer Science, 2017, 44(Z11): 437-441.
[14] CAO Ru-sheng, NI Shi-hong and ZHANG Peng. Bayesian Networks Structure Learning Algorithm Based on Cloud Genetic Annealing [J]. Computer Science, 2017, 44(9): 239-242.
[15] LIU Dong-rui, LIU Dong-sheng, ZHANG Li-ping, HOU Min and WANG Chun-hui. Prediction of Code Clone Quality Based on Bayesian Network [J]. Computer Science, 2017, 44(4): 165-168.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!