一种新型解决非光滑伪凸优化问题的神经网络方法

doi:10.11896/jsjkx.181001926

摘要/Abstract

摘要： 优化问题的研究一直以来深受科研工作者的关注,凸优化问题作为优化问题的一个重要部分更是成为研究重点,许多应用神经网络思想提出的模型已经被应用到实际问题中。然而,在机器学习、信号处理、生物信息学等领域中涉及的优化问题往往不是凸优化问题,而是伪凸优化及非凸优化的问题,因此解决后一类问题变得刻不容缓。针对目标函数是非光滑伪凸函数、约束函数由等式和不等式函数构成的优化问题,基于罚函数以及微分包含的思想,构建了一个新型的不含惩罚参数的单层神经网络模型。该模型的主要设计思路是根据已经提出的神经网络模型思想,为目标函数的梯度设计一个制约的函数,使其值始终保持在一个范围之内,再结合一个关于时间的函数,确保其值随时间变小。同时,考虑到不等式约束对状态解进入等式约束之前的收敛方向有影响,加入一个条件函数来限制它。与已提出的神经网络模型相比,所提模型具有结构简单、无须提前进行惩罚参数的计算、对初始点的位置无特殊要求等优势。而且,对于任意初始点,理论证明了状态解的有界性、状态解能够在有限时间内收敛到等式约束内并永驻其中、状态解能够在有限时间内收敛到可行域并永驻其中以及状态解最终收敛到优化问题的最优解。在MATLAB环境下,通过数学仿真实验,状态解都能快速地收敛到一个最优解。同时,用已经提出的类似神经网络模型解决同样的优化问题时,若罚参数或初始点选择不恰当则会导致状态解不能很好地收敛。这不仅验证了所提出的理论结果的正确性,同时也说明了所提网络具有更广泛的应用范围。

关键词: 非光滑优化, 神经网络, 微分包含, 伪凸函数

Abstract: The research of optimization problem is favored by researchers.As an important part of optimization pro-blem,convex optimization problem is the focus of research.Many models based on neural network are applied to practical problems.However,the optimization problems involved in machine learning,signal processing,bioinformatics and other fields are often not convex optimization problems,but pseudoconvex optimization and nonconvex optimization problems.Therefore,it is urgent to solve the latter kind of problems.To solve the optimization problem that the objective function is nonsmooth pseudoconvex function and constraint function is equality and inequality function,this paper constructeda new single-layer neural network model without penalty parameter based on the idea of penalty function and differential inclusion.The main idea of the design is that according to the proposed neural network model,a constrained function can be designed for the gradient of the objective function so that the value of the objective function is always kept within a range,and then a function about time is combined to ensure that its value decreases with time.At the same time,considering that inequality constraints affect the convergence direction of the state solution before it enters the equation constraint,a conditional function is added to restrict it.Compared with the proposed neural network model,it has the advantages of simple structure,no need to calculate penalty parameters in advance,and no special requirements for the position of the initial point.Furthermore,it is theoretically proved that for any initial point,the state solution can converge to the equality constraints in finite time and stay there thereafter,the boundedness of the state solution,the state solution can converge to the feasible region in finite time and stay there thereafter,and the state solution can finally converge to the optimal solution of the optimization problem.Under the environment of MATLAB,by mathematical simulation experiments,the state solution can converge to the optimal solution quickly.At the same time,if the penalty parameters or initial points are not selected properly,the state solution will not converge well when the same optimization problem is solved by using the proposed similar neural network model.This not only verifies the correctness of the theoretical results,but also shows that the proposed network has a wider range of applications.

Key words: Differential inclusion, Neural network, Nonsmooth optimization, Pseudoconvex function

中图分类号:

TP183

喻昕, 马崇, 胡悦, 伍灵贞, 汪炎林. 一种新型解决非光滑伪凸优化问题的神经网络方法[J]. 计算机科学, 2019, 46(11): 228-234. https://doi.org/10.11896/jsjkx.181001926

YU Xin, MA Chong, HU Yue, WU Ling-zhen, WANG Yan-lin. New Neural Network Method for Solving Nonsmooth Pseudoconvex Optimization Problems[J]. Computer Science, 2019, 46(11): 228-234. https://doi.org/10.11896/jsjkx.181001926

参考文献

[1]TANK D W,HOPFIELD J J.Simple ‘neural’ optimization networks:An A/D converter,signal decision circuit,and a linear programming circuit[J].IEEE Transaction Circuits System,1986,33(5):533-541.
[2]KENNEDY M P,CHUA L O.Neural Networks for Nonlinear Programming[J].IEEE Trans.Circuits Syst.,1988,35(5):554-562.
[3]GAO X B.A novel neural network for nonlinear convex programming[J].IEEE Transactions on Neural Networks,2004,15(3):613-621.
[4]LIU Q,WANG J.A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming[J].IEEE Trans.Neural Netw.,2008,19(4):558-570.
[5]QIN S T,XUE X P.A two-layer recurrent neural network for nonsmooth convex optimization problems[J].IEEE Trans.Neural Netw.and Learn.Syst.,2015,26(6):1149-1160.
[6]LI Q,LIU Y,ZHU L.Neural network for nonsmooth pseudoconvex optimization with general constraints[J].Neurocomputing,2014,131(131):336-347.
[7]QIN S T,YANG X,XUE X P,et al.A one-Layer recurrent neural network for pseudoconvex optimization problems with equa-lity and inequality constraints[J].IEEE Trans. Cybern,2017,47(10):3063-3074.
[8]BIAN W,MA L T,QIN S T,et al.Neural network for nonsmooth pseudconvex optimization with general convex constraints[J].Neural Networks,2018,101:1-14.
[9]AUBIN J P,FRANKOWSKA H.Set-Valued Analysis[M].Berlin,Germany:Birkäuser,1990:1-39.
[10]CLARKE F.Optimization and Non-Smooth Analysis[M].New York:Wiley,1983:25-155.
[11]AUBIN J P,CELLINA A.Differential Inclusions[M].Berlin,Germany:Springer-Verlag,1984:78-164.
[12]BETOUNES D.Differential Equations:Theory and Applications[M].New York:Springer-Verlag,2009:100-203.
[13]FORTI M,NISTRI P,QUINCAMPOIX M.Generalized neural network for nonsmooth nonlinear programming problems[J].IEEE Transactions on Circuits & Systems I Regular Papers,2004,51(9):1741-1754.
[14]BIAN W,XUE X P.Subgradient-based neural networks for nonsmooth convex optimization problems[J].IEEE Trans.Circuits Syst.I,2008,55(8):2378-2391.
[15]LIU Q,WANG J.A one-Layer recurrent neural network forconstrained nonsmooth optimization[J].IEEE Transactions on Systems Man & Cybernetics Part B,2011,41(5):1323-1333.
[16]QIN S,FAN D,WU G,et al.Neural network for constrainednonsmooth optimization using Tikhonov regularization[J].Neural Netw,2015,63(C):272-281.
[17]LIU S,WANG J.A simplified dual neural network for quadratic programming with its KWTA application[J].IEEE Transactions on Neural Networks,2006,17(6):1500.
[18]LI G C,SONG S J,WU C.Generalized gradient projection neural networks for nonsmooth optimization problems[J].Science China (Information Sciences),2010,53(5):990-1005.
[19]BIAN W,XUE X P.Neural network for solving constrainedconvex optimization problems with global attractivity[J].IEEE Trans.Circuits Syst.I,2013,60(3):710-723.
[20]BIAN W,XUE X P.Subgradient-based neural networks for nonsmooth nonconvex optimization problems[J].IEEE Trans.Nneural Netw.,2009,20(6):1024-1038.
[21]WANG J.A deterministic annealing neural network for convex programming[J].Neural Networks,1994,7(4):629-641.
[22]GAO X,LIAO L Z.A new one-layer neural network for linear and quadratic programming[J].IEEE Transactions on Neural Networks,2010,21(6):918.
[23]CHENG L,HOU Z G,LIN Y,et al.Recurrent neural network for non-smooth convex optimization problems with application to the identification of genetic regulatory networks[J].IEEE Trans.Neural Netw.,2011,22(5):714-726.
[24]HOSSENINI A.A non-penalty recurrent neural network forsolving a class of constrained optimization problems[J].Neural Networks,2016,73:10-25.
[25]FORTI M,NISTRI P,QUINCAMPOIX M.Convergence ofneural network for programming problems via a nonsmooth Lojasiewicz Inequlity[J].IEEE Trans.Neural Netw.,2006,17(6):1471-1486.
[26]LIU Q S,GUO Z S,WANG J.A one-layer recurrent neural network for constrained pseudoconvex optimization and its application for dynamic portfolio optimization[J].Neural Networks,2011,26:99-109.
[27]BIAN W,XUE X P.Neural network for solving constrainedconvex optimization problems with global attractivity[J].IEEE Trans.Circuits Syst.I,2013,60(3):710-723.

相关文章 15

[1]	宁晗阳, 马苗, 杨波, 刘士昌. 密码学智能化研究进展与分析 Research Progress and Analysis on Intelligent Cryptology 计算机科学, 2022, 49(9): 288-296. https://doi.org/10.11896/jsjkx.220300053
[2]	周芳泉, 成卫青. 基于全局增强图神经网络的序列推荐 Sequence Recommendation Based on Global Enhanced Graph Neural Network 计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[3]	周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[4]	李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[5]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6]	王润安, 邹兆年. 基于物理操作级模型的查询执行时间预测方法 Query Performance Prediction Based on Physical Operation-level Models 计算机科学, 2022, 49(8): 49-55. https://doi.org/10.11896/jsjkx.210700074
[7]	陈泳全, 姜瑛. 基于卷积神经网络的APP用户行为分析方法 Analysis Method of APP User Behavior Based on Convolutional Neural Network 计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[8]	朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[9]	檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[10]	闫佳丹, 贾彩燕. 基于双图神经网络信息融合的文本分类方法 Text Classification Method Based on Information Fusion of Dual-graph Neural Network 计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[11]	金方焱, 王秀利. 融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取 Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM 计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[12]	彭双, 伍江江, 陈浩, 杜春, 李军. 基于注意力神经网络的对地观测卫星星上自主任务规划方法 Satellite Onboard Observation Task Planning Based on Attention Neural Network 计算机科学, 2022, 49(7): 242-247. https://doi.org/10.11896/jsjkx.210500093
[13]	费星瑞, 谢逸. 基于HMM-NN的用户点击流识别 Click Streams Recognition for Web Users Based on HMM-NN 计算机科学, 2022, 49(7): 340-349. https://doi.org/10.11896/jsjkx.210600127
[14]	赵冬梅, 吴亚星, 张红斌. 基于IPSO-BiLSTM的网络安全态势预测 Network Security Situation Prediction Based on IPSO-BiLSTM 计算机科学, 2022, 49(7): 357-362. https://doi.org/10.11896/jsjkx.210900103
[15]	齐秀秀, 王佳昊, 李文雄, 周帆. 基于概率元学习的矩阵补全预测融合算法 Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning 计算机科学, 2022, 49(7): 18-24. https://doi.org/10.11896/jsjkx.210600126

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed