Abstract: The loss function of the traditional model in the field of machine learning is convex,so it has a global optimal solution.The optimal solution can be obtained through the traditional gradient descent algorithm (SGD).However,in the field of deep learning,due to the implicit expression of the model function and the interchangeability of neurons in the same layer,the loss function is a non-convex function.Traditional gradient descent algorithms cannot find the optimal solution,even the more advanced optimization algorithms such as SGDM,Adam,Adagrad,and RMSprop cannot escape the limitations of local optimal solutions.Although the convergence speed has been greatly improved,they still cannot meet the actual needs.A series of existing optimization algorithms are improved based on the defects or limitations of the previous optimization algorithms,and the optimization effect is slightly improved,but the performance of different data sets is inconsistent.This article proposes a new optimization mechanism Rain,which combines the dropout mechanism in deep neural networks and integrates it into the optimization algorithm to achieve.This mechanism is not an improved version of the original optimization algorithm.It is a third-party mechanism independent of all optimization algorithms,but it can be used in combination with all optimization algorithms to improve its adaptability to data sets.This mechanism aims to optimize the performance of the model on the training set.The generalization problem on the test set is not the focus of this mechanism.This article uses Deep Crossing and FM two models with five optimization algorithms to conduct experiments on the Frappe and MovieLens data sets respectively.The results show that the model with the Rain mechanism has a significant reduction in the loss function value on the training set,and the convergence speed is accelerated,but its performance on the test set is almost the same as the original model,that is,its generalization is poor.

Key words: Convergence speed, Deep learning, Dropout mechanism, Optimization algorithm, Rain mechanism