Abstract: We considered a type of learning problems whose objective functions can be formulated as an average of a large number of component functions,and assumed that each component function is smooth.Among many machine learning methods,online learning provides a favorable tool for big data learning,not only due to its simple operation and fast convergence rate,but also for its ability to automatically update model.To solve such problems,we developed a stochastic spectral gradient descent (S²GD) method,which employs the Rayleigh quotient to collect second-order information to construct Hessian inverse approximations.S²GD can be viewed as an approach that extends the spectral gradient method working in deterministic setting to stochastic setting.At each iteration,the generated search direction guarantees descent property.The existing conclusion indicates that the S²GD method is of convergence.Preliminary experimental results on LIBSVM data sets are reported to demonstrate the feasibility and effectiveness of our approach.

Key words: Online learning,Stochastic optimization,Convex optimization,Stochastic gradient,Spectral gradient