Abstract: The traditional classification models are insufficient to take full advantage of the sequential data with their continuous and explosive growth due to the imprecision of the data.Therefore, the incremental learning is provided to handle this problem.However, the difference sequence of the training samples may have strong impact on performance of a classifier.Especially when the classifier is undertrained, traditional incremental learning method takes the risk of utilizing the noise samples with wrong labels to train the classifier.To overcome this problem, this paper proposes an incremental classification model based on Q-learning algorithm.The model employs the classical Q-learning algorithm in reinforcement learning to select the sequence samples incrementally, which is capable of softening the negative impact of the noise data and labels samples automatically as well.To overcome the problem of computational complexity along with the increasing of state space and action space of Q-learning, an improved batch incremental classification model based on Q-learning algorithm is proposed.Compared with the traditionally trained classifiers, the proposed model combines the ideas of online incremental learning and reinforcement learning, which is able to achieve high accuracy and can be updated online.Finally, the validity of the model is verified on three UCI datasets.The experimental results show that choosing training sets incrementally is helpful to improve the performance of the classifier and the precision of the classifier trained by different incremental training sequences varies greatly as well.The proposed incremental classification model based on Q-learning algorithm can make use of the limited available dataset for supervised initial training, and then construct new-added self-supervised training set based on the Q value of each unlabeled sample to improve the accuracy of the classifier.Therefore, the incremental classification model based on Q-learning algorithm can be used to solve the problem of lack of supervisory information, and has a potential application.

Key words: Classification, Incremental learning, Online learning, Q-learning, Reinforcement learning