Abstract: Temporal Difference (Abbr. TD) learning algorithm was used to adjust weights of evaluation function by using Connect6 game as testbed in this paper,which makes the weights adjustment process can be done automatically. A new evaluation scheme was proposed,which can solve the difficult to combine the prior knowledge and multi-layer neural network organically. On account of the specific application,the method selecting part of the whole TD sectuence to learn was proposed, by which the interference of useless states is prevented to a certain extent. After 10020 self-learning training, the winning rate is increased with 8 % around against the same Connect6-playing program, which is a good result.

Key words: Computer games,Temporal difference learning,Connect6