Abstract: Feature selection is one of the core issues in designing pattern recognition systems and has attracted conside- ruble attention in the literature. Most of the feature selection methods in the literature only handle relevance and redun- dancy analysis from the point of view of the whole class, which neglect the relation of features and the separate class la- bels. To this end, a novel KL-divergence based feature selection algorithm was proposed to explicitly handle the rele- vance and redundancy analysis for each class label with a separate-class strategy. A KL-divergence based metric of effec- tive distance was also introduced in the algorithm to conduct the relevance and redundancy analysis. Experimental re- sups show that the proposed algorithm is efficient and outperforms the three representative algorithms CFS,FCI3F and RcliefF with respect to the quality of the selected feature subset.

Key words: Feature selection, KI= divergence, Separatcclass strategy, Effective distance