Variable Weighted Maximal Relevance Minimal Redundancy Criterion for Feature Selection Using Normalized Mutual Information
Sanghamitra Bandyopadhyay, Tapas Bhadra and Ujjwal Maulik
Feature selection is an automatic choice for many pattern recognition tasks where dimensionality reduction is sought for minimizing the processing time. In spite of being a well-explored domain, mutual information based feature selection methods are currently in emergence because of their significant performance improvement. In this paper, we propose a weighted version of the well-known Maximal Relevance Minimal Redundancy criterion for the purpose of feature selection. The weight of the average redundancy of the candidate feature against all the selected features is continuously incremented with respect to the number of features already selected, while the weight of the class relevance of the candidate feature is kept fixed. An existing variant of normalized mutual information score is utilized for the first time to compute both the relevance as well as the average redundancy. The performance of the proposed approach is demonstrated to be superior to those of several conventional mutual information based feature selection techniques as well as some of the state-of-art feature selection approaches based on analyses on some real-life high dimensional datasets.
Keywords: Pattern recognition, feature selection, mutual information, normalized mutual information.