Dimensionality Reduction and Fusion Strategies for the Design of Parametric Signal Classifiers

Author

Srinivas Kota, Southern Illinois University CarbondaleFollow

Date of Award

12-1-2010

Degree Name

Doctor of Philosophy

Department

Electrical and Computer Engineering

First Advisor

Gupta, Lalit

Abstract

This dissertation focuses on two specific problems related to the design of parametric signal classifiers: dimensionality reduction to overcome the curse of dimensionality and information fusion to improve classification by exploiting complementary information from multiple sensors or multiple classifiers. Dimensionality reduction is achieved by introducing a strategy to rank and select a subset of principal component transform (PCT) coefficients that carry the most useful discriminatory information. The criteria considered for ranking transform coefficients include magnitude, variance, inter-class separation, and classification accuracies of individual transform coefficients. The ranking strategy not only facilitates overcoming the dimensionality curse for multivariate classifier implementation but also provides a means to further select, out of a rank-ordered set, a smaller set of features that give the best classification accuracies. Because the class-conditional densities of transform feature vectors are often assumed to be multivariate Gaussian, the dimensionality reduction strategy focuses on overcoming the specific problems encountered in the design of practical multivariate Gaussian classifiers using transform feature vectors. Through experiments with event related potentials (ERPs) and ear pressure signals, it is shown that the dimension of the feature space can be decreased quite significantly by means of the feature ranking and selection strategy. Furthermore, the resulting Gaussian classifiers yield higher classification accuracies than those reported in previous classification studies on the same signal sets. Amongst the four feature selection criteria, Gaussian classifiers using the maximum magnitude and maximum variance selection criteria gave the best classification accuracies across the two sets of classification experiments. For the multisensor case, dimensionality reduction is achieved by introducing a spatio-temporal array model to observe the signals across channels and time, simultaneously. A two-step process which uses the Kolmogrov-Smirnov test and the Lilliefors test is formulated to select the array elements which have different Gaussian densities across all signal categories. Selecting spatio-temporal elements that fit the assumed model and also statistically differ across the signal categories not only decreases the dimensionality significantly but also ensures high classification accuracies. The selection is dynamic in the sense that selecting spatio-temporal array elements corresponds to selecting samples of different sensors at different time-instants. Each selected array element is classified using a univariate Gaussian classifier and the resulting decisions are fused into a decision fusion vector which is classified using a discrete Bayes classifier. The application of the resulting dynamic channel selection-based classification strategy is demonstrated by designing and testing classifiers for multi-channel ERPs and it is shown that strategy yields high classification accuracies. Most noteworthy of the two dimensionality reduction strategies is the fact that the multivariate Gaussian signal classifiers developed can be implemented without having to collect a prohibitively large number of training signals simply to satisfy the dimensionality conditions. Consequently, the classification strategies can be beneficial for designing personalized human-machine-interface (HMI) signal classifiers for individuals from whom only a limited number of training signals can reliably be collected due to severe disabilities. The information fusion strategy introduced is aimed at improving the performance of signal classifiers by combining signals from multiple sensors or by combining decisions of multiple classifiers. Fusion classifiers with diverse components (classifiers or data sets) outperform those with less diverse components. Determining component diversity, therefore, is of the utmost importance in the design of fusion classifiers which are often employed in clinical diagnostic and numerous other pattern recognition problems. A new pairwise diversity-based ranking strategy is introduced to select a subset of ensemble components, which when combined, will be more diverse than any other component subset of the same size. The strategy is unified in the sense that the components can be either polychotomous classifiers or polychotomous data sets. Classifier fusion and data fusion systems are formulated based on the diversity selection strategy and the application of the two fusion strategies are demonstrated through the classification of multi-channel ERPs. From the results it is concluded that data fusion outperforms classifier fusion. It is also shown that the diversity-based data fusion system outperforms the system using randomly selected data components. Furthermore, it is demonstrated that the combination of data components that yield the best performance, in a relative sense, can be determined through the diversity selection strategy.