Classifier Combining for Handwritten Digit Recognition

Dejan Gorgevik

Abstract: The ability of pattern recognition is certainly one of the key features of intelligent behavior. Pattern recognition as a feature of technical systems is one of the main challenges of the machine intelligence area. Its task is to react purposefully on certain signals coming from the environment. In order to improve recognition results one can improve the type and number of features used or the classifiers themselves. One of the processes that can be used to improve the classification results is to combine the decisions of several classifiers instead of using only one. Data from more than one source that are processed separately can often be profitably recombined to produce more concise, more complete and/or more accurate situation description. This thesis present an effort to understand the individual strengths and weaknesses of some classifier combining methodologies and to deduce what is the most promising combining strategy under given circumstances. A revision of the state of the art in combining information from several classifiers is presented. Some new methodologies for classifier combining are proposed. Modification and extension of some of the classifier combining methodologies were also presented. Extensive experiments in combining multiple classifiers on the task of handwritten digit recognition were carried out. For that purpose an effort was made to extract original sophisticated features from the digit images. A new approach for detecting the slant angle of the isolated digits is suggested. The classifiers that were used are neural network classifier and support vector machine (SVM) utilizing linear and Gaussian kernel. A new approach for discovering the optimal parameters for training of the SVM classifiers that is much faster and does not require limiting the search to a certain range of parameters is presented. A comparison of the obtained gains when combining classifiers using different features is presented and analyzed regarding the type of the individual classifiers, their strength as well as the number of patterns used to train the classifiers and the combiner. An efficient three-stage classifier for handwritten digit recognition using a sequential combination of two neural networks and one SVM is also presented.

Keywords: classifier combining, handwritten digit recognition, support vector machines,
neural networks, decision fusion