Gjorgji Madzarov, Dejan Gjorgjevik

Abstract: A novel architecture of Support Vector Machine classifiers utilizing binary decision tree (SVMDTA) for solving multiclass problems is proposed in this paper. A clustering algorithm was used to determine the hierarchy of binary decision subtasks performed by the SVM binary classifiers. The applied clustering model utilizes Mahalanobis distance measures at the kernel space for better consistency with the used SVM kernel. The proposed SVM based Binary Decision Tree architecture takes advantage of both the efficient computation of the decision tree architecture and the high classification accuracy of SVMs. The performance of the proposed SVM-DTA was estimated on a problem of recognition of handwritten digits and letters. The experiments were conducted with samples from Pendigit and Statlog databases of segmented digits and letters. The results of the experiments indicate that the proposed method is faster to be trained than the other methods. Also, due to its Log complexity, the proposed SVM-DTA is much faster than the widely used multi-class SVM methods like “one-against-one” and “one-against-all”, maintaining comparable accuracy. The experiments also showed that this method becomes more favorable as the number of classes in the recognition problem increases.