Gjorgji Madzarov, Dejan Gjorgjevikj, Ivica Dimitrovski

Abstract: In this paper a novel architecture of Support Vector Machine classifiers utilizing binary decision tree (SVMBDT) for solving multiclass problems is presented. The hierarchy of binary decision subtasks using SVMs is designed with clustering algorithm. For consistency between the clustering model and SVM the clustering model utilizes distance measures at the kernel space, not at the input space. The proposed SVM based Binary Decision Tree architecture takes advantage of both the efficient computation of the decision tree architecture and the high classification accuracy of SVMs. The performance of the proposed SVMBDT architecture was measured on a problem of recognition of handwritten digits and letters. The experiments were conducted with samples from MNIST, Pendigit, Optdigit and Statlog databases of segmented digits and letters. The results of the experiments indicate that maintaining comparable accuracy, SVM-BDT is faster to be trained than the other methods. Especially in classification, due to its Log complexity, it is much faster than the widely used multi-class SVM methods like “one-against-one” and “oneagainst- all” for multiclass problems. The experiments showed that this method becomes more favorable as the number of classes in the recognition problem increases.