Experimental Results on Statlog Data Sets

This section reports the experiments conducted to evaluate the effects of QuickRBF package. In particular, we are interested in how the mechanism proposed in this paper performs in comparison with the SVM and other famous classifiers in data classification benchmark data sets.

The experiments in this section are conducted to evaluate the performance of the QuickRBF classifier against other famous classifiers, the RVKDE classifier [Oyang et al., 2005], APC-III [Hwang and Bang, 1997], LIBSVM [Chang and Lin, 2001] and KNN. The discussions of the experiments will focus on the following two issues: classification accuracy and execution efficiency. Also, regarding to the parameter settings of other classifiers for comparison, we adopted the parameter settings suggested by the authors in their original papers.

Table 1 lists the main characteristics of the three data sets used in the experiments. All three data sets, satimage, letter, and shuttle, are from the Statlog collection [Michie et al., 1994]. The hardware platform used in the experiments is a workstation with dual Pentium-III-1GHz CPUs, 2GB RAM, and the FreeBSD UNIX-release 4.10.

Table 2 lists the number of support vectors of the three data sets, and we use the support vectors which are selected by SVM as our centers to compare with SVM.

Table 3 compares the accuracy delivered by alternative classification algorithms with the three benchmark data sets. As Table 3 shows, the proposed method basically delivers the same level of accuracy with other famous classifiers, SVM and RVKDE, while the KNN and APC-III based classifier do not produce comparable generation results.

Table 4 compares the execution time of the RVKDE classifier, the SVM, the APC-III based classifier and the proposed method with the Statlog data sets. In Table 4, the total time taken to construct classifiers based on the given training data sets are listed in the rows marked by "Make classifier". The time listed in "Make classifier" row are the time of cross validation for RVKDE classifier, the time of model selection for SVM, and the time of clustering process, calculating bandwidths and weights of APC-III based classifier. Also, for proposed classifier, the reported time is the time of calculating weights. In addition, the time taken by alternative classifiers to predict the classes of the testing instances are listed in the rows marked by "Prediction".

The mechanism used in the QuickRBF package is more efficient than the SVM classifier for constructing a data classifier. Also, the QuickRBF classifier is basically at the same level or even more efficient than other classifiers. In addition, the QuickRBF classifier delivers comparable execution efficiency as the LIBSVM in the prediction phase and enjoys a 10 times speedup over the RVKDE classifier in this regard.