Please check our explanation on the LIBLINEAR webpage.
In the package there is a README file which details all options, data format, and library calls. Please also check the appendix of our SVM guide about when and how to solve linear SVMs.
You can find implementation details in the following two papers
C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S. Keerthi,
and S. Sundararajan.
A dual coordinate descent method for large-scale linear SVM.
ICML 2008.
C.-J. Lin, R. C. Weng, and S. S. Keerthi.
Trust region Newton method for large-scale logistic
regression.
Journal of Machine Learning Research 9(2008), 627--650.
Very likely you use a large C or don't scale data. If your number of features is small, you may use the option
-s 0by solving the primal problem. More examples are in the appendix of our SVM guide.
See the change log. You can download earlier versions here.
You can use grid.py of libsvm to check cross validation accuracy of different C.
First, you need to modify three places from
cmdline = '%s -c %s -g %s -v %s %s %s' % \
(svmtrain_exe,c,g,fold,pass_through_string,dataset_pathname)
to
cmdline = '%s -c %s -v %s %s %s' % \
(svmtrain_exe,c,fold,pass_through_string,dataset_pathname)
Note that these three places are similar but slightly different.
Second, run
> grid.py -log2c -3,0,1 -log2g 1,1,1 -svmtrain ./trainto check CV values at C=2^-3, 2^-2, 2^-1, and 2^0
We guess that you are comparing
> time ./train -v 5 -e 0.001 datawith the environment used in our paper, and find that liblinear is slower. Two reasons may cause the diffierence.
We carefully studied such issues, and decided to use the current setting. For data classification, one doesn't need very accurate solution, so numerical issues are less important. Moreover, log1p is not available on all platforms. Please let us know if you observe any numerical problems.
For document classification, our experience indicates that if you normalize each document to unit length, then not only the training time is shorter, but also the performance is better.
Assume k is the total number of classes and n is the number of features. In the model file, after the parameters, there is an n*k matrix W, whose columns are obtained from solving two-class problems: 1 vs rest, 2 vs rest, 3 vs rest, ...., k vs rest. For example, if there are 4 classes, the file looks like:
+-------+-------+-------+-------+ | w_1vR | w_2vR | w_3vR | w_4vR | +-------+-------+-------+-------+
Please see the answer in libsvm faq.
To correctly obtain decision values, you need to check the array
labelin the model.