LIBLINEAR FAQ

Some questions are listed in LIBSVM FAQ.

Table of Contents

Please check our explanation on the LIBLINEAR webpage. Also see appendix B of our SVM guide.

Please see the descriptions at LIBLINEAR page.

Please cite the following paper:

R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIBLINEAR: A Library for Large Linear Classification, Journal of Machine Learning Research 9(2008), 1871-1874. Software available at http://www.csie.ntu.edu.tw/~cjlin/liblinear

The bibtex format is

@Article{REF08a,
  author = 	 {Rong-En Fan and Kai-Wei Chang and Cho-Jui Hsieh and Xiang-Rui Wang and Chih-Jen Lin},
  title = 	 {{LIBLINEAR}: A Library for Large Linear Classification},
  journal = 	 {Journal of Machine Learning Research},
  year = 	 {2008},
  volume =	 {9},
  pages =	 {1871--1874}
}

See the change log and directory for earlier versions.

Generally we recommend linear SVM as its training is faster and the accuracy is competitive. However, if you would like to have probability outputs, you may consider logistic regression.

Moreover, try L2 regularization first unless you need a sparse model. For most cases, L1 regularization does not give higher accuracy but may be slightly slower in training.

Among L2-regularized SVM solvers, try the default one (L2-loss SVC dual) first. If it is too slow, use the option -s 2 to solve the primal problem.

[rcv1_test_labels,rcv1_test_inst] = libsvmread('../rcv1_test.binary');
save rcv1_test.mat rcv1_test_labels rcv1_test_inst;

save -mat7-binary rcv1_test.mat rcv1_test_labels rcv1_test_inst;

load rcv1_test.mat

help save

-s 2

You can use grid.py of LIBSVM to check cross validation accuracy of different C.

First, you need to modify three places from

        cmdline = '%s -c %s -g %s -v %s %s %s' % \
          (svmtrain_exe,c,g,fold,pass_through_string,dataset_pathname)

        cmdline = '%s -c %s -v %s %s %s' % \
          (svmtrain_exe,c,fold,pass_through_string,dataset_pathname)

Second, run

> grid.py -log2c -3,0,1 -log2g 1,1,1 -svmtrain ./train

We guess that you are comparing

> time ./train -s 0 -v 5 -e 0.001 data

1. The above timeing of LIBLINEAR includes time for reading data, but in the paper we exclude that part.
2. In the paper, to conduct 5-fold (or 2-fold) CV we group folds used for training as a separate matrix, but LIBLINEAR simply uses pointers of the corresponding instances. Therefore, in doing matrix-vector multiplications, the former sequentially uses rows in a continuous segment of the memory, but the latter does not. Thus, LIBLINEAR may be slower but it saves the memory.

We carefully studied such issues, and decided to use the current setting. For data classification, one doesn't need very accurate solution, so numerical issues are less important. Moreover, log1p is not available on all platforms. Please let us know if you observe any numerical problems.

Assume k is the total number of classes and n is the number of features. In the model file, after the parameters, there is an n*k matrix W, whose columns are obtained from solving two-class problems: 1 vs rest, 2 vs rest, 3 vs rest, ...., k vs rest. For example, if there are 4 classes, the file looks like:

+-------+-------+-------+-------+
| w_1vR | w_2vR | w_3vR | w_4vR |
+-------+-------+-------+-------+

Please see the answer in LIBSVM faq.

To correctly obtain decision values, you need to check the array

label

LIBSVM uses more advanced techniques for SVM probability outputs. We don't know yet if they should be included in LIBLINEAR.

If you really would like to have probability outputs for SVM in LIBLINEAR, you can consider using the simple probability model of logistic regression. Simply remove the following if statament in the subroutine predict_probability in linear.cpp.

int predict_probability(const struct model *model_, const struct feature_node *x, double* prob_estimates)
{
	if(model_->param.solver_type==L2R_LR)
	{