LIBSVM FAQ

Last modified: Wed Jul 24 00:08:51 CST 2002

Some courses which have used libsvm as a tool

Institute for Computer Science, Faculty of Applied Science, University of Freiburg, Germany
Division of Mathematics and Computer Science. Faculteit der Exacte Wetenschappen Vrije Universiteit, The Netherlands.
Department of (Applied) Computer Science (Faculty of Sciences), Vrije Universiteit Brussel, Belgium
Electrical and Computer Engineering Department, University of Wisconsin-Madison
Technion (Israel Institute of Technology), Israel.
Computer and Information Sciences Dept., University of Florida
The Institue of Computer Science, University of Nairobi, Kenya.

Installation and and running the program

Where can I find documents of libsvm ?
In the package there is a README file which details all options, data format, and library calls. The paper LIBSVM : a library for support vector machines discusses the implementation of libsvm in detail.
What are changes in previous versions?
See the change log.
I have modified the source and would like to build the graphic interface "svm-toy" on MS windows. How should I do it ?
Build it as a project by chooising "Win32 Application." On the other hand, for "svm-train" and "svm-predict" you want to choose "Win32 Console Application."
I would like to cite libsvm. Which paper should I cite ?
Please cite the following document:
Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
The bibformat is as follows
```
@Manual{CC01a,
  author =	 {Chih-Chung Chang and Chih-Jen Lin},
  title =	 {{LIBSVM}: a library for support vector machines},
  year =	 {2001},
  note =	 {Software available at {\tt http://www.csie.ntu.edu.tw/\verb"~"cjlin/libsvm}},
}
```

Data preparation

Why sometimes not all attributes of a data appear in the training/model files ?
libsvm uses the so called "sparse" format where zero values do not need to be stored. Hence a data with attributes
```
1 0 2 0
```
is represented as
```
1:1 3:2
```
What if my data are non-numerical ?
Currently libsvm supports only numerical data. You may have to change non-numerical data to numerical. For example, you can use several binary attributes to represent a categorical attribute.

Training and prediction

Should I use float or double to store numbers in the cache ?
We have float as the default as you can store more numbers in the cache. In general this is good enough but for few difficult cases (e.g. C very very large) where solutions are huge numbers, it might be possible that the numerical precision is not enough using only float.
Does libsvm have special treatments for linear SVM ?
No, at this point libsvm solves linear/nonlinear SVMs by the same way. Note that there are some possible tricks to save training/testing time if the linear kernel is used. Hence libsvm is NOT particularly efficient for linear SVM, especially for problems whose number of data is much larger than number of attributes. If you plan to solve this type of problems, you may want to check some software which are specially suitable for linear SVMs.
The number of free support vectors is large. What should I do ?
This usually happens when the data are overfitted. If attributes of your data are in large ranges, try to scale them. Then the region of appropriate parameters may be larger. Note that there is a scale program in libsvm.
The prediction rate is low. How could I improve it ?
Try to use cross validation option -v and find out good parameters.
My data are unbalanced. Could libsvm handle such problems ?
Yes, there is a -wi options. For example, if you use
svm-train -s 0 -c 10 -w1 1 -w-1 5 data_file
the penalty for class "-1" is larger.
What is the difference between nu-SVC and C-SVC ?
Basically they are the same thing but with different parameters. The range of C is from zero to infinity but nu is always between [0,1]. A nice property of nu is that it is related to the ratio of support vectors and the ratio of the training error.
The program keeps running without showing any output. What should I do ?
You may want to check your data. Each training/testing data must be in one line. It cannot be separated. In addition, you have to remove empty lines.
The program keeps running (with output, i.e. many dots). What should I do ?
In theory libsvm guarantees to converge if the kernel matrix is positive semidefinite. Hence, this may mean that you are using the signomidal kernel where for some parameters the kernel matrix is not PSD.
The training time is too long. What should I do ?
This may happen for some difficult cases (e.g. -c is large). You can try to use a looser stopping tolerance with -e. If that still doesn't work, you may want to contact us. We can show you some tricks on improving the training time.
I want to know the value of the decision function. How do I get it ?
We print out decision values for regression. For classification, as we have to solve several binary SVMs for multi-class cases, we do not print out them. However, you can easily modify the program for printing them.Just add
```
       printf("%f ", sum);
```
after the line
```
       sum -= model->rho[p++];
```
of the file svm.cpp
For some problem sets if I use a large cache (i.e. large -m) on a linux machine, why sometimes I get "segmentation fault ?"
This may happen only when the cache is large, but each cached row is not large enough. Note: This problem is specific to gnu C library which is used in linux. The solution is as follows:
In our program we have malloc() which uses two methods to allocate memory from kernel. One is sbrk() and another is mmap(). sbrk is faster, but mmap has a larger address space. So malloc uses mmap only if the wanted memory size is larger than some threshold (default 128k). In the case where each row is not large enough (#elements < 128k/sizeof(float)) but we need a large cache , the address space for sbrk can be exhausted. The solution is to lower the threshold to force malloc to use mmap and increase the maximum number of chunks to allocate with mmap.
Therefore, in the main program (i.e. svm-train.c) you want to have
```
      #include <malloc.h>
```
and then in main():
```
      mallopt(M_MMAP_THRESHOLD, 32768);
      mallopt(M_MMAP_MAX,1000000);
```
You can also set the environment variables instead of writing thems in the program:
```
$ M_MMAP_MAX=1000000 M_MMAP_THRESHOLD=32768 ./svm-train .....
```
More information can be found by
```
$ info libc "Malloc Tunable Parameters"
```
Now for different classes I can assign weights. What if I want to give weights to different data ?
Please contact us so we can give you our preliminary code.
How do I disable screen output of svm-train and svm-predict ?
Simply update svm.cpp:
```
#if 1
void info(char *fmt,...)
```
to
```
#if 0
void info(char *fmt,...)
```
I would like to use my own kernel but find out that there are two subtourines for kernel evaluations: k_function() and kernel_function(). Which one should I modify ?
The reason why we have two functions is as follows: For the RBF kernel exp(-g |xi - xj|^2), if we calculate xi - xj first and then the norm square, there are 3n operations. Thus we consider exp(-g (|xi|^2 - 2dot(xi,xj) +|xj|^2)) and by calculating all |xi|^2 in the beginning, the number of operations is reduced to 2n. This is for the training. For prediction we cannot do this so a regular subrouting using that 3n operations is needed. The easiest way to have your own kernel is to put the same code in these two subourines by replacing any kernel.
What method does libsvm use for mulsit-class SVM ?
It is one-against-one. We chose it after doing the following comparison: C.-W. Hsu and C.-J. Lin. A comparison of methods for multi-class support vector machines , IEEE Transactions on Neural Networks, 13(2002), 415-425.

Graphic Interface

How can I save images drawn by svm-toy?
For Microsoft windows, first press the "print screen" key on the keyboard. Open "Microsoft Paint" (included in Windows) and press "ctrl-v." Then you can clip the part of picture which you want. For X windows, you can use the program "xv" to grab the picture of the svm-toy window.
I press the "load" button to load data points but why svm-toy does not draw them ?
The program svm-toy assumes both attributes (i.e. x-axis and y-axis values) are in (0,1). Hence you want to scale your data to between a small positive number and a number less than but very close to 1.

Java version of libsvm

What is the difference between Java version and C++ version of libsvm ?
They are the same thing. We just rewrote the C++ code in Java.
Is the Java version significantly slower than the C++ version?
This depends on the VM you used. We have seen good VM which leads the Java version to be quite competitive with the C++ code. (though still slower)
While training I get the following error message: java.lang.OutOfMemoryError. What is wrong ?
You should try to increase the maximum Java heap size. For example,
```
java -Xmx256m svm_train.java ...
```
sets the maximum heap size to 256M.

Python Interface

Using python on MS windows, it fails to load the dll file.
It seems the dll file is version dependent. So far we haven't found out a good solution. Please email us if you have any good suggestions.
Except the python-C++ interface provided, could I use Jython to call libsvm ?
Yes, an example is here