LIBLINEAR MKL : A Fast Multiple Kernel Learning L1/L2-loss SVM solver in MATLAB

Ming-Hen Tsai

Introduction

Multiple kernel learning is a model to merge multiple kernels by linear combination. Mostly solving the models are slow due to explicit computation of kernels.
Here, we propose to approximate kernel map function explicitly in finite dimensional space. Then, we use dual coordinate descent to solve the SVM. By storing the solutions in primal, we do not have to compute the kernel explicitly. A group lasso regularization on kernel weights is solved with SVM alternatingly.
This is a side-project in my research projects with Dr. Yi-Ren Yeh and Dr. Frank Wang in Academia Sinica.

Platforms

(Unix-like) Matlab

Solvers Included

1. L2-regularized L1-loss linear SVM
2. L2-regularized L2-loss linear SVM

Features Included

1. Multiclass classification
2. VLfeat homogeneous kernel map (binary)

To Do

1. Try to include various appoximated or exact mapping of kernel functions
2. Try to extend to multiclass setting
3. Try to add other kernels combination parameter regularization
4. Try some benchmark data sets (e.g. Caltech 101)

Possible Applications

1. Fast feature (set) selection

Installation

Download it, go to "./src/matlab" directory. Unzip it and type "make" in the directory.

Usage

There are two functions:
function [mklmodel] = train_mkl(y,x,phis,liblinear_options)
function [decision_values predicted_results] = predict_mkl(y,x,mklmodel)
, where x are the training instances matrix in row format, y are the labels, and phis are the explicit mapping functions in matlab cell format.

Run and See "./src/matlab/run_mkl.m" as an example.

Results

We ran the software on some data sets included in [1] with linear mappings, and results could be found here and here.

Download

Download zip (Last modified June,26,2011)
The newest version will be in GitHub. For Linux users, you can checkout by typing "git clone git://github.com/minghen/liblinear-mkl.git".

Documents

Detailed derivation of LIBLINEAR MKL

Reference

1. Xu et al, Simple and Efficient Multiple Kernel Learning by Group Lasso, ICML 2010
2. Vedaldi and Zisserman, Efficient Additive Kernels via Explicit Feature Maps, CVPR 2010
3. Sreekanth et al, Generalized RBF Feature Maps for Efficient Detection, 2010
4. Hsieh et al, A Dual Coordinate Descent Method for Large-scale Linear SVM, ICML 2008
5. Chang et al, Training and Testing Low-degree Polynomial Data Mappings via Linear SVM, JMLR V11
6. Fan et al, LIBLINEAR: A library for large linear classification, JMLR V9

Comments

I will be glad to know any good methods that I haven't included in the software, so I can implement them and include them in it. Also, I'd like to know how can I improve the software. Thus, please let me know by sending e-mail to scan33scan33 AT gmail.com if you have any comments.

Last modified: June,27 2011 17:23:58 (UTC+8)