LIBLINEAR -- A Library for Large Linear Classification

Machine Learning Group at National Taiwan University
Contributors

• A new python interface (using ctypes) is added.
• An extension for large data which cannot fit in memory is available at LIBSVM Tools. See our KDD 2010 paper.

Introduction

LIBLINEAR is a linear classifier for data with millions of instances and features. It supports

• L2-regularized classifiers
  L2-loss linear SVM, L1-loss linear SVM, and logistic regression (LR)
• L1-regularized classifiers (after version 1.4)
  L2-loss linear SVM and logistic regression (LR)

Main features of LIBLINEAR include

• Same data format as LIBSVM, our general-purpose SVM solver, and also similar usage
• Multi-class classification: 1) one-vs-the rest, 2) Crammer & Singer
• Cross validation for model selection
• Probability estimates (logistic regression only)
• Weights for unbalanced data
• MATLAB/Octave, Java, Python, Ruby interfaces

FAQ is here

When to use LIBLINEAR but not LIBSVM

% time libsvm-2.85/svm-train -c 4 -t 0 -e 0.1 -m 800 -v 5 rcv1_train.binary
Cross Validation Accuracy = 96.8136%
345.569s
% time liblinear-1.21/train -c 4 -e 0.1 -v 5 rcv1_train.binary
Cross Validation Accuracy = 97.0161%
2.944s

Warning:If you are a beginner and your data sets are not large, you should consider LIBSVM first.

Download LIBLINEAR

The package includes the source code in C/C++. A README file with detailed explanation is provided. For MS Windows users, there is a subdirectory in the zip file containing binary executable files.

Please read the COPYRIGHT notice before using LIBLINEAR. 

Documentation and Codes used for experiments in our papers

General information
R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIBLINEAR: A library for large linear classification Journal of Machine Learning Research 9(2008), 1871-1874.

Some examples are in Appendix C of our SVM guide.

The approach for L1-SVM and L2-SVM is a coordinate descent method:
C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S. Keerthi, and S. Sundararajan. A dual coordinate descent method for large-scale linear SVM. ICML 2008.

Multiclass classification for Crammer and Singer's formula:
S. S. Keerthi, S. Sundararajan. K.-W. Chang, C.-J. Hsieh, and C.-J. Lin, A sequential dual method for large scale multi-class linear SVMs . KDD 2008.

For LR and L2-SVM, we implement a trust region Newton method:
C.-J. Lin, R. C. Weng, and S. S. Keerthi. Trust region Newton method for large-scale logistic regression. Journal of Machine Learning Research 9(2008), 627--650.

Code used for experiments in our LIBLINEAR papers can be found here.

Interfaces to LIBLINEAR