This directory includes sources used in the following paper:

Guo-Xun Yuan, Chia-Hua Ho, and Chih-Jen Lin.
An Improved GLMNET for L1-regularized Logistic Regression and
Support Vector Machines, Technical report, 2011.

This code has been tested under 64-bit Linux environments with
Python 2.6, MATLAB R2008b and R 2.9.

You will be able to regenerate results in the paper. However, results
may be slightly different due to the randomness, the CPU speed, and
the load of your computer. The running time is long as we run every
solver to accurately solve optimization problems.

Please cite the above article if you find this tool useful.
Please also read the COPYRIGHT before using this tool.

Introduction
============

You can choose solvers/data sets for comparisons. See descriptions
in subsequent sections.

We implement CDN and newGLMNET. GLMNET must be downloaded
from its web site. To obtain function values, testing accuracy,
and number of w's non-zeros after each run, we provide a patch
and you need to apply them.

Compare Solvers for Logistic Regression w/ Bias: CDN, GLMNET, and newGLMNET
===========================================================================

Edit 'compare_lrb.py' to indicate solvers and data sets for comparison.
Comment the solver that you are not interested in. For example,

#solvers['CDN'] = './liblinear/train'

In the same way, you can exclude some data sets from comparison. For
example,

#bestC['rcv1_test.binary'] = 4.0

After deciding data sets and solvers, you must prepare the data sets
and install the solvers. Please see Sections 'Prepare Data Sets for
Experiments' and 'Installation for Experiments' for more details.

Type

% python ./compare_lrb.py

to compare solvers. The results are stored in the 'logLRb' directory.

Compare Solvers for L2-loss SVM w/o Bias: CDN and newGLMNET
============================================================

Edit 'compare_svm.py' to indicate solvers and data sets for comparison.
See the section for 'compare_lr.py' for details.

Type

% python ./compare_svm.py

to compare solvers. The results are stored in the 'logSVM' directory.

Prepare Data Sets for Experiments
=================================

Edit 'gen_data.py' to indicate data sets for generation.

Type

% python ./gen_data.py

The script will download two data sets (news20.binary and rcv1_test.binary)
from LIBSVM Data page. For those data w/o training and testing data sets,
we do a 80/20 split for training and testing. It then stores *.train and
*.test in the 'data' directory.

Note that you need bunzip2, which is called by gen_data.py

Install CDN for Experiments
===========================

% cd liblinear
% make clean all

Install newGLMNET for Experiments
=================================

% cd newGLMNET
% make clean all

Install GLMNET for Experiments
==============================

Download the .tar.gz file from
http://cran.r-project.org/src/contrib/glmnet_1.5.3.tar.gz

Note that the patch only works for version 1.5.3 of GLMNET.
Once GLMNET is upgraded, the link above may be broken.
Please find version 1.5.3 of GLMNET at the CRAN archive
http://cran.r-project.org/src/contrib/Archive/glmnet

% tar xvzf glmnet_1.5.3.tar.gz
% cd glmnet
% patch -p1 < ../patch/GLMNET.patch
% cd ..
% tar cvzf glmnet_1.5.3.tar.gz glmnet
% R CMD INSTALL -l lib glmnet_1.5.3.tar.gz

Note that the following R packages are required: SparseM, Matrix and
e1071. They are available at http://cran.r-project.org/web/packages/.
To install them, the command is similar to the above. For instance,

% R CMD INSTALL -l lib e1071_1.5-24.tar.gz

Generate Figures
================

To generate Figures 1-2, please type

% python figure.py lrb

To generate Figures 3, please type

% python figure.py sh

To generate Figures 4-5, please type

% python figure.py svm

The output figures are stored in the 'figures' directory.
