This directory includes sources used in the following paper:

Ching-pei Lee, Po-Wei Wang, Weizhu Chen, and Chih-Jen Lin,
Limited-memory Common-directions Method for Distributed Optimization
and its Application on Empirical Risk Minimization, 2016.  You will be
able to regenerate experiment results in the paper. However, results
may be slightly different due to the randomness, the CPU speed, and
the load of your computer.

Please cite the above article if you find this tool useful. Please
also read the COPYRIGHT before using this tool.


System Requirement
==================
This experiment is supposed to be run on UNIX machines. The following
commands are required:
- mpi (mpicxx, mpirun)
- UNIX commands (mv, ln, cp, cat, patch, etc)
- bash
- wget
- make
- python2.6 or newer versions except python 3.x.
- MATLAB


Introduction
============
You can choose solvers/data sets/regularization parameter for
comparisons. See descriptions in subsequent sections.

Distributed Environment
=======================
Please specify the ip or domain name of the machines being used in the
file 'machinelist', one machine per line. If multiple cores per
machine are used, then simply duplicate the ip or domain name for
#cores times.

Data Preparation
================
Please run data/get_data.sh to obtain and pre-process the data sets
for distributed environments.
The script 'split.py' for spliting data into segments and send the
segments to different machines is modified from the tools in
MPI LIBLINEAR.

Compare Different Distributed Solvers
=====================================
Edit 'compare_methods.py' to indicate solvers, data, and regularization
parameters for comparison. Remove the solvers, data sets, and loss parameters
that you are not interested in. For example, change

data = ['epsilon_normalized','kddb','news20.binary','rcv1_test.binary','url_combined','webspam_wc_normalized_trigram.svm']

to

data = ['epsilon_normalized','kddb']

In the same way, you can exclude some solvers or some loss parameters from comparison. For
example, change

methodlist = ['VL-BFGS','TRON','L-COMMDIR-BFGS','L-COMMDIR-STEP','L-COMMDIR-GRAD','NEWTON']

to

methodlist = ['VL-BFGS','TRON','L-COMMDIR-BFGS']

After deciding data sets and solvers, you must prepare the distributed
environment and the data sets. Please see Sections 'Distributed
Environment' and 'Data Preparation' for more details.

Type

% python ./compare_methods.py

to compare solvers. The results are stored in the 'figures/' directory.

The files compare.m will automatically executed by this python script
to generate the figures after the solver logs are available. The file
const.mat contains some stored constants including the optimal
objective values and the default stopping points that will be used in
figure plotting.
