We provide this MATLAB/Octave code. Please
The usage is
metalevel_cv(label_vector, instance_matrix, size_of_groups [, 'libsvm_options']);Assume you have the following data
users labels features 1 1 0.5 1 -0.4 1 1 ... 1 1 ... 2 -1 ... 2 -1 ... 3 1 ... 3 1 ... 3 1 ... 4 -1 ... 4 -1 ... 4 -1 ... 4 -1 ... ...Then the variable size_of_groups is a vector of
3 2 3 4 ...to indicate the number of instances of each user. You can download the example file heart_scale.mat and run the following commands
>> load heart_scale.mat >> who Your variables are: logindex train_x train_y >> metalevel_cv(train_y, train_x, logindex, '-v 3 -e 0.01');to conduct three-fold CV. We pass the '-e 0.01' option to LIBSVM. Note that we assume instances of the same group have been put together in the variable train_x.
The default number of CV folds is five. Currently, we do not take the size of each group into consideration. Therefore, data folds may not be very balanced.
To ensure that the same split of data is used in conducting parameter selection, we fix the random seed internally. To use random seed by time, comment the following line in the file.
To select parameters by grid.py of LIBSVM, we provide a Python code metalevel_cv.py to act like svm-train of LIBSVM. For example, grid.py calls
> ./svm-train -v 5 -c 4 -g 0.25 your_data Cross Validation Accuracy = 90.1122%in trying one parameter set. We replace svm-train by this python code:
> ./metalevel_cv.py -v 5 -c 4 -g 0.25 your_data Cross Validation Accuracy = 84.1234%
Internally metalevel_cv.py calls metalevel_cv.m by Octave. We consider Octave rather than MATLAB because grid.py will launch many CV jobs in parallel and you may not have enough MATLAB licenses.
Because the data must contain meta information, we make the following requirements
> pwd > /home/you/data_directory > ls ../libsvm-3.17/tools/grid.py ../libsvm-3.17/tools/grid.py > octave octave> load your_data.mat octave> who train_y train_x logindex octave> quit > ../libsvm-3.17/tools/grid.py -log2c -2,3,2 -log2g -3,1,2 -m 500 -v 3 -svmtrain ../libsvm-3.17/matlab/metalevel_cv.py your_data.mat