T.-F. Wu, C.-J. Lin, and R. C. Weng. Probability Estimates for Multi-class Classification by Pairwise Coupling. Journal of Machine Learning Research, 5(2004), 975-1005, the authors have to evaluate different probability estimates. In Section 7 you will see that two criteria taught in this class are used. In Section 6, however, only MSE is used. Your job is to use the log loss for the experiment in Section 6.
To do this homework, you first must roughly understand the paper. Code for doing experiments in section 6 are in ~cjlin/latex/papers/03-3/newfig/fig1a.m. You then modify it for your experiments.
Basically you would like to try two formulas. One with p assumed known and another unknown. The first one will be
-p_1 log p_1' - ... - p_k log p_k',where p (p') is the true (predicted) probability vector. The second is
-log p'_{c},
where c is the true class label.