Jason Rennie's Matlab Code

An Brief Introduction: How to run Fast MMMF

Theory & experiments: Fast Maximum Margin Matrix Factorization for Collaborative Prediction.

Examples of how to use this code for Fast MMMF: weak.m, strong.m. Here's the data to run weak.m: marlin.mat.

For the MMMF obj/grad functions, conjgrad returns a vector of all of the parameters, U, V and theta. Here's how to extract:

U = reshape(v(1:n*p),n,p);
V = reshape(v(n*p+1:n*p+m*p),m,p);
theta = reshape(v(n*p+m*p+1:n*p+m*p+n*(l-1)),n,l-1);

X = U*V';
Y = m3fSoftmax(X,theta);

The Code

• Common Code (which you'll need to run many of the below routines)
  • paramgt.m
  • findNonZeroAlpha.m
  • backtrack.m
  • checkConditions.m
  • saveAlpha.m
  • restoreAlpha.m
  • pround.m
• Optimization Routines
  • Conjugate Gradients: conjgrad.m
  Example usage:

  [w] = conjgrad(randn(d,1),@cgLineSearch,{},@m3fshc,{train,lambda,l},'verbose',1,'tol',1e-6);

• Line Searches
  • For use with conjgrad.m: cgLineSearch.m
• Objective/Gradient Functions (conjgrad.m only)
  • Maximum-Margin Matrix Factorization (MMMF)
    • All-threshold with Smooth Hinge: m3fshc.m (normalized columns: m3fshc_norm.m; requires normCols.m; U,V must be run though normCols)
    • All-threshold with Logistic: m3flogistic.m (normalized columns: m3flogistic_norm.m; requires normCols.m; U,V must be run though normCols)
    • Least Squares: m3fls.m
    • Natural Parameter Multinomial: m3fmultinomial.m
  • MMMF with fixed V (learn U, theta)
    • All-threshold with Smooth Hinge: m3fshcFixedV.m
    • All-threshold with Least Squares: m3flscFixedV.m
    • All-threshold with Logistic: m3flogisticFixedV.m
    • All-threshold with Modified Least Squares: m3fmlsFixedV.m
  • Multiclass Classification
    • Logistic (i.e. softmax): mcclogistic.m
    • Modified Least Squares: mccmls.m
    • Smooth Hinge: mccsh.m
    • Least Squares: mccls.m
    Note: this multiclass code sums loss over all classes (not just the max-loss class, as is more common)
    Note: these codes are designed for a collaborative prediction task (hence the two-dimensional rating/label matrix). For a more traditional multi-class task, assume n=1 so that Y is a row vector containing the (positive, integer) labels.
  • Binary Classification
    • Smooth Hinge Loss: bcsh.m
    • Modifed Least Squares: bcmls.m
    • Least Squares: bcls.m
  • Term Frequency Models
    • The Log-Log Model: loglog.m
  • Standard Statistics Distributions
    • Multivariate Normal: multiNormal.m
• Objective/Gradient/Hessian Functions
  • Regression
    • Least squares: ls.m
• Objective/Gradient Checker
  • Use this to make sure your objective/gradient function is bug free: checkgrad2.m
• Evaluation
  • Mean Absolute Error: mae.m
  • Zero-one Error: zoe.m
  • Maximum Margin Matrix Factorization
    • Use this to calculate predicted labels: m3fSoftmax.m

AFS

addpath '/afs/csail.mit.edu/u/j/jrennie/public_html/matlab'

Other People's Code

• Carl Rasmussen's implementation of Polak-Ribiere Conjugate Gradients.
• Tom Minka's Lightspeed toolbox.
• Tom Minka's Fastfit toolbox.
• Tom Minka's tips on accelerating Matlab.

Other Code

• Naive Bayes
  • Binary Features - makes independent features assumption, only uses presence/absence
  • Count Features - uses Multinomial model
• Classifier Evaluation
  • Breakeven - finds threshold that maximizes some function of the classification errors (use with functions below)
    • Accuracy - number of correctly labeled examples
    • F1-measure - harmonic mean of precision and recall
    • R1-measure - harmonic mean of accuracy rate on positive examples (recall) and accuracy rate on negative examples
• Solving Ax=b and Mx=y for x - the Ax=b code assumes that A is symmetric and positive definite; the Mx=y code is simply an interface to the Ax=b code.
• Sparse Diagonal Matrices.
• Precision-based rounding. Select the number of digits of significance to preserve.
• Safe, fast calculation of log-determinant.