Scribd
Upload
58 views

Matrix Factorization Techniques For Recommender Systems: Collaborative Filtering

This document discusses matrix factorization techniques for recommender systems. Matrix factorization involves finding lower-dimensional factor vectors that describe users and items in a way that can be used to predict unknown ratings. It addresses overfitting through regularization and includes additional parameters for biases and temporal effects to improve accuracy. The document evaluates previous research on matrix factorization and discusses open questions around applying it to datasets with different item types and explicit dependencies between items.

Uploaded by

Shashank Baravani
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
58 views

Matrix Factorization Techniques For Recommender Systems: Collaborative Filtering

This document discusses matrix factorization techniques for recommender systems. Matrix factorization involves finding lower-dimensional factor vectors that describe users and items in a way that can be used to predict unknown ratings. It addresses overfitting through regularization and includes additional parameters for biases and temporal effects to improve accuracy. The document evaluates previous research on matrix factorization and discusses open questions around applying it to datasets with different item types and explicit dependencies between items.

Uploaded by

Shashank Baravani
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Matrix Factorization Techniques

For Recommender Systems

Collaborative Filtering

Markus Freitag, Jan-Felix Schwarz 28 April 2011


Agenda

1.  Paper Backgrounds


2.  Latent Factor Models
3.  Overfitting & Regularization
4.  Learning Algorithms
5.  Biases & Temporal Dynamics
6.  Paper Evaluation
7.  Application For KDD Cup
8.  Discussion

Matrix Factorization Techniques For Recommender Systems


Paper Backgrounds

Yehuda Koren, Yahoo Research

Robert Bell and Chris Volinsky,


AT&T Labs-Research

Paper published in August 2009

Authors won the grand Netflix Prize


in September 2009

Matrix Factorization Techniques For Recommender Systems


Latent Factor Models

■  Find features that describe the characteristics of rated objects


■  Item characteristics and user preferences are described with
numerical factor values
■  Assumption: Ratings can be inferred from a model put together
from a smaller number of parameters

0
Drama | Comedy

Matrix Factorization Techniques For Recommender Systems


Latent Factor Models

■  Items and users are associated with a factor vector


■  Dot product captures the user’s estimated interest in the item:

■  Challenge: How to compute a mapping of items and users to


factor vectors?

■  Approaches:
□  Singular Value Decomposition (SVD)
□  Matrix Factorization

Matrix Factorization Techniques For Recommender Systems


Singular Value Decomposition

6 User Feature Matrix (F x N)

f1 1 -4 1
Rating Matrix (N x M)
f2 -2 0 -3

5 3 5 f3 0 -5 1

4 2 1
Movie Feature Matrix (F x M)
0 3 3

f1 -1 0 -2

f2 4 -4 1

f3 0 2 2
Matrix Factorization Techniques For Recommender Systems
Singular Value Decomposition

.
=

Matrix Factorization Techniques For Recommender Systems


SVD - Problems

■  Conventional SVD is undefined for incomplete matrices!

■  Imputation to fill in missing values


□  Increases the amount of data
□  “SVD of ginormous matrices is... well, no fun“ (Simon Funk)

■  We need an approach that can simply ignore missing ratings

Matrix Factorization Techniques For Recommender Systems


Matrix Factorization

: known rating of user u for item i


remember:
predicted rating

Matrix Factorization Techniques For Recommender Systems


Matrix Factorization - Overfitting

10

A model is built to represent the training data – not to reproduce


the training data.

Matrix Factorization Techniques For Recommender Systems


Matrix Factorization - Regularization

11

Idea: penalize complexity

: constant to control the extend of regularization


 determined by cross-validation

Matrix Factorization Techniques For Recommender Systems


Learning Algorithms

■  Stochastic gradient descent


□  Modification of parameters (qi, pu) relative to prediction error
□  Recommended algorithm

■  Alternating least squares


□  Allows massive parallelization
□  Better for densely filled matrices

Matrix Factorization Techniques For Recommender Systems


Learning Algorithms

■  Calculation of the prediction error


□  Error = actual rating – predicted rating
□ 

■  Modification
□  By magnitude proportional to γ
□  In the opposite direction of the gradient
□ 

Matrix Factorization Techniques For Recommender Systems


Biases

■  Item or user specific rating variations are called biases


■  Example:
□  Alice rates no movie with more than 2 (out of 5)
□  Movie X is hyped and rated with 5 only
■  Matrix factorization allows modeling of biases

■  Including bias parameters in the prediction:

Matrix Factorization Techniques For Recommender Systems


Temporal Dynamics

■  Ratings may be affected by temporal effects


□  Popularity of an item may change
□  User’s identity and preferences may change
■  Modeling temporal affects can improve accuracy significantly

■  Rating predictions as a function of time:

Matrix Factorization Techniques For Recommender Systems


Paper Evaluation

■  High-level overview of matrix factorization techniques

■  Mathematical foundations are pointed out but not elaborated

■  Many useful references to related work

■  Authors do not reveal their secret implementation tidbits

Matrix Factorization Techniques For Recommender Systems


Application For KDD Cup

■  Different item types


□  We must assume different prediction models!
□  We have explicit dependencies between items

■  How to apply Matrix Factorization to the KDD data set?


□  Segment training data in separate sets for each type
□  Consider ratings for dependent items to make a prediction

Matrix Factorization Techniques For Recommender Systems


Application For KDD Cup - Hypotheses

■  Users change their taste in music

■  Users tend to rate new songs better


□  Users get tired of songs

■  Some artists and albums are hyped for a while

■  Evergreens
□  Loved by many, but maybe also hated by some

■  ...

Matrix Factorization Techniques For Recommender Systems


References

[1] C. Volinsky et al.: “Matrix Factorization Techniques for Recommender Systems“ In:
IEEE Computer, Vol. 42 (2009) , pp. 30-37 .

[2] Simon Funk on the Stochastic Gradient Descent algorithm: http://sifter.org/~simon/


journal/20061211.html (Dec 2006)

[3] Y.Koren et al.: “Collaborative filtering with temporal dynamics“ In: Proceedings of
the 15th ACM SIGKDD international conference on Knowledge discovery and data
mining (2009), pp. 447-456.

[4] Chih-Chao Ma: “A Guide to Singular Value Decomposition for Collaborative Filtering“
In: csientuedutw (2008)

[5] Y.F. Hu, Y. Koren, C. Volinsky: “Collaborative Filtering for Implicit Feedback
Datasets“ In: Proc. IEEE Int‘l Conf. Data Mining (2008), pp. 263-372

Matrix Factorization Techniques For Recommender Systems


Discussion

Summary
■  Matrix factorization is a promising approach for collaborative
filtering
■  Factor vectors are learned by minimizing the RSME
■  Regularization to prevent overfitting
■  Addition of bias parameters and temporal dynamics further
improve accuracy

Outlook
■  Develop strategies for applying matrix factorization on our data
set with different item types
■  Make use of the available dependencies between items
■  Explorer biases and rating behaviors specific for our music domain
Matrix Factorization Techniques For Recommender Systems

You might also like