Recommendation System For Sport Videos

139 Pages·2017·2.79 MB·English

by  Simen Røste Odden

Recommendation System for Sports
Videos

Choosing Recommendation System Approach in
a Domain with Limited User Interaction Data

Simen Røste Odden

Master’s Thesis
Informatics: Programming and Networks
60 credits

Department of Informatics
The Faculty of Mathematics and Natural Sciences

UNIVERSITY OF OSLO

May 2017
II
Recommendation System for Sports
Videos
Choosing Recommendation System Approach in a Domain with Limited User Interaction
Data
III
© Simen Røste Odden
2017
Recommendation System for Sports Videos
Simen Røste Odden
http://www.duo.uio.no/
Print: Reprosentralen, Universitetet i Oslo
IV
Abstract
Recommendation systems help users find interesting items and reduce the information
overflow problem on websites. Much research has been conducted on such systems the last
decades, and there exist several recommendation approaches with different strengths and
weaknesses. In this thesis, we investigate which recommendation approach or combination
of approaches that can recommend the most interesting content for each individual user of
the sports video application Forzify.
We use previous research and literature to find the approaches that are best suited for the
data gathered in Forzify and the features wanted in a new recommendation system. Three
approaches turn out to be the best suited: item-based collaborative filtering, model-based
collaborative filtering and content-based filtering. We implement one algorithm from each
of these approaches and a baseline algorithm. The four algorithms are evaluated in an
offline evaluation to find out which of the approaches that performs best in terms of
recommendation accuracy, both for new and old users, and scalability. As Forzify so far
has gathered limited user interaction data, we have to test the approaches on other datasets.
To increase the validity, we investigate the accuracy of the algorithms in datasets from
different domains. This makes it possible to check whether it is consistencies in the
accuracy of the algorithms across the domains.
From our evaluation of the accuracy of the algorithms, we can see both differences and
similarities across the domains. The accuracy of the different algorithms is more even in
some domains than in others, and some domains generally have higher accuracy, but there
is a tendency that the algorithms performing well in one domain also do so in the other
domains. Due to this cross-domain consistency, our results provide a good basis for
choosing the best approach for Forzify. We conclude that the model-based collaborative
filtering approach is the best choice for Forzify. It gives accurate recommendations for
both new and old users across the datasets, and it scales well for larger numbers of users
and items.

V
VI
Acknowledgments
I would like to thank my supervisor, Pål Halvorsen, for valuable guidance and discussions,
and for giving me the opportunity to work on this research topic. I also want to thank
Forzify, who have made this thesis possible.
I want to thank my mother, Sigrun, my father, Odd-Erik, and my sister, Oda, for good
support. In addition, I will thank all my friends and fellow students.
Oslo, April, 2017
Simen Røste Odden

VII
VIII
Table of contents
1 Introduction ................................................................................................................... 1
1.1 Motivation and background .................................................................................... 1
1.2 Problem statement .................................................................................................. 3
1.3 Limitations .............................................................................................................. 4
1.4 Research method ..................................................................................................... 5
1.5 Main contributions .................................................................................................. 6
1.6 Outline .................................................................................................................... 6
2 Recommendation systems ............................................................................................. 8
2.1 Recommendation systems explored ....................................................................... 8
2.2 Recommendation system approaches ................................................................... 11
2.2.1 Collaborative filtering ................................................................................... 11
2.2.2 Content-based filtering .................................................................................. 15
2.2.3 Demographic-based ....................................................................................... 16
2.2.4 Knowledge-based .......................................................................................... 17
2.2.5 Community-based ......................................................................................... 19
2.2.6 Hybrid ............................................................................................................ 20
2.3 Recommendation systems in practice ................................................................... 20
2.3.1 Amazon ......................................................................................................... 20
2.3.2 Netflix ............................................................................................................ 21
2.3.3 Facebook ....................................................................................................... 22
2.3.4 YouTube ........................................................................................................ 23
2.4 Comparison of approaches ................................................................................... 25
2.5 How to evaluate recommendation systems........................................................... 29
2.5.1 Experimental settings .................................................................................... 29
2.5.2 Dimensions and metrics ................................................................................ 31
2.5.3 Measuring accuracy ....................................................................................... 33
2.5.4 Datasets ......................................................................................................... 37
2.6 Summary ............................................................................................................... 41
3 Case: Forzify ............................................................................................................... 43
3.1 Context .................................................................................................................. 43
3.2 Recommendations in Forzify ................................................................................ 44
IX
3.3 Data in Forzify ...................................................................................................... 47
3.4 Features wanted in the recommendation system .................................................. 49
3.5 Discussion of suitable approaches ........................................................................ 50
3.6 Summary ............................................................................................................... 51
4 Implementation of a recommendation system............................................................. 53
4.1 Recommendation frameworks and libraries ......................................................... 53
4.1.1 Mahout .......................................................................................................... 53
4.1.2 LensKit .......................................................................................................... 54
4.1.3 MyMediaLite ................................................................................................. 54
4.1.4 Spark MLlib .................................................................................................. 55
4.1.5 Comparison of frameworks ........................................................................... 55
4.2 Algorithms ............................................................................................................ 57
4.2.1 Item-based collaborative filtering ................................................................. 58
4.2.2 Model-based collaborative filtering .............................................................. 63
4.2.3 Content-based filtering .................................................................................. 68
4.2.4 Popularity baseline ........................................................................................ 71
4.3 Summary ............................................................................................................... 72
5 Evaluation .................................................................................................................... 74
5.1 Experimental design ............................................................................................. 74
5.1.1 Experimental setting and metrics .................................................................. 74
5.1.2 Datasets ......................................................................................................... 80
5.1.3 Implementation of an evaluation framework ................................................ 85
5.2 Results .................................................................................................................. 86
5.2.1 General accuracy ........................................................................................... 86
5.2.2 Accuracy for new users ................................................................................. 94
5.2.3 Scalability .................................................................................................... 100
5.3 Discussion of cross-domain accuracy ................................................................. 105
5.4 Discussion of best algorithms for Forzify .......................................................... 107
5.5 Summary ............................................................................................................. 110
6 Conclusion ................................................................................................................. 113
6.1 Research questions ............................................................................................. 113
6.1.1 Q1: Approaches suited for Forzify according to literature.......................... 113
6.1.2 Q2: Differences in accuracy across domains .............................................. 114
X