Prediction and Ranking of Highly Popular Web Content

236 Pages·2017·8.43 MB·English

Prediction and Ranking
of Highly Popular Web
D
Content
Nuno Miguel Pereira Moniz
Programa Doutoral em Ciência de Computadores
Departamento de Ciência de Computadores
2017
Orientador
Luís Fernando Rainho Alves Torgo, Professor Associado,
Faculdade de Ciências da Universidade do Porto

Dedicated to
My parents, M´ario and Maria.

Acknowledgments
This thesis is the results of four years of work for which many people contributed, directly
and indirectly.
First and foremost, I would like to thank my supervisor Professor Lu´ıs Torgo for accompa-
nying me in this journey and helping me throughout this process that now culminates with
this thesis. Throughout this bumpy process, he was a key factor in keeping my motivation
high and motivating me to go further and deeper into the areas that were studied, but also
in providing important advices and warnings when concentration and focus were di�cult to
maintain. For that, he has my utmost respect and gratitude.
ToallmycolleaguesinRoom1.70, Ant´onioGon¸calves, JoanaCˆorte-Real, Jo˜aoSantos, Mar-
iana Oliveira, Miguel Areias and V´ıtor Cerqueira, with whom I established good friendships,
a big thank you, for enduring both the happy and the stressful times in the o�ce.
Also, a special acknowledgment to Paula Branco and Rita Ribeiro for allowing me to ”pick
their brain” with multiple ideas, and their availability to discuss many of the doubts that I
had throughout this process.
I would also like to express my gratitude to all my colleagues in LIAAD - INESC Tec,
specially to Professor Al´ıpio Jorge and Joana Dumas, and to Alexandra Ferreira.
A special word of appreciation to Allan Tucker and Stephen Swift from Brunel University
(London, UK), and to Magdalini Eirinaki from San Jose State University (California, USA).
By making it possible for me to spend time at your respective universities, you had a major
impactontheworkthatIwasabletodothroughoutmyPhD.Also,aspecialwordtoRenato
Arau´jo Soeiro, for the conversations and scientific discussions.
To Alex Gomes, B´arbara Silva, Hugo Monteiro, Jo˜ao Carlos, Jo˜ao Mineiro, Jo˜ao Paup´erio,
MarianaGomes,PedroRodriguesandRicardoS´aFerreira. Fortheirpatienceandfriendship;
for providing many of the moments that kept me motivated, but also for providing much
needed moments of distraction.
A special thank you to my family: my parents M´ario and Maria, and my brother Bruno.
7
You are the reason I’m here today.
Finally, to Raquel. For always believing in me, and for providing all the a↵ection and
confidence that I needed.
The work presented in this thesis was funded by the Portuguese Science and Technology
Foundation (FCT) with the PhD grant SFRH/BD/90180/2012, by the ERDF – Euro-
pean Regional Development Fund through the COMPETE 2020 Programme within project
POCI-01-0145-FEDER-006961, and by National Funds through FCT as part of project
UID/EEA/50014/2013.
8
Abstract
This thesis addresses prediction and ranking tasks using web content data. The main
objective is to improve the ability to accurately predict and rank recent and highly popular
content, thus enabling a faster and more precise recommendation of such items. The main
motivation relates to the profusion of online content, and the increasing demand of users
concerning a fast and easy access to relevant content.
To fulfill these tasks an extensive review of previous work is carried out in order to define the
state-of-the-artandtoidentifyimportantresearchopportunities. Asaresult,threeproblems
are identified and addressed in this thesis: (1) the lack of an interpretable and robust
evaluation framework to correctly assess web content popularity prediction models focusing
on highly popular web content; (2) issues concerning proposals of popularity prediction
models and their ability to predict the rare cases of highly popular content; and (3) the need
for recommendation frameworks concerning such items using multi-source data. For each
of these problems novel solutions are proposed and extensively evaluated in comparison to
existing work.
The first problem (1) concerns the evaluation methods commonly used in web content
popularity prediction tasks. According to previous work, the popularity of web content
is best described by a heavy-tail distribution. As such, at any given moment, most of the
content under analysis has a low level of popularity, and a small set of cases has high levels
of popularity. Standard evaluation metrics focus on the average behaviour of the data,
assuming that each case is equally relevant. Given the predictive focus on highly popular
content, it is argued that such assumption may lead to an over-estimation of the models’
predictive accuracy. Therefore, an evaluation framework is proposed, allowing for a robust
interpretation of the prediction models’ ability to accurately forecast highly popular web
content.
Thesecondproblem(2) isrelatedtothefactthatproposalsconcerningwebcontentpopular-
itypredictionmodelsarebasedonstandardlearningapproaches. Thesearecommonlybiased
towards capturing the dynamics of the majority of cases. Given the skewness of web content
popularity data, this may lead to poor accuracy towards under-represented cases of highly
9
popular items. An evaluation with a diverse set of such proposals is carried out, confirming
their issues when learning to predict such items. Also, it is additionally confirmed that the
use of standard evaluation metrics often presents an over-estimated ability to accurately
predict the most popular items. Novel approaches are proposed for the prediction of web
content popularity focusing on accuracy towards highly popular items.
The third and final problem (3) concerns the task of ranking, but also evaluating, web
content by its predicted popularity. Although the task of ranking may be trivial in most
cases, when considering scenarios with multiple sources of data such task is considerably
di�cult. Notwithstanding, ranking tasks and their evaluation in single-source scenarios are
notexemptofissuesconcerningtheabilitytoaccountforhighlypopularcontent. Theability
to rank web content based on models’ predictions is discussed, given an extensive evaluation
in both single-source and multi-source scenarios.
Each of these problems is evaluated using real-world data concerning online news feeds from
both o�cial and social media sources. This type of web content provides a di�cult setting
for the early and accurate prediction of highly popular items, given their short lifespan.
Experimental evaluations show that the approaches proposed in this thesis concerning the
predictionandrankingofhighlypopularcontentobtainedencouragingresultsdemonstrating
a significant advantage in comparison to state-of-the-art work.
10
Contents
List of Tables 16
List of Figures 23
1 Introduction 25
1.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.2 Context and Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.3 Motivation and Main Contributions . . . . . . . . . . . . . . . . . . . . . . . 27
1.4 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.5 Bibliographic Note . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2 Literature Review 31
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2 Social Media and Web Content . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.3 Popularity Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.3.1 Classification Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.3.2 Regression Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.3.3 Time Series Forecasting Tasks. . . . . . . . . . . . . . . . . . . . . . . 46
2.3.4 Other Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
11
2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3 The Case of Online News Feeds 61
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.2 Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.2.1 Single-Source Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.2.2 Multi-Source Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4 Learning with Imbalanced Domains 91
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.2 Imbalanced Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.2.1 Utility-based Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.3 Utility-Based Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.3.1 Relevance Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.3.2 Utility Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.4 Rule-Based Utility Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.5 Evaluation Metrics for Imbalanced Learning . . . . . . . . . . . . . . . . . . . 106
4.6 Utility-Based Evaluation Framework . . . . . . . . . . . . . . . . . . . . . . . 108
4.7 Experimental Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.7.1 Materials and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.7.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.7.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5 Popularity Prediction Models 125
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
12