Collaborative Filtering without Explicit Feedbacks for Digital Recorders

0.39 MB·English

by  Alessandro Basso

Collaborative Filtering without Explicit Feedbacks for
Digital Recorders
Alessandro Basso, Marco Milanesio, André Panisson, Giancarlo Ruffo
DipartimentodiInformatica
UniversitàdegliStudidiTorino
Torino,Italy
{basso,milane,panisson,ruffo}@di.unito.it
1 ABSTRACT phonesandsoon). Forexample,Orb1 isafreewarestream-
1 Recommendationisusuallyreducedtoapredictionproblem ingsoftware thatenablesuserstoaccess remotely theirme-
0 over the function r(ua,ei) that returns the expected rating diaviaInternet. Inthispaper,whilefocusingontherecord-
2
ofelementeiforuserua. IntheIPTVdomain,wedealwith ingofradioandTVbroadcasts,wekeepinmindthegeneral

n an environment where the definitions of all the parameters media sharing domain.
a involvedinthisfunction(i.e., userprofiles,feedbackratings
J and elements) arecontroversial. To ourknowledge,thispa- Recordingandpersonalizationischangingthewayproviders

7 perrepresentsthefirstattempttoruncollaborativefiltering insert advertisements during content delivery. The emer-
1 algorithmswithoutinnerassumptions: westartouranalysis genceofnewbehaviorsandcontentconsumingtrendsisforc-
from anunstructuredset ofrecordings, beforeperforming a ing advertisers to look for new ways (e.g., a pullmodel) for

] data pre-processing phase in order to extract useful infor- spreadingtheircommercials,tryingtoavoidthefast-forward
R
mation. Hence, we experiment with a real Digital Video of pre-recordedvideostoskip commercials. Inthisdomain,
I RecordersystemwhereEPGhavenotbeenprovidedtothe thus, recommender systems [1] are becoming worthwhile as
.
s userforselectingeventtimingsandwhereexplicitfeedbacks retention tools for customers. When a user is satisfied with
c were not collected. suggestions, she dedicates more attention to the proposed
[
links and references. Even if some recommendation engine
1 Categories andSubject Descriptors has been tailored for DVRs (e.g., Neptuny’sContent Wise,
v H.3.3[InformationStorageandRetrieval]: Information ReignSoft’sImpress),acomprehensiveanalysisofthepecu-
1 SearchandRetrieval—Informationfiltering;J.4[Computer liarities of thisdomain is still missing.
4 Applications]: Social AndBehavioral Sciences
3 First of all, usage data collection is subject to serious pri-
3 vacyconcerns. Usersarenotwillingtoloosecontrolofdata
General Terms
. they produce while taking advantage of recommendations
1 Algorithms, Experimentation,Human Factors.
andpersonalized information. Moreover,whenasharedde-
0
vice is used (e.g., the television) also family control issues
1 Keywords
arise,anditcanbedifficulttoprovidepersonalizedinforma-
1
: Digital Video Recorders, TV Broadcasts, Recommendation tion. Even for such reasons, costumers protect themselves
v Systems, Collaborative Algorithms, Implicit Data behind fictitiouson line identities, and this is an important
i challenge for many recommendation algorithms, that need
X
1. INTRODUCTIONANDCONTEXT user profiles to increase their accuracy.
r
a InthewidecontextofIPTVservices,DigitalVideoRecorders Another important problem in the IPTV domain ([5, 6]) is
(DVR) (a.k.a. Personal Video Recorder (PVR)), are hard- that, differently from other domains where recommenders
wareorsoftwaredevicesthatrecorddigitalvideotoamem- can learn from explicit user ratings over content, in DVR
ory medium, for a further access (e.g., stand-alone set-top- systems we need to operate according to implicit feedbacks,
boxes(STB),portablemediaplayers,andPCbasedDVRs). such as recordings of a given event, downloading of pre-
One of the most important exploitations of the DVRs is recordedvideos,or-whenpossible-monitoringifthevideo
the media sharing: users may want to access to all their has been effectively watched by theuser.
personal media resources (e.g., pictures, podcasts, TV pro- Athirdrelevantproblem,underestimatedbypreviousworks
grams) from any of their own devices (e.g., laptops, smart- on this subject, is the difficulty of discriminating items.
DVRs usually provide Electronic Program Guides (EPGs)
to help users. Unfortunately, in a wider context, EPGs are
not reliable to identify recommendable elements: TV chan-
nels may not respect schedules, media and podcasts avail-
able on the Web do not use a common schedule’s format,
usersmaybeinterestedonlyinsinglepartsofanevent,and
thus they will set up their own timings over a given chan-
nel. In opposition with VoD,content description is difficult
1http://www.orb.com
in the TV domain. Broadcasts are announced, but often toeveryrecordedtransmissionisthereforedemandedtosin-
lack of structured meta-information: thus, collaborative fil- gleusers. Sinceitistheirprimaryinteresttomakesurethat
teringisusuallypreferredtocontent-basedrecommendation theinformationinsertedintheFaucetPVRareasmuchpre-
algorithms, even if itsexecution is not straightforward. cise as possible, we can consider such data reliable enough
to be used in the recommendation process. Furthermore, a
Recommendationisusuallyreducedtoapredictionproblem numberofinferencescanbededucedfrom theuseractivity,
over the function r(ua,ei) that returns the expected rating andconsideringalsothegoodpopularityofthesystem,also
ofelementei foruserua. Asobservedabove,wearedealing numerical processing is statistically reliable.
withanenvironmentwherethedefinitionsofalltheparam-
eters involved in this function (i.e., user profiles, feedback The Faucet PVR involves three different steps to be taken
ratings and elements) are controversial. To our knowledge, byanuserwhensheisinterestedinrecordinganevent: the
this is the first attempt to run collaborative filtering algo- parameters setting, the execution and the downloading of
rithms without innerassumptions. therecordeditem. Allthreestepsareperformedindifferent
moments, in the aforementioned order. In the parameters
Inthispaper,westartouranalysisfromanunstructuredset setting step, the user chooses a channel, periodicity, name,
ofrecordings,beforeperformingadatapre-processingphase startingandendingtimes. Thisstepmustbedonebeforethe
in order to extract useful information. Hence, in Section 2 beginningoftheprogram. Theexecutionphasestartsatthe
wepresentarealsystemwithoutEPGswhereexplicitfeed- startingtimeandfinishesat endingtime. Thedownloading
backs are not collected. Then, we describe the procedure step is available only after the recording finishes.
for extracting meaningful information from the large and
unstructured amount of provided data (Section 3) and the As mentioned above, an event can be periodic: if the user
analyzed recommendation algorithms (Section 4). Finally, wishes, the system records the desired event in regular in-
theevaluation of the chosen algorithms is presented in Sec- tervals. Theseintervalscanbeofaweekoraday,andinthe
tion 5, before drawing conclusions. case of a daily event,theuser can choose to skip weekends.
Eventsclassified asnon-periodichaveabsolutestartingand
endingtimes. However,inthecaseofperiodicevents,start-
2. THEFAUCET PVRENVIRONMENT
ing and ending do not represent absolute times, but rather
Our analysis is based on real data generated by the Faucet a weekday and daytime (in the case of weekly events) or
PVR system, integrated in a web-based podcasting service just a daytime (in the case of daily events). Also, in the
named VCast2. Faucet allows users to record their favorite case of periodic events, thereis one parameter setting step,
(Italian) TV and Radioprograms, and to furtherdownload but the execution step can occur an undefined number of
them into their devices (e.g., iPod, PC, notebook) [4]. The times. Aftereach execution step,adownload referring to it
user can set up her own programming and see or download is made available. The system limits the number of accu-
her recordings by the use of a simple web interface. Bring- mulatedrecordingsto3inordertosaveresources(onlythe
ing the ability to record and group into a single feed public last 3 executions are available to download).
and private channels (such as radio and TV recorded pro-
grams), Faucet PVR offers a single framework for creating Thefactthatthedatasetincludesinformationaboutperiod-
and aggregating personal podcast compilations. icity impliessomeissues inproperly determiningtheevents
broadcastedonTVandradiofromtheamountofrecordings
TheFaucetPVRproducesaveryrichanddynamicdataset3, madebyusers. Ontheotherside,itdecreasesthecomplex-
populatedbyrealusersexpressingtheirpreferencesthrough ity of calculating recommendations, resulting in an overall
the recorded programs. Such a context, however, is char- improvement in their novelty. In fact, without taking into
acterized by a number of constraints which we had to deal account the periodicity, as in [9], the recommender has to
with, in order to be able to perform the analysis on the explicitlyignoreperiodicelementsrecentlyseenbytheuser,
recommendation algorithms. Inparticular,theintrinsicdy- in order to provide a more valuable and accurate recom-
namism and variability of the recordings, as well as the mendation. Inourdomain, astheperiodicity isanintrinsic
lack of any permanent event, require that a series of pre- feature of therecommendable items, we donot havesuch a
processing steps have to be undertaken prior to be able to constraint, being theseelements automatically excluded.
apply any recommender.
The goal of a recommendation system in the PVR context
A noticeable property characterizing the context in which istosuggestapersonalizedsetoftransmissionstotheusers.
we operate is the lack of a well defined programming for However,datacomingfromtheFaucetPVRarenotimmedi-
recorded contents. Despite several EPG sources do exist, atelyusabletoidentifyeventssuchasthetransmissions,but
wecannotconsiderthemreliableenoughtobeusedforex- assumetheformofunstructuredinformation,whichhaveto
tracting the input information of the recommender engine. be properly processed. In particular, let T be the set of
Therefore, we decided to opt for a different approach. Ex- transmissionsduringadayandti beaspecifictransmission
ptmoloitditteeinfidgnoeantbthhoetetmommoasjtuorpreTalepVvpasrnoatancpdhr,roapwdeeirotrsie.elsyoofntuhseeresv’eknntoswtlreadngse- dabtarottiaiomdnceaesenttegi.dinTeo,hneacnsh,awtniencllealnacstbi∀e,tsdti∈arertcTitnl.ygOuatsnetditmhineetbchtoienatrrenacdroyme,nmddaeinntga-
More precisely, the task of defining the parameters related collectedbytheFaucetsystem(i.e.,users’recordings)differ
2http://www.vcast.it/ fromtiinthesensethattheydefineasetRofseveralevents
3The Vcast dataset is publicly available at: riwithatemporalvalidity,eachreferringtoaspecificevent.
However, recordings with different timings may refer to the
http://secnet.di.unito.it/vcast
samebroadcast: giventhepairri,rj ∈R,theymayreferto representing a list of predefined channels, and a periodicity
thesametransmissionevenifri6=rj. Clearly,thisproperty p∈{no−repeat,weekly,daily,mon−fri,mon−sat},rep-
does not hold with discrete and well defined events such as resenting all the possible periodicities allowed in the PVR
thetransmissions. system. Besides, the user is required to annotate his/her
recording with a (possibly) meaningful title.
As well as we can not exploit any EPG to identify the
recorded contents,we can not even rely on any information LetR={r1,r2,...,rm}bethesetofthebroadcastedrecorded
about the specific content of each recording. Indeed, the programs. EachelementinR(arecording)isatupleri=<
Faucet PVR does not provide any reference to the type of ui,ci,pi,ti,bi,fi > set by a user ui ∈ U who recorded on
transmissionrecorded(e.g.,sportprogram,news,orcomedy- the channel ci ∈ C with periodicity pi ∈ P a program ti-
movie), nor we can rely on information inserted by users in tled ti with start time bi and end time fi. Thus, we can
title field, as the insertion of titles and annotations is com- assume that there exists a function mapping every user to
pletelyfree,andthisresultsinverydiversifiedand,possibly, herrecordings.
incorrect descriptions.
The set R is first processed by means of clustering; then,
A further implication due to the lack of information about aggregation and collapsing are carried out in sequence on
thecontentoftherecordingsistheimpossibilityofapplying theoutputoftheclustering. Thethreephasesaredescribed
content based recommenders, which focus on the descrip- in thefollowing.
tion of item and user profiles in order to recommend items
[14]. On the contrary, our approach relies only on the be- Clustering. Dueto thelack of information about thecon-
havior and characteristics of large interconnected networks, tent of each recording, they are clustered wrt the channel,
by exploiting relations between their users. Each specific the periodicity and the difference between timings. Specifi-
user follows her personal behavioral pattern in the usage of cally, ∀ri,rj ∈R|cri =crj ∧pri =prj we havethat
the Faucet PVR, and we can investigate these patterns to
computea similarity among users. ri rj iff |bri −brj|<δb∧|fri −frj|<δf,
]
As a final observation, broadcasting is characterized by the where is the clustering function and δb,δf determine the
expiration ofsomeevents: wecansuggesttheusertorecord maximumclusteringdistanceforthestartandendtimes,re-
U
onlyfuturebroadcasts,andevenifsomeshowsareserialized, spectively. The identified clusters contain recordings equal
the recording of the single episode should be programmed in the channel and periodicity, and similar on the timing.
in advance. This phenomenon is (partially) due to copy- The recording that minimizes the intra-cluster timing dis-
rightmanagement,sincethecontentproviderarenotwilling tances is elected as the centroid of the cluster. At the end
to authorize service providers to store previously recorded of the clustering, each cluster identifiesan event.
event for further distribution. Nevertheless, recording of
a broadcast is still allowed, because it is seen as a single Aggregation. AstheFaucetplatformproducesnewrecord-
useractivity. Asaconsequence,wehavetodeal(also) with ingswithahourlyfrequency,weperformtheclusteringonce
volatile content, and this differs very much with the VoD ahourobtainingaset of newlygenerated events. Afurther
domain, that hasbeen exhaustivelyexplored in thecontext step is then required to possibly aggregate similar events,
of recommendation systems. i.e., the new one with those previously created. Such an
operation is performed as follows: (1) we compare each ele-
3. DATAEXTRACTION mentgenerated with theclusteringwith theexistingevents
wrt channel, periodicity and timings; (2) if the timings are
Due to the specific domain, we are required to perform a
similar,wecorrectthepropertiesofexistingeventswiththe
pre-processofthedataobtainedfromtheFaucetPVRprior
valuesof thenewly created ones. Thelist of theusersasso-
tobeabletousesuchinformationasinputforarecommen-
ciated to theeventis updated accordingly.
dationalgorithm. ThisisneededbecausetheFaucetsystem
generates a set of recordings in the continuous domain of
timings,whilearecommendersystemrequirestooperatein Collapsing. Similar discrete elements, i.e. with the same
thediscrete domain of events. channelandperiodicitybuttimingswithinafixedrange,are
merged into a single event. All features of the new events
Asaconsequence,thefirstgoal that wehavetoaccomplish are computed by means of the values of the collapsed dis-
is the identification of the broadcasted transmissions from creteelements. Thisoperation isrequiredbasically because
theamount of unstructureddataresulting from therecord- eventscan becreated in subsequentmoments, byaggregat-
ing process. This is a multi-step procedure, whose aim is ing recordings referring to the same broadcasted transmis-
to identify a set of discrete elements as the representatives sions. Due to the high variability of the timings, especially
of the broadcasted events. Basically, a discrete element is when a new transmission appears, such events slowly and
obtainedastheresultoftheaggregation ofseveraldifferent independently converge to more stable timeframes, deter-
recordings. Apreliminaryinvestigationontheextractionof mining theneed of collapsing them into single events.
eventsfrom recordings is given in [2].
As a result of the whole process, we obtain a number of
Let U = {u ,u ,...,u } be the set of distinct users in the events,each beinga tupledefinedas follows:
1 2 k
Faucetplatform. EachuserinsetU hasrecordedsomepro-
grams in the past and scheduled some for the future. To ej =<{uej},cj,tlj,bj,fj,pj >
schedule a program, a user must choose a channel c ∈ C, where:
•{uej} is the list of users who set a recording referring recommendation algorithm is to suggest items specifically
to that event; tailored to theuser’s preferences.
•cj is thechannel;
•tlj is thetitle chosen among thosegiven byusers; 4.1 Algorithms Overview
•bj andfj arethethestartingandendingtimesrespec- Twowell-knownrecommendationtechniquesareconsidered
tively;
in this work: (1) the memory based collaborative filtering
•pj ∈{no−repeat,weekly,daily,mon−fri,mon−sat} approach named k-Nearest Neighbors (kNN) [12]; (2) the
is theperiodicity.
model based approach based on theSVD transform [13].
Exploitingthebasicideaofthenearest neighbors approach,
weapplybothvariantsofthekNNalgorithm: theuser-based
In Figure1,we can observethebehaviorof thesystem in a
one [8], by identifying users interested in similar contents;
one year timeframe, i.e., from June 2008 to June 2009, wrt
andtheitem-basedapproach[7],byfocusingonitemsshared
the numberof users, eventsand recordings. As thenumber
by two or more users. In addition, we also analyze theper-
of active recordings and events (b) tends to increase over
formanceofavariantoftheSVDtechniquebasedonimplicit
time, the number of users follows a different, less constant,
ratings, presentedin [9].
trend. Specifically, we can notice a considerable increase in
the number of users in the system between November 2008
and March 2009. Such a happening implies a consequent User-based kNN. In the user-based kNN algorithm, the
raise in the number of recordings, due to the augmented weight of an element ei for an user uk can bedefinedas:
activity in the system. Analogously, the number of events
generated by the aggregations of the recordings grows up, w(uk,ei)= r(ua,ei)·c(uk,ua), (1)
although less noticeably if compared totherecordings. ua∈XN(uk)
50k ((ab)) 112400kk ((ab)) where r(ua,ei)=(cid:26) 01 iiff eeii ∈∈/ EEuuaa
40k
mber of Users2300kk mber of Events1860000kkk tfiEsehrueteahnteitospnst-iehmNiegihslanebrteoiitrogyhfhoebfoluoednrmscoetfoniortundssseerrreaecudroekrb,ddyeliidsmcubuistyesesrduedssbiemyriniuclaaosre,nicwtstyiidh.oeinlrsTitn4hNg.2e.o(dunIiklfny)-
Nu Nu 40k case the number of neighbors is limited by the chosen sim-
10k ilarity function to a number lower than k, we also consider
20k
the 2nd-level neighbors, i.e., for each user ua belonging to
0k 0k
Jul 200A8ug 20S0e8p 200O8ct 200N8ov 20D0e8c 20J0a8n 200F9eb 20M0a9r 20A0p9r 200M9ay 20J0u9n 2009 Jul 200A8ug 20S0e8p 200O8ct 200N8ov 20D0e8c 20J0a8n 200F9eb 20M0a9r 20A0p9r 200M9ay 20J0u9n 2009 2Nn(du-kle)vweleucsoermspisutteheNn(uusae)d.tTohdeefionveertahllesuesterosfs1imst-illaervetloaunkd,
1000k as previously described.
(a)
ngs 800k (b) The coefficient c(uk,ua) represents the neighbor’s informa-
ordi tionweightforuseruk. InmostofthekNN-basedalgorithms
Rec 600k [8],thecoefficient usedisthesimilarity between uk andua.
er of 400k In other cases [3] the coefficients are calculated using de-
mb rived interpolation weights. It is worth noting that, in case
Nu 200k of considering 2nd-level neighbors, the coefficient c(uk,ua)
in eq. (1) has to be computed taking into account the sim-
0k Jul 2008Aug 2008Sep 2008Oct 2008Nov 2008Dec 2008Jan 2009Feb 2009Mar 2009Apr 2009May 2009Jun 2009 iFlaorriteyxabmetpwlee,enconthsiedecroinnsgiduesreerdunke,ihgherbonreiagnhdbofruurtahearndonheesr.
2nd-levelneighboru , we have:
b
Figure 1: Number of users, events and recordings c(uk,ub)=c(uk,ua)∗c(ua,ub),
(a) Total and (b) Active in the considered period
that is a combination of the similarities computed between
theneighbors pairs for theconsidered user.
4. RECOMMENDATION MostPopular. TheMostPopular algorithmcanbealsode-
In our context, we can identify two different approaches to fined by means of eq. (1), assuming the number of neigh-
recommendation, depending on the specific target which is bors unbounded, which implies N(uk) = U, ∀uk ∈ U; and
considered. As a first attempt, we define a set of events c(ua,ub)=1, ∀ua,ub ∈U.
whichcanbeofinteresttothemajorityoftheusers. Insuch Theweightofanelementeitoanuserukisthereforedefined
acase, wearetryingtoidentify themost frequent eventsin as:
the systems, i.e., those programs which have been recorded
by the largest subset of users. This is done by means of a w(uk,ei)= r(ua,ei) (2)
recommendation algorithm which we name MostPopular. uXa∈U
Asecondapproachfocusesonidentifyingthoseeventswhich Aftercalculating theweight of allelements, theyaresorted
canbeofanyinterestforasingleuserofthesystem. Inthis in descendant order. In the MostPopular algorithm, as the
case, which we can refer to as user-oriented, the aim of the set of neighbors is independentof theuser, all usersreceive
the same recommended elements, i.e., the most popular el- Item-to-item. Thesimilarityamongitems,S(e,f),isbased
ements. on the same measures already mentioned before, yet rede-
finedconsideringtwoitemse,f andtheirsetsofusersUe,Uf
Item-based kNN. In the item-based kNN algorithm, the who recorded them.
weight of an element ei for an user uk is definedas: ∀e∈E we can compute the subset Ne ⊆E of neighbors of
item e. An item f such that Ue ∩Uf 6= ∅ is thus defined
w(uk,ei)= r(uk,ej)·c(ei,ej), (3) as a neighbor of e. Starting from the neighborhood of e,
ej∈XN(ei) similarity with e is computed for each pair < e,f > such
N(ei) is the set of n items most similar to ei and recorder Wthaetcfan∈tNheen. decide whether a couple of items is similar or
byuk,andc(ei,ej)istheneighbor’sinformationweightwrt not. Items e is considered similar to f, i.e., there is an arc
item ei. connectingtheminthesimilarity network,ifS(e,f)>0. A
similarityorderamongtheneighborsofeisthusdetermined.
Differently from the user-based case, using k = ∞ in the
item-based approach doesnot lead totheMost Popular set
5. EXPERIMENTAL RESULTS
of elements. In fact, the algorithm simply takes all items
ej ∈Euk as neighbors of ei, makingN(ei) user-dependent. Ourevaluationisbasedontryingtomeasurehowaccurateis
each recommendation algorithm in predicting the elements
SVD. The Singular Value Decomposition technique ana- that users would program. This is achieved by computing
lyzed in this work makes use of implicit feedbacks and im- precisionandrecallonthepredicteditems. Themoreaccu-
plementsthemethodproposedin[9]. Specifically,giventhe rate is this prediction, the more valuable elements are rec-
observations of the behavior of user u wrt item i, rui, we ommended. It is important to underline that we do not
can definethe user’s preference as: consider any feedback related to the user’s interest in the
recommended items, but we only focus on the prediction
pui= 10 iiff rruuii >=00 ability of thealgorithms analyzed.
(cid:26)
To start evaluating a recommendation algorithm, we fix an
where rui is set to 1 when u records item i, 0 otherwise.
arbitrary timet after thedatacollection started and before
the data collection stopped. The value of t should be care-
Afterassociatingeachuseruwithauser-factorsvectorxu ∈
Rf and each item i with an item-factors vector yi ∈ Rf, fully chosen not to betoo close to thedatacollection start,
since we do not have sufficient data to make good predic-
we can predict the unobserved value by user u for item i
through the inner product: xTuyi. Factors are computed by tions. Also, the time t should not be close to the end of
data collection, because we need a good amount of data to
minimizing thefollowing function [9]:
make the verification if the algorithm was able to predict
it. As the data collection started January 23rd 2008 and
min (pui−xTuyi)2+λ kxuk2+ kyik2 ended November 19th 2009, we choose values of t varying
x*y* !
Xu,i Xu Xi from June 1st 2008 toJune 1st 2009.
4.2 Computing Neighborhood
5.1 Metrics
In order to provide recommendation on the discrete ele-
Given the set E of events in our framework, we define the
ments, we have to define a similarity function for grouping
following subsets:
similar users/itemsfrom which choosingtheappropriateel-
ements to recommend. The definition of the similarity is
based only on implicit ratings resulting from observing the
•A(t)⊂E, active eventsat time t (bj >t);
behaviorofusers: ifsherecordssomething,thenweassume
•R(u,t)⊂E,eventsrecorded byuser u before time t;
that sheis interested in it; otherwise, we can not infer any-
•V(u,t)⊂A(t),eventsrecorded byuseru aftertime t;
thing about the interest of the user for that element. We
•Rec(u,t) ⊂ A(t), events recommended to user u at
are therefore considering binary feedbacks.
time t.
User-to-user. Let u and v be two users and Eu,Ev the
setsofrecordedelementsassociated tothem;wecanchoose It is important to notice that A(t) is also the set of all ele-
thesimilaritymetric,S(u,v),consideringseveralwellknown mentssuitableforrecommendationattimet. Theaimofour
measures, such as: the Jaccard’s coefficient, the Dice’s co- recommendationalgorithmsistopredictwhicheventsarein
efficient, the Cosine similarity and the Matching similarity V(u,t). For that, for each user, the algorithms associate a
[11]. weight w(u,s) to each element s ∈ A(t) which represents,
Then,∀u,wecanthencomputethesubsetNu ⊆U ofneigh- from therecommender’spointofview,howmuchreliableis
bors of user u. A user v such that Ev ∩Eu 6= ∅ is thus the fact that s ∈ V(u,t). Furthermore, a recommendation
defined as a neighborof u. Startingfrom theneighborhood algorithm useonly the information in
of u, similarity with u is computed for each pair < u,v >
such that v∈Nu. R(u,t), with R(u,t)∩V(u,t)=∅.
Finally, if S(u,v)>0, we consider u similar tov, i.e., there u[∈U
is an arc connecting them in the similarity network. The To recommend items to users, we use only the top n rec-
valueS(u,v)isusedtoweight sucharelation, thereforede- ommended elements Rec(n,u,t)⊂Rec(u,t), i.e., the top n
termining a similarity order among theneighborhood of u. elementsinRec(u,t),orderedbyweight. Theprecision and
recall at time t are computed as the average of all users’ the results are similar with other similarity functions. In
precision and recall values computed using the top n rec- addition, in Figure 3(b), as well as in Figure 3(c), we omit
ommended elements [13]. Finally, we compute the system thevaluesofk={500,700}sincetheresultsareverysimilar
precisionandrecallatdifferenttimes,andcalculatethesys- to thecase of k=300.
tem overall precision and recall as theaverage of it. We can observe that a value k = 100 is not sufficient to
outperform the MostPopular algorithm, due to the lower
Asin[9],alsoinourcontextprecisionmeasuresarenotvery valueoftherecall. Ontheotherside,averyhighnumberof
meaningful,becausewedonothavefeedbacksregardingthe neighbors allows to perform better than the MostPopular.
user’sinterestinthoseitemswhichhavenotbeenconsidered However, we could notice that, already with k = 2000, the
(i.e., not programmed, nor downloaded). On the contrary, algorithm startstoconvergetotheMostPopular, character-
recall-oriented measures are more suitable. Infact, we can ized by an unbounded number of neighbors by definition.
assume that ei is of any interest for user u only if ei ∈ Therefore, we can consider the range [100,2000] as suitable
V(u,t), otherwise no assumption on user’s interests can be to identify theoptimal valuefor k.
made. Anyway,forsakeofcompleteness,wealsoreportthe Forthispurpose,wetestthevaluesk={300,500,700}, ob-
analysis of precision values. taining very similar performance. Considering the top 10
recommended elements, we can achieve better results for
5.2 Evaluation k = 300, whilst k = 500 is more suitable when taking the
top11to30elements. Asinmostcases10elementsaresuf-
As a first step in the evaluation, we attempt to define the
ficientforarecommendation,k=300isagoodcompromise
specificupperandlowerboundswhichcharacterizetherec-
between the ability of providing valuable recommendations
ommendations in the PVR domain. In particular, we com-
and the resource consumption in calculating the neighbor-
pare the MostPopular recommender, which identifies the
hood.
most frequent elements among all users (Section 4.1), with
the following two algorithms: (1) a random recommender,
To better observe the trend of both recall and precision,
which simply chooses n random elements among those of
Figure3(c)showsthetwovaluescombined. Again,k=300
A(t),definingthelowerboundtoourexperiment;(2)anex-
performsbetterifwetakeonlythetop10recommendedele-
act predictor recommender, which has knowledge about the
ments,asitalsoyieldstogoodresultsintermsofprecision.
elementsinV(u,t),thusyieldingtothebestpossibleresults
Considering more than 10 recommendations, it would seem
and definingtheupperbound.
appropriate to increase the number of neighbors to 500, as
TheresultsaredepictedinFigure2(a),whichclearlyshows
theresultsfor precision andrecall are slightly better. How-
that,asexpected,eventheMostPopular algorithmcaneas-
ever,theoverallbehaviorofthealgorithmisalmostidentical
ily outperform a random predictor. However, it is still far
withkintherange{300,700}. Nevertheless,theabovemen-
from beingabletomakea completepredictionof all theel-
tionedconsiderationsregardingthesuperiorperformanceof
ements, especially when the considered top n are just few
the kNN algorithm with k = 300 in terms of computation
items.
requirementsstill applywhenwe takeintoaccountthepre-
cision metric.
The second step in our evaluation is to study how different
similarityfunctionsaffecttheresultsofuser-basedkNNrec-
AninterestingcomparisonamongthethreekNNalgorithms
ommendationalgorithms. Wecan observefrom Figure2(b)
analyzed, i.e., user-based, item-based and MostPopular, is
that,incaseoftheuser-basedalgorithm,allchosensimilar-
depicted in Figure 4(a). We can observe that the latter is
ities show nearly thesame performances.
clearlyoutperformedbytheothertwoalgorithmsintermsof
On the contrary, the Matching similarity considerably out-
recall, especially whenmorethan7recommendeditemsare
performs the other measures when it comes to the item-
considered. Betweentheitem-basedandtheuser-basedver-
based algorithm, as displayed in Figure 2(c). Again, both
sionofthekNN,thelatterperformsslightlybetter,although
Diceand Jaccard showaverysimilar behavior,beingsupe-
the gap is mostly noticeable when more than 15 items are
riortotheCosinemetricalreadywhenmorethan5elements
recommended. In general, item-based algorithms tend to
are recommended. In both Figures 2(b) and 2(c), the Jac-
perform better because usually thenumberof items is con-
card similarity is not shown being almost identical to the
siderably lower than the users [12]. Such a property does
Dice.
not hold in our domain, hence making the user-based ver-
sion superior in terms of recall, as we initially expected.
Anotherstepinourevaluationistofindtheconsequencesof
adding second-level neighbors in the neighborhood of user-
A final experiment is attempted in order to measure the
based kNNrecommendation algorithms. In Figure3(a),we
behavior of the SVD approach wrt the performance of the
can observe that increasing the number of first level neigh-
kNN method. The implementation of the SVD algorithms
bors (when it is lower than k) by adding the second level
described in Section 4.1 is tested with different parameters,
onesimpliesabetterperformanceofthealgorithms. Inthis
with the purpose of identifying the more suitable ones in
example, we used Dice similarity and k =300, however the
our context. In particular, we try different sizes for user-
results are similar when applying second-level neighbors to
factors anditem-factors vectors,valuesfor theλ parameter
other similarities.
and numberof training steps.
Results are depicted in Figure 4(b). The best prediction is
Inthenexttests,wetrytofindanoptimalvalueforkinthe
obtained with 100 features, λ = 500, α = 40 and 15 train-
user-basedkNNalgorithm. Figure3(b)showstheresultsof
ing steps. However, the behavior of thelatent factor model
kNNuser-basedwithk∈{100,300,500,700,2000}, andthe
based on SVDin theanalyzedcontextis worse if compared
MostPopular recommender. We used Dice similarity, but
100% 35% 35%
30% 30%
80%
25% 25%
60%
Recall REMxaoanscdtt oP pmorpe pudrliaecrdtiiocntion Recall20% Recall20%
15% 15%
40%
10% 10%
20%
5% Dice similarity 5% Dice similarity
Cosine similarity Cosine similarity
Matching similarity Matching similarity
0%0 5 10 15 20 25 30 0%0 5 10 15 20 25 30 0%0 5 10 15 20 25 30
top selection top selection top selection
(a) Upperand lower boundsfor recall (b) Comparison between similarity func-(c) Comparison between similarity func-
tions in user-based kNN tions in item-based kNN
Figure 2: Comparison between recommenders and similarity functions.
40% 40%
Most Popular
18% k=2000
35% 35% k=300
16% k=100
30% 30%
14%
25% 25%
Recall20% Recall20% Precision12%
15% 15% 10%
10% 10% 8%
Most Popular
5% 5% k=2000 6%
Neighborhood level = 1 k=300
Neighborhood level = 2 k=100
0%0 5 10 15 20 25 30 0%0 5 10 15 20 25 30 4%0% 5% 10% 15% 20% 25% 30% 35%
top selection top selection Recall
(a)Comparisonofone-levelandtwo-level (b)Recall for user-based kNN (c) Precision vs Recall for user-based
neighborhoods for user-based kNN kNN
Figure 3: Neighborhoods comparison, precision and recall for user-based kNN.
40% 40% 20%
35% 35%
30% 30% 15%
25% 25%
Recall20% Recall20% Precision10%
15% 15%
10% 10% 5%
User-based KNN
5% User-based KNN 5% Most Popular Item-based KNN
Item-based KNN User-based KNN Most Popular
Most Popular SVD SVD
0%0 5 10 15 20 25 30 0%0 5 10 15 20 25 30 0%0% 5% 10% 15% 20% 25% 30% 35% 40%
top selection top selection Recall
(a) Recall for kNN (k = 300) wrt Most- (b) Recall for SVD wrt kNN (k = 300) (c) Precision vs Recall for kNN (k =
Popular User-based and MostPopular 300), SVD and MostPopular
Figure 4: Precision and recall for the analyzed algorithms.
to a neighborhood model such as kNN. As the reader can shows to be more precise than the item-based in determin-
notice, the kNN user-based is able to substantially outper- ingtherecommendableitems,forthesamereasonpreviously
formtheSVDtechnique,whoseresultsintermsofrecallare mentioned considering recall.
slightlybetterthanthoseoftheMostPopularalgorithmonly
when a considerable numberof items are recommended. It could appear surprising that the prediction performance
of the SVD recommender is worse than other techniques,
Similarly,resultsrelatedtotheprecisionoftherecommenda- as this algorithm normally performs better in several other
tions (Figure 4(c)) show an analogous behavior of the kNN contexts[13,10,9]. Webelievethatthemotivationsforsuch
algorithmswrtSVD,withtheMostPopular beingconsider- anunusualbehaviorresideinthedatasetcharacteristics. In
ablylessprecisethanothers. Also,theuser-basedalgorithm particular, areason mightbeidentifiedin theso called cold
start problem, whose effects involve users, items and com- 2007 Seventh IEEE International Conference on Data
munities [15]. Mining,pages 43–52, Washington, DC, USA,2007.
Inourcontext,thecoldstartproblemisparticularlynotice- IEEE ComputerSociety.
able with items and is duetothe lack of relevant feedbacks [4] F. Carmagnola, A. Loffredo, and G. Bernardi. Vcast
when a new event first appears in the system. Such an is- on facebook: Bridging social and similarity networks.
sue is made worse by thefact that items to recommend are InHT ’09: Proceedings of the Twentieth ACM
generally new ones, i.e. those eventshaving a starting time Conference on Hypertext and Hypermedia, NewYork,
in the future. This property holds for no-repeat events as NY,USA,July 2009. ACM.
well as for repetive ones (the starting time is updated ac- [5] P.Cremonesi and R. Turrin.Analysis of cold-start
cordingtotheirperiodicity). So,eventswhosestartingtime recommendations in iptv systems. In RecSys ’09:
has passed are nolonger elegible for recommendation. Proceedings of the third ACM conference on
Recommender systems, pages 233–236, New York,NY,
Thefactthatrecommendationsareaffectedbythecoldstart USA,2009. ACM.
problem is one key factor that may influence SVD perfor- [6] P.Cremonesi, R. Turrin,and R. Bambini. A
mance,asthisalgorithmneedssupportofuser’spreferences recommendersystem for an iptvservice provider: a
to perform well. On the contrary, a neighborhood-based real large-scale production environment.In P.Kantor,
approach such as kNN appears to better deal with newly F. Ricci, L. Rokach,and B. S.(eds.), editors,
introduced items, as also reported in [5]. Recommender Systems Handbook, chapter30, pages
200–220. Springer,2009.
6. CONCLUSION
[7] M. Deshpandeand G. Karypis. Item-based top-n
We experimented with a real digital recording service, and, recommendation algorithms. ACM Trans. Inf. Syst.,
accordinglytotheabovementionedrestrictions,wedecided 22(1):143–177, 2004.
to run our analysis under the strongest assumptions: no
[8] J. L. Herlocker, J. A.Konstan, A.Borchers, and
EPGsareavailable,userscansetuptimingsaswellaschan-
J. Riedl. Analgorithmic framework for performing
nels,explicitfeedbacksarenotcollected,andsoon. Inaddi-
collaborative filtering. InSIGIR ’99: Proceedings of
tion, the intrinsically dynamic nature of the analyzed PVR
the 22nd annual international ACM SIGIR conference
domain, which determines a continuous process of creation
on Research and development in information retrieval,
anddeletionofeventsandaconsequentamplificationofthe
pages 230–237, New York,NY,USA,1999. ACM.
coldstartproblem,makessuchacontextsensiblydifferentin
[9] Y.Hu,Y.Koren, and C. Volinsky.Collaborative
terms ofrecommendation ifcompared tothosewhereitems
filteringfor implicit feedback datasets. InICDM ’08:
haveno timevalidity (i.e., netflix,movielens, etc.).
Proceedings of the 2008 Eighth IEEE International
Conference on Data Mining,pages 263–272,
Despite these constraints, our results showed that neigh-
Washington,DC, USA,2008. IEEE ComputerSociety.
borhood based strategies, such as kNN, can return in good
[10] Y.Koren. Collaborative filtering with temporal
predictionaccuracyand,ifcorrectlytuned,theycanoutper-
dynamics.Commun. CACM,53(4):89–97, 2010.
form SVD-based techniques as well as most popular strate-
[11] B. Markines, C. Cattuto, F. Menczer, D.Benz,
gies, that dangerously leverage the phenomenon of many
A.Hotho, and G. Stumme.Evaluatingsimilarity
users concentrated on veryfew relevant events.
measures for emergent semantics of social tagging. In
Finally, there is evidence that digital recorders differ from
WWW ’09: Proceedings of the 18th international
otherinterestbasedservices,becausefactorsotherthanper-
conference on World wide web, pages 641–650, New
sonaltastesmightinfluencetheuser’sbehaviorandthesuc-
York,NY,USA,2009. ACM.
cess of a recommendation engine. In fact, the direct social
[12] B. Sarwar, G. Karypis, J. Konstan, and J. Reidl.
influenceoffriendsandthevolatilenatureofeventsaresup-
Item-basedcollaborative filtering recommendation
posed to be relevant factors in causing a user to schedule a
algorithms. In WWW ’01: Proceedings of the 10th
recording.
international conference on World Wide Web, pages
In our opinion, a possibile future research direction could
285–295, New York,NY,USA,2001. ACM.
beindeedtheidentificationandstudyofthosesocialfactors
whichaffectuser’sbehaviorinsystemscharacterizedbyhigh [13] B. M. Sarwar, G. Karypis,J. A. Konstan,and J. T.
dynamism and short lifetime of items. Riedl.Application of dimensionality reduction in
recommendersystem - a case study.In In ACM
7. REFERENCES WebKDD Workshop, 2000.
[1] G. Adomaviciusand A. Tuzhilin.Toward thenext [14] J. Schafer, D.Frankowski, J. Herlocker, and S. Sen.
generation of recommender systems: A surveyof the Collaborative filtering recommendersystems. The
state-of-the-art and possible extensions. IEEE Trans. Adaptive Web, pages 291–324, 2007.
on Knowl. and Data Eng., 17(6):734–749, 2005. [15] J. B. Schafer, J. Konstan, and J. Riedi. Recommender
[2] A.Basso, M. Milanesio, and G. Ruffo.Events systemsin e-commerce. In EC ’99: Proceedings of the
discovery for personal videorecorders. InEuroITV 1st ACM conference on Electronic commerce, pages
’09: Proceedings of the seventh european conference on 158–166, New York,NY,USA,1999. ACM Press.
European interactive television conference, pages
171–174, New York,NY,USA,2009. ACM.
[3] R.M. Bell and Y.Koren. Scalable collaborative
filtering with jointly derived neighborhood
interpolation weights.InICDM ’07: Proceedings of the
0
10
s
g
n -1
i 10
d
r
o
c
e
r

x
-2
e 10
v
a
h

t
n
e
m -3
10
e
l
e

n
a

t
a -4
h 10
t

y
t
i
l
i
b
a
b 10-5
o
r
P
-6
10
0 1 2 3 4
10 10 10 10 10
number of recordings