Integrating Reviews into Personalized Ranking for Cold Start Recommendation

0.53 MB·

by  Guang-Neng Hu

Integrating Reviews into Personalized Ranking for
Cold Start Recommendation
Guang-NengHu
DepartmentofComputerScienceandEngineering,
HongKongUniversityofScienceandTechnology,HongKong
[email protected]
Abstract. Itemrecommendationtaskpredictsapersonalizedrankingoveraset
7 ofitemsforindividualuser.Oneparadigmistherating-basedmethodsthatcon-
1 centrateonexplicitfeedbacksandhencefacethedifficultiesincollectingthem.
0 Meanwhile,theranking-basedmethodsarepresentedwithrateditemsandthen
2 rank the rated above the unrated. This paradigm uses widely available implicit

n feedbackbutitusuallyignoressomeimportantinformation:itemreviews.Item
a reviewsnotonlyjustifythepreferencesofusers,butalsohelpalleviatethecold-
J startproblemthatfailsthecollaborativefiltering.Inthispaper,weproposetwo

1 novelandsimplemodelstointegrateitemreviewsintomatrixfactorizationbased
3 Bayesianpersonalizedranking(BPR-MF).Ineachmodel,wemakeuseoftext
featuresextractedfromitemreviewsviawordembeddings.Ontopoftextfea-

] turesweuncoverthereviewdimensionsthatexplainthevariationinusers’feed-
R
backandthesereviewfactorsrepresentapriorpreferenceofauser.Experiments
I
. onreal-worlddatasetsshowthebenefitsofleveragingitemreviewsonranking
s
prediction.Wealsoconductanalysestounderstandtheproposedmodels.
c
[

1
1 Introduction
v
8
8 Usersconfrontwiththe“informationoverload”dilemmaanditisincreasinglydifficult
8
for them to choose the preferred items over others because of the growing large item
8
set, e.g., hundreds of millions products at Amazon.com and tens of thousands videos
0
. atNetflix.com[6,1].Recommendersystems(RSs)assistusersintacklingthisproblem
1
andhelpthemmakechoicesbyrankingtheitemsbasedontheirpastbehaviorhistory.
0
7 Itemrecommendationpredictsapersonalizedrankingoverasetofitemsforindividual
1 userandhencealleviatesthedilemma.
: The rating-based (or point-wise) methods predict ratings that a user will give to
v
i items and then rank the items according to their predicted ratings. Many methods are
X
proposedandmatrixfactorizationbasedmodelsaremostpopularduetotheirscalabil-
r ity, simplicity, and flexibility [10,5,2,4]. This paradigm concentrates on explicit feed-
a
backanditfacesthedifficultiesincollectingthem.Meanwhile,theranking-based(pair-
wise)methodsarepresentedwithseenitemsandthenranktheseenabovetheunseen.
Bayesian personalized ranking (BPR-MF) and collaborative item selection are typi-
calrepresentatives[14,11].Thisparadigmtakesadvantageofwidelyavailableimplicit
feedbackbutitusuallyignoresakindofimportantinformation:itemreviews.
2 G.N.Hu
Relatedworks. Itemreviewsjustifythepreferencesofusersandhelpalleviate
the cold-start problem; they are a diverse and complementary data source for recom-
mendation beyond the user-item co-rating information. The CMF method [15] can be
adaptedtofactorizetheuser/item-wordmatrixconstructedfromtheitemreviews.The
CTR [16] and HFT [7] models integrate explicit ratings with item content/reviews to
build better rating predictors; they employ topic modeling to learn hidden topic fac-
tors which explain the variations of users’ preferences. The CTRank model [17] also
adopts topic modeling to exploit item meta-data like article title and abstract via bag-
of-words representation for one-class collaborative filtering [12], while the CDR [18]
andCKE[19]modelsadoptstackeddenoisingautoencoders.Nevertheless,integrating
itemreviewsintotheranking-basedmethodspresentsbothopportunitiesandchallenges
fortraditionalBayesianpersonalizedranking.Therearefewworksonleveragingitem
reviewstoimprovepersonalizedranking.
Inthispaperweproposetwonovelandsimplemodelstoincorporateitemreviews
into BPR-MF. Like HFT, they integrate item reviews and unlike HFT they generate a
ranked list of items for individual ranking. Like CTRank, they focus on personalized
rankingandunlikeCTRanktheyarebasedonmatrixfactorizationandusingwordem-
beddingstoextractfeatures.LikeBPR-MF,theyrankpreferreditemsoverothersand
unlike BPR-MF they leverage the information from item reviews. In each of the two
models,wemakeuseoftextfeaturesextractedfromitemreviewsusingwordembed-
dings. And on top of text features we uncover the review dimensions that explain the
variationinusers’feedback.Thesereviewfactorsrepresentapriorpreferenceofauser.
Onemodeltreatsthereviewfactorspaceindependentofthelatentfactorspace;another
connectsimplicitfeedbackanditemreviewsthroughtheshareditemspace.
Thecontributionsofthisworkaresummarizedasfollows.
1.Weproposetwonovelmodelstointegrateitemreviewsintomatrixfactorization
basedBayesianpersonalizedranking(Section3.2and3.3).Theygeneratearankedlist
ofitemsforindividualuserbyleveragingtheinformationfromitemreviews.
2. For exploiting item reviews, we build the proposed models on the top of text
featuresextractedfromthem.Wedemonstrateasimpleandeffectivewayofextracting
featuresfromitemreviewsbyaveragelycomposingwordembeddings(Section4).
3. We empirically evaluate the proposed models on multiple real-world datasets
which contains over millions of feedback in total. The experimental results show the
benefitofleveragingitemreviewsonpersonalizedrankingprediction.Wealsoconduct
analyses to understand the proposed models including the training efficiency and the
impactofthenumberoflatentfactors.
2 NotationandProblemStatement
Before proposing our models, we briefly review the personalized ranking task and
thendescribetheproblemstatement.Tothisend,wefirstintroducethenotationsused
throughout the paper. Suppose there are M users U = {u ,...,u } and N items
1 M
I = {i ,...,i }. We reserve u,v for indexing users and i,j for indexing items. Let
1 N
X ∈ RM×N denote the user-item binary implicit feedback matrix, where x is the
u,i
preferenceofuseruonitemi,andwemarkazeroifitisunknown.DefineN asthe
u
IntegratingReviewsintoPersonalizedRankingforColdStartRecommendation 3
setofitemsonwhichuseruhasanaction:N ≡ {i|i ∈ I ∧x > 0}.Rating-based
u u,i
methods [10,5] and ranking-based methods [14,3] are mainly to learn the latent user
factorsP =[P ,...,P ]∈RF×M andlatentitemfactorsQ=[Q ,...,Q ]∈RF×N
1 M 1 N
frompartiallyobservedfeedbackX.
Itemimayhavetextinformation,e.g.,reviewd commentedbyuseru.Weaggre-
ui
gate all reviews of a particular item as a ‘doc’ d = ∪ d . Approaches like CTR
i u∈U ui
andHFT[16,7]integrateitemcontent/reviewswithexplicitratingsforratingprediction
usingtopicmodeling.Anotherapproachistolearnwordembeddingsandthencompose
themintodocumentlevelastheitemtextfeatures;weadoptthiswayofextractingtext
featuresf ∈RD fromd (seeSection4).
i i
2.1 ProblemStatement
Our work focuses on the item recommendation or personalized ranking task where a
rankedlistofitemsisgeneratedforeachindividualuser.Thegoalistoaccuratelyrank
theunobserveditemswhichcontainbothtrulynegativeitems(e.g.,theuserdislikesthe
NetflixmoviesorisnotinterestinginbuyingAmazonproducts)andmissingones(e.g.,
theuserwantstoseeamovieorbuyaproductinthefuturewhensheknowsit).
Instead of accurately predicting unseen ratings by learning a model from training
samples(u,i,x )wherex >0,personalizedrankingoptimizesforcorrectlyrank-
u,i u,i
ingitempairsbylearningamodelfromtrainingtuples D ≡ {(u,i,j)|u ∈ U ∧i ∈
S
N ∧j ∈ I\N }.Themeaningofitempairsofauser(u,i,j)isthatsheprefersthe
u u
former than the latter, i.e., the model tries to reconstruct parts of a total order > for
u
each user u. From the history feedback X we can infer that the observed items i are
ranked higher than the unobserved ones j; and for both observed items i ,i or both
1 2
unobserveditemsj ,j wecaninfernothing.Random(negative)samplingisadopted
1 2
sincethenumberofsuchpairsishuge.SeetheoriginalBPRpaper[14]formoredetails.
Problem1. PersonalizedRankingwithItemReviews.
Input:1)AbinaryimplicitfeedbackmatrixX,2)anitemreviewscorpusC,and
3)auseruintheusersetU .
Output:Arankedlist> overtheunobserveditemsI\N .
u u
In Problem 1, to generate the ranked list, we have item reviews to exploit besides
implicitfeedback.
3 TheProposedModels
Inthissection,weproposetwomodelsasasolutiontoProblem1whichleverageitem
reviewsintoBayesianpersonalizedranking.Onemodeltreatsthereviewfactorspace
independent of the latent factor space (Section 3.2). Another model connects implicit
feedbackanditemreviewsthroughtheshareditemspace(Section3.3).Ineachofthe
two proposed models, we make use of text features extracted from item reviews via
wordembeddings(Section4).Ontopoftextfeaturesweuncoverthereviewdimensions
thatexplainthevariationinusers’feedbackandthesereviewfactorsrepresentaprior
preferenceofauser.Bothmodelsarebasedonbasicmatrixfactorization(Section3.1)
andlearnedundertheBayesianpersonalizedrankingframework(Section3.4).
4 G.N.Hu
3.1 BasicMatrixFactorization
Thebasicmatrixfactorization(BasicMF)ismainlytofindthelatentuser-specificfea-
turematrix[P ]M anditem-specificfeaturematrix[Q ]N toapproximatethepartially
u 1 i 1
observedfeedbackmatrixX intheregularizedleast-squares(orridgeregression)sense
bysolvingthefollowingproblem.
min(cid:88) (x −xˆ )2+λ((cid:107)P(cid:107)2 +(cid:107)Q(cid:107)2), (1)
u,i u,i F F
P,Q xu,i(cid:54)=0
whereλistheregularizationparametertoavoidover-fitting.Thepredictedscoresxˆ
u,i
canbemodeledbyvariousformswhichembodytheflexibilityofmatrixfactorization.
AbasicformisxˆBasic =α+β +β +PTQ ,whereα,β andβ arebiases[5].
u,i u i u i u i
3.2 IntegratingItemReviewsintoBasicMF:DifferentSpaceCase
In this section, we propose our first model TBPR-Diff to integrate item reviews with
implicit feedback. Analogical to the Basic MF which factorizes the ratings into user-
and item- latent factors, we can factorize the reviews into user- and item- text factors
(see the illustration in Figure 1—Up). The TBPR-Diff model sharpens this idea and
teasesaparttheratingdimensionsintolatentfactorsandtextfactors:
xˆDiff =α+β +β +PTQ +θT(Hf )+β(cid:48)Tf , (2)
u,i u i u i u i i
where the term θT(Hf ) is newly introduced to capture the text interaction between
u i
user u and item i. To exploit item reviews, text features f ∈ RD are firstly extracted
i
fromitemreviewsviawordembeddings(hencetheyareknownandfixed).Theshared
embeddingkernelH ∈ RK×D linearlytransformsoriginaltextfeaturesf fromhigh-
i
dimensionalspace(e.g.,200)intoalowertextratingspace(e.g.,15),andthenit(Hf )
i
interactswithtextfactorsofuserθ ∈ RK.Atextbiasvectorβ(cid:48) isalsointroducedto
u
modelusers’overallpreferencestowardstheitemreviews.Thedetailsoftextfeatures
extractedfromitemreviewsusingwordembeddingsaredescribedlater(seeSection4).
Sincethetextfactorsofuserθ andofitem(Hf )areindependentoflatentfactors
u i
P andQ ,thereisnodeepinteractionsbetweentheinformationsourcesofobserved
u i
feedback and item reviews, and hence they cannot benefit from each other. Also ad-
ditional parameters increase the model complexity. Based on these observations, we
proposeanothermodeltoalleviatetheabovechallenges.
3.3 IntegratingItemReviewsintoBasicMF:SharedSpaceCase
In this section, we propose our second model TBPR-Shared to integrate item reviews
with implicit feedback more compactly. For an item i, its latent factors Q learned
i
fromfeedbackcanbeconsideredascharacteristicsthatitprocesses;meanwhile,these
characteristicsareprobablydiscussedinitsreviewsandhenceexhibitinitstextfactors
Hf (seetheillustrationinFigure1—Down).Foruseru,ifweletQ and{Hf |k ∈
i i k
N } be in the same space then it leads to deep interactions between text factors of
u
useruandthelatentfactorofitemi.TheTBPR-Sharedmodelsharpensthisideaand
IntegratingReviewsintoPersonalizedRankingforColdStartRecommendation 5
enablesthedeepinteractionsbetweentextfactorsandlatentfactorsaswellasreduces
complexityofthemodel:
xˆShared = QT(P + |N |−1/2(cid:88) Hf ) + α + β + β + β(cid:48)Tf . (3)
u,i i u u k u i i
k∈Nu
On the right hand, the last four terms are the same with the TBPR-Diff model.
Different from the TBPR-Diff model, the shared item factors Q now have two-fold
i
meanings: one is item latent factors that represent items’ characteristics; another is to
interact with item text factors that capture items’ semantics from item reviews. Also
different from the TBPR-Diff model, the preferences of a user now have a prior term
whichshowsthe‘textinfluenceofherrateditems’capturedbythetextfactorsofcorre-
spondingitems.Insummary,ontopoftextfeaturestheTBPR-Sharedmodeluncovers
the review dimensions that explain the variation in users’ feedback and these factors
representapriorpreferenceofuser.
F-dimensional
item latent factors
Biases
they work very well easy
to clean we wash them in Embedding
the dishwasher…
Predictor
very soft and comfortable Linear Map
and warmer …
Item factors User factors
Item Reviews D-dimensional K-dimensional
text features item text factors
F-dimensional
user latent factors Biases
they work very well easy
to clean we wash them in Embedding
the dishwasher…
Predictor
very soft and comfortable Linear Map
and warmer …
User factors Item factors
Item Reviews D-dimensional F-dimensional
text features user text prior
Fig.1. Illustrating the Preference Predictors of Our Proposed Two Models. Up—TBPR-Diff
model:Theratingdimensionsaretoteaseapartintotextfactorsandlatentfactorsforbothuser
anditem.Down—TBPR-Sharedmodel:Theratingdimensionsaretoteaseapartintotextfactors
andlatentfactorsforonlyuserswherethetextfactorstransformedfromtextfeaturesactasprior
preferenceandshowthe‘textinfluenceofherrateditems’.
Remarks I.TheVBPRmodel[3]proposedananalogicalformulationwithEq(2).
It exploits visual features extracted from item images and we leverage item features
extracted from item reviews. The SVD++ and NSVD [5,13] models proposed similar
formulaswithEq(3).Theylearnanimplicitfeaturematrixtocaptureimplicitfeedback
andwelearnatextcorrelationmatrixtocapturetextfactors;notethattheydidn’texploit
item reviews and hence they had no the text bias term. II. There can be an adjustable
6 G.N.Hu
weight on the term of text (i.e., θT(Hf ) in Eq (2) and QT|N |−1/2(cid:80) Hf in
u i i u k∈Nu k
Eq (3)) to balance the influence from feedback and from reveiws, but here we just let
feedbackandreviewsbeequallyimportant.
Beforewedelveintothelearningalgorithm,thepreferencepredictorsofTBPR-Diff
andofTBPR-SharedmodelsareshowninFigure1.
3.4 ModelLearningwithBPR
Revisit Problem 1, we need to generate a ranked list of items for individual user.
Bayesianpersonalizedranking[14]isagenericpair-wiseoptimizationframeworkthat
learns from the training item pairs using gradient descent. Denote the model parame-
ters as Θ and let xˆ (Θ) (for simplicity we omit model parameters, and the notation
uij
x isthesamewithx )representanarbitraryreal-valuedmappingunderthemodel
ui u,i
parameters.ThentheoptimizationcriterionforpersonalizedrankingBPR-OPTis
(cid:88)
L(Θ)≡ lnσ(xˆ )−λ(cid:107)Θ(cid:107)2, (4)
uij
(u,i,j)∈DS
wherexˆ ≡xˆ −xˆ ,andthesigmoidfunctionisdefinedasσ(x)=1/(1+exp(−x)).
uij ui uj
The meaning behind BPR-OPT requires ranking items accurately as well as using a
simplemodel.
Under the generic BPR-OPT framework, we derive the learning process for our
proposed models TBPR-Diff and TBPR-Shared by embodying xˆ with xˆDiff and
ui ui
xˆShared,respectively.TheBPR-OPTdefinedinEq(4)isdifferentiableandhencegra-
ui
dientascentmethodscanbeusedtomaximizeit.Forstochasticgradientascent,atriple
(u,i,j)israndomlysampledfromtrainingsetsD andthenupdatethemodelparam-
S
etersby:
∂xˆ
Θ←Θ+η(σ(−xˆ ) uij −λΘ). (5)
uij ∂Θ
Thesamegradientsforuserlatentfactorsandbiastermsofbothmodelsare:
∂ ∂ ∂ ∂
xˆ =Q −Q , xˆ =f −f , xˆ =1, xˆ =−1.
∂P uij i j ∂β(cid:48) uij i j ∂β uij ∂β uij
u i j
ParametergradientsofthemodelTBPR-Diffare:
∂ ∂ ∂ ∂
xˆ =P , xˆ =−P , xˆ =H(f −f ), xˆ =θ (f −f )T.
∂Q uij u ∂Q uij u ∂θ uij i j ∂H uij u i j
i j u
ParametergradientsofthemodelTBPR-Sharedare:
∂ (cid:88) ∂ (cid:88)
xˆ =P +|N |−1/2 Hf , xˆ =−(P +|N |−1/2 Hf ),
∂Qi uij u u k∈Nu k ∂Qj uij u u k∈Nu k
∂ (cid:88)
xˆ =|N |−1/2(Q −Q )( f )T.
∂H uij u i j k∈Nu k
Complexity of Models and Learning. The complexity of model TBPR-Diff
is (M + N)F + (M + D)K + D while the complexity of model TBPR-Shared is
IntegratingReviewsintoPersonalizedRankingforColdStartRecommendation 7
(M +N)F +(D+1)K.Wecanseethatthelattermodelreducesthecomplexityby
O(MK),i.e.,theparameters[θ ]M.Forupdatingeachtrainingsample(u,i,j)∈D ,
u 1 S
thecomplexityoflearningTBPR-Diffislinearinthenumberofdimensions(F,K,D)
whilethecomplexityoflearningTBPR-Sharedisalsolinearprovidedthatthescaleof
(cid:80)
rateditemsofusersisamortizingconstant,i.e., |N |/|U|≈const(cid:28)|I|,which
u∈U u
holdsinreal-worlddatasetsbecauseofsparsity(seeTable1).
4 FeatureRepresentationsofItemReviews
Recall that when generating the ranked list of items for individual user, we have item
reviews to exploit besides implicit feedback. To exploit item reviews, we extract text
featuresfromthem,i.e.,thereisafeaturevectorforeachitem.Ourproposedtwomodels
are both built on the top of text features ([f ]N ) and hence they are important for
i i=1
improvingpersonalizedranking.Inthissection,wegiveonesimplewaytoextracttext
featuresfromreviewsofitem—wordembedding.
The SGNS model [9] is an architecture for learning continuous representations of
words from large corpus; these representations, or word embeddings, can capture the
syntacticandsemanticrelationshipsofwords.WefirstruntheGoogleword2veccode
onAmazonreviewscorpus(seeTable1)usingthedefaultsetting(particularly,dimen-
sionalityD =200)tolearnavectore foreachwordw.Andthenwedirectlysumup
w
alloftheembeddingsinanitem’sreviews(excludingstopwords)andgetacomposition
vectorasthetextfeatureforthisitem:
1 (cid:88)
f ≡ e . (6)
i |di| w∈di w
To get f , we can also use complex methods (e.g., tensor networks to compose word
i
embeddingsorlearningthedocrepresentationdirectly);theyareleftforfuturework.
5 Experiments
WeevaluateourtwomodelsonmultipleAmazon.comdatasetsintermsofrankingper-
formance(Section5.1).TheyintegrateitemreviewsintoBayesianpersonalizedranking
optimizationcriterionandwewanttoknowthebenefitfromthem.Sowecomparewith
BPR-MF[14]whichignoresthemandalsowiththemostpopular(POP)baselinethat
doesn’tshowpersonalizedranking(Section5.2).Wereporttheresultsindifferentset-
tings(Section5.3)andanalysetheproposedmethods(Section5.4).
Table1.StatisticsofDatasets
Datasets#Users #Items#Feedback#Words#ColdUser#ColdItem Density(%)
Girls 778 3,963 5,474 302M 572 3,946 0.177
Boys 981 4,114 6,388 302M 787 4,080 0.158
Baby 1,238 4,592 8,401 302M 959 4,482 0.147
Men 21,793 55,647 157,329 302M 15,821 52,031 0.013
Women 62,928157,656 504,847 302M 41,409 143,444 0.005
Phones 58,741 77,979 420,847 210M 43,429 67,706 0.009
8 G.N.Hu
5.1 DatasetsandEvaluationProtocol
Datasets. WeevaluateourmodelsonsixAmazondatasets[8]http://jmcauley.
ucsd.edu/data/amazon/.Theyconsistoffivefromclothingandshoescategory,
andonefromcellphonesandaccessories.Weusethereviewhistoryasimplicitfeed-
back and aggregate all users’ reviews to an item as a doc for this item. We draw
the samples from original datasets such that every user has rated at least five items
(i.e., ∀u ∈ U : |N |≥ 5) and the statistics of final evaluation datasets are show
u
in Table 1. From the table we can see that: 1) the observed feedback is very sparse,
typically less than 0.01%; 2) the average feedback events for users are typical about
(cid:80)
ten, i.e., |N |/|U| ≈ 10 (cid:28) |I| holds; 3) more than half of the users and
u∈U u
of the items are cold and have feedback less than seven. Note that the cold-users/-
itemsarethosethathavelessthansevenfeedbackevents,andthefeedbackDensity=
#Feedback/(#Users∗#Items).
Wespliteachofthewholedatasetsintothreeparts:training,validation,andtest.In
detail,foreachuseru ∈ U,werandomlysampletwoitemsfromherhistoryfeedback
fortestsetTest ,twoforvalidationsetValid ,andtherestfortrainingsetTrain ;
u u u
andhenceN = Train ∪Valid ∪Test .Thisisthereasonthatwediscardusers
u u u u
whorateditemslessthanfivetoensurethatthereisatleastonetrainingsampleforher.
Evaluation Protocol. For item recommendation or personalized ranking, we need
togeneratearankedlistovertheunobserveditems.Thereforeforthehold-outtestitem
i∈Test ofindividualuseru,theevaluationcalculateshowaccuratelythemodelrank
u
i over other unobserved items j ∈ I\N . The widely used measure Area Under the
u
ROCCurve(AUC)sharpenstherankingcorrectnessintuition:
1 (cid:88) 1 (cid:88)
AUC = δ(xˆ >xˆ ), (7)
|U| |E(u)| u,i u,j
u∈U (i,j)∈E(u)
where E(u) = {(i,j)|i ∈ Test ∧ j ∈ I ∧ j ∈/ N } and the δ(·) is an indicator
u u
function.AhigherAUCscoreindicatesabetterrecommendationperformance.
ThevalidationsetV =∪ Valid isusedtotunehyperparametersandwereport
u∈U u
thecorrespondingresultsonthetestsetT =∪ Test .
u∈U u
5.2 ComparingMethods
We compare our proposed models TBPR-Diff (see Eq (2)) and TBPR-Shared (see
Eq (3)) with the Most Popular (POP) and BPR-MF [14] baselines. The difference of
modelsliesintheirpreferencepredictors.
Reproducibility. Weusethereleasedcodein[3]toimplementthecomparingmeth-
ods and our proposed models. The hyperparameters are tuned on the validation set.
Referringtothedefaultsetting,fortheBRP-MFmodel,thenorm-penaltyλ=11,and
learningrateη = 0.005.AswithourproposedmodelsTBPR-DiffandTBPR-Shared,
the norm-penalty λ = 11 for latent factors and λ = 5 for text factors, and
latent text
learningrateη =0.001.Forsimplicity,thenumberoflatentfactorsequalstothenum-
ber of text factors; the default values for them are both fifteen (i.e., F = K = 15).
Since the raw datasets, comparing code, and parameter setting are given publicly, we
confidentlybelieveourexperimentsareeasilyreproduced.
IntegratingReviewsintoPersonalizedRankingforColdStartRecommendation 9
Table2.AUCPerformanceResults(#factors=15,bestresultisboldfaced).
DatasetsSetting POP BPR-MFTBPR-DiffTBPR-Shared Improv1Improv2
Girls All 0.1699 0.5658 0.5919 0.5939 4.966 7.097
Boys All 0.2499 0.5493 0.5808 0.5852 6.535 11.99
Baby All 0.3451 0.5663 0.5932 0.6021 6.321 16.18
All 0.5486 0.6536 0.6639 0.6731 2.983 18.57
Men Cold 0.4725 0.5983 0.6114 0.6225 4.044 19.23
All 0.5894 0.6735 0.6797 0.6842 1.588 12.72
Women Cold 0.4904 0.6026 0.6110 0.6152 2.090 11.22
All 0.7310 0.7779 0.7799 0.7809 0.386 6.396
Phones Cold 0.5539 0.6415 0.6464 0.6467 0.811 5.936
5.3 PerformanceResults
The AUC performance results on eight Amazon.com datasets are shown in Table 2
wherethelastbutonecolumnis(AUC −AUC )/AUC ×
TBPR−Shared BPR−MF BPR−MF
100%,andthelastcolumnis(AUC −AUC )/(AUC −
TBPR−Shared BPR−MF BPR−MF
AUC )×100%.Foreachdatasettherearethreeevaluationsettings:TheAllItems
POP
orAllsettingevaluatesthemodelsonthefulltestsetT;theColdStartorColdsetting
evaluatesthemodelsonasubsetT ⊆ T suchthatthenumberoftrainingsamples
cold
foreachitemwithinT isnogreaterthanthree(i.e.,|Train |≤ 3or|N |≤ 7);the
cold u u
Warm setting evaluates the models on the difference set of All and Cold. Revisit the
Table1wecanseethat:1)almostalloftheitemsarecold-itemfordatasetsGirls,Boys,
and Baby; and hence the results of Cold setting are almost the same with All and the
results of Warm setting is not available to get a statistical reliable results; and 2) for
otherthreedatasets,thepercentofcold-itemsisalsomorethan86%whichrequiresthe
modeltoaddresstheinherentcoldstartnatureoftherecommendationproblem.
Thereareseveralobservationsfromtheevaluationresults.
1.UndertheAllsetting,TBPR-Sharedisthetopperformer,TBPR-Diffisthesec-
ond, with BPR-MF coming in third and POP the weakest. These results firstly show
that leveraging item reviews besides the feedback can improve the personalized rank-
ing;andalsoshowthatthepersonalizationmethodsaredistinctlybetterthantheuser-
independentPOPmethod.Forexample,TBPR-Sharedaveragelyobtainsrelative4.83%
performance improvement compared with BPR-MF on the first three smaller datasets
intermsofAUCmetric,and2.74%intotalsixdatasets.Thistwofiguresshow,tosome
extent,thattransferringtheknowledgefromauxiliarydatasource(hereitemreviews)
helpsmostwhenthetargetdatasource(hereratingfeedback)isnotsorich.
2.UndertheColdsetting,TBPR-Sharedisthetopperformer,TBPR-Diffisthesec-
ond, with BPR-MF coming in third and POP is also the weakest. These results firstly
show that leveraging item reviews besides the feedback can improve the personalized
ranking even in the cold start setting; and also show that the personalization meth-
ods are distinctly better than the user-independent POP method since the cold items
arenotpopular.Indetail,TBPR-Sharedaveragelyobtainsrelative2.31%performance
improvement compared with BPR-MF in terms of AUC metric. Furthermore, TBPR-
Shared compared with BPR-MF, the relative improvement in the cold start setting is
10 G.N.Hu
about1.6timesthanthatintheAllsettingwhichimpliesthatintegratingitemreviews
more benefits when observed feedback is sparser. As with the results on the Phones
dataset,revisitingTable1wecanseethattheratioofcolditemsoverallitemis86.8%
whichisfarlessthanthoseonothertwodatasets(∼ 92.2%).Andinthiscaseadding
auxiliaryinformationdoesn’thelpmuch.
We also evaluate on the Warm setting (not shown in Table 2), and all of the per-
sonalized,complexmethodsareworsethantheuser-independent,simplemethodPOP.
Warm items are more likely to be popular and show less personalized characteristics.
Itremindsusthecommonplacethatrecommendationplaysanimportantroleinlong-
taileditems.
5.4 AnalysisoftheProposedModels
After demonstrating the benefits of leveraging item reviews, we analyse the proposed
models from two points; one is the impact of number of latent factors, and one is the
trainingefficiencyandconvergenceanalysis.Moredepthinvestigationliketheimpact
of embedding dimensionality and of corpus source to train the embeddings, is left to
futurework.
ImpactoftheNumberofLatentFactors. ThetwoproposedmodelsTBPR-Shared
andTBPR-Diffhavetwoimportanthyperparameters;oneisthenumberoflatentfactors
F andoneisthenumberoftextfactorsK.Forsimplicity,weletthetwovaluesequal.
We vary the number of latent factors #factors = {5,10,15,20,25} to observe the
performanceresultsofdifferentmethods.ThetestAUCscoresareshowninFigure2.
OntheGirlsandBoysdatasets,bothofthepersonalizedmodelsaretoperformbetter
asthenumberoffactorsincreases;ontheotherdatasets,theperformanceimprovesas
the number of factors increases to around fifteen; then it doesn’t go up and may even
downgrade.Wesetthedefaultvalueas15.
Alsotheplotsvisuallyshowthebenefitsofintegratingitemreviews(TBPR-Shared
vs. BPR-MF) and of generating a personalized ranking item list for individual user
(TBPR-SharedandBPR-MFvs.POP).
0.75
0.7 PBOPRP−MF 0.7 PBOPRP−MF PBOPRP−MF
TBPR−Shared TBPR−Shared 0.7 TBPR−Shared
0.6 0.6 0.65
0.5 0.5 0.6
AUC0.4 AUC0.4 AUC0.55
0.3 0.3 0.5
0.2 0.45
0.2
0.1 5 10 15 20 25 5 10 15 20 25 0.4 5 10 15 20 25
Number of Factors Number of Factors Number of Factors
Fig.2.PerformanceResultsofAUContheTestSetwithVaryingtheNumberofLatentFactors.
Fromtoptobottomandlefttoright,thedatasetsareGirls,Boys,andMen(duetolimitedspace
weomittheresultsonBaby,WomenandPhones).Forclarity,weomittheresultsofTBPR-Diff
wheretheyareslightlybetterthanBPR-MFandslightlyworsethanTBPR-Shared.