Table Of ContentA Probabilistic View of Neighborhood-based
Recommendation Methods
Jun Wang Qiang Tang
University of Luxembourg University of Luxembourg
[email protected] [email protected]
Abstract—Probabilistic graphic model is an elegant frame-
work to compactly present complex real-world observations by
modelinguncertaintyandlogicalflow(conditionallyindependent
7 factors). In this paper, we present a probabilistic framework of
1 neighborhood-basedrecommendationmethods(PNBM)inwhich
0 similarityisregardedasanunobservedfactor.Thus,PNBMleads
2 theestimationofuserpreferencetomaximizingaposteriorover
similarity. We further introduce a novel multi-layer similarity
n descriptorwhichmodelsandlearnsthejointinfluenceofvarious
a features under PNBM, and name the new framework MPNBM.
J
Empirical results on real-world datasets show that MPNBM
5 allows very accurate estimation of user preferences.
] I. INTRODUCTION
R
Collaborative filtering, which leverages user history infor-
I Fig.1:AgeneralstructureofNBM
. mation to predict users’ unknown preference, is one of the
s
c most successful techniques to build recommender systems
[ [17]. Matrix factorization (MF) [12] and neighborhood-based
strategies [5]. Similarity also serves as the basis for neighbor
1 methods (NBMs) [9] are two representative approaches. MF
selection, thus we concentrate upon similarity in this paper.
v family attracts more attention due to its ability of modeling
0 influence of various features (e.g. [22], [20], [11]), thus to Generally, there are two main approaches to compute similar-
5 ity. One introduces different kinds of correlation coefficients
improve accuracy. However, it is difficult to provide explain-
2 as similarity [9], such as Pearson and Cosine correlations.
able recommendation results. NBM family, shown as Fig.
1
However, some researchers argue that such kind of methods
0 1, is very popular mainly due to the fact that it naturally
isolate the relations between two items without leveraging
. explainsrecommendationresults(e.g.Anitemwhichissimilar
1
with what you bought before). Similarity serves as the basis global information. The other approach learns similarity via
0
regressionmodels.[7],[18]introduceawaytolearnsimilarity
7 of weighting neighbors which is crucial to the accuracy
1 of NBM recommender systems. However, existing similarity by minimizing mean squared error between observed ratings
: computation scheme is incapable of capturing influence from and their corresponding estimation. [18] factors similarity
v
i different features which hampers further polishing similarity matrix via low-rank approximations. [6] presents a weighted
X error function which gives more weight to the users who
to improve accuracy. In this paper, we first present a basic
r probabilistic framework of NBM family (PNBM) which leads rated items most similar to the estimated item. [14], [15]
a
learningsimilaritytoaregressionproblem.Thenweintroduce simplify standard neighborhood-based models to a simple
a novel multi-layer similarity descriptor which models and linear regression problem for top-N recommendation based
on binary databases.
learns the joint influence of different features under PNBM.
A number of probabilistic models have been introduced
A. Related Work
to collaborative filtering. However, only a very small portion
Commonly, NBMs are divided into two classes [9]. One is of them are NBM related. [15] presents a generic Bayesian
user-based approach which predicts the rating that a user will personalized ranking framework which is optimized for the
assign to an unrated item by referring to other users who are area under ROC (AUC) metric. [21] introduces a probabilistic
similar to this user. The other is item-based approach which memory-based collaborative filtering method in which they
estimatesauser’spreferencetoanunrateditembasedonother use a mixture Gaussian model built on the basis of a set of
itemsthataresimilartothisunrateditem.Thetwoapproaches user profiles and use the posterior distribution of user ratings
follow the same principle. for prediction. [8] builds a Markov network using Pearson-
With respect to NBM, researches have mainly focused on correlation NBM as basis. People also place probabilistic
similarity computation schemes [9] and neighbor selection prior assumptions to observations to model uncertainty. Such
as, [10] places Dirichlet distribution on the absolute value does not contain r . Moreover, we assume the testing set is
ui
of rating difference. [5] uses different probabilistic density excluded from the training set, when we predict rˆ in the
ui
functions to sample neighbors from a predefined similarity testing set, r in the training set is always zero.
ui
matrix (vectors).
III. PROBABILISTICFRAMEWORKOFNBM
Unfortunately, these models are incapable of representing
complex features, and none of them discusses NBM family
itself from a Bayesian perspective.
B. Contribution
In this paper, we present a probabilistic (Bayesian) frame-
work of NBM family, and our contribution is twofold.
• First,wepresentageneralgraphicalmodelofNBMfam-
ily(PNBM)whichleadstheestimationofuserpreference
to maximizing a posterior over similarity.
• Then, we introduce a novel multi-layer similarity de-
scriptor which is capable of modeling and learning the
jointinfluenceofvariousfeatures(e.g.rating,text,genre)
under PNBM, and we name the new framework as
MPNBM.
MPNBM is evaluated on three popular real-world datasets
Fig.2:GraphicalmodelofPNBM
via root-mean-square-error (RMSE) metric. Empirical results
show that MPNBM consistently outperform state-of-art ap-
In this section, we present a probabilistic graphical model
proaches on the datasets we choose.
of NBMs (PNBM), shown in Fig. 2. It is a Bayesian network
II. PRELIMINARY which describes the following factorization:
Suppose we have a data set organized in form of User×
Item matrix R ∈ RN×M, it contains N users and M items. p(Si,Ru−,rui,αS,αR)= (3)
S ∈ RM×M is item similarity matrix, sij denotes similarity p(Ru−)p(αS)p(αR)p(rui|Si,Ru−,αR)p(Si|αS)
betweenitemiandj,wefurtherassumes =s .I ∈BN×M
ij ji In our context, placing prior distribution on hyper-
presents indicator matrix, and B = {0,1}. I = 1 if user u
ui parameters Θ{α ,α } does not significantly improve accu-
rated item i, otherwise I = 0. R>0 ⊂ R denotes all the S R
ui racy while dramatically increasing time complexity. For the
observed ratings.
sakeofsimplicityandreductionoftimecomplexity,wesimply
So far, many neighborhood-based methods have been pro-
let p(α ), p(α ) be constant, and p(R−) is also constant. So
posed, as surveyed in [9]. For simplicity, we take a variant of S R u
we can simplify Equation (3) to
mean-centering NBM [16] as instance throughout the paper.
The predication formula is defined in Equation (1).
(cid:80) s (r(cid:48) −r¯ )I p(Si,Ru−,rui,αS,αR)∝p(rui|Si,Ru−,αR)p(Si|αS) (4)
rˆ(cid:48) =r¯ + j∈I\{i} ij uj j uj (1)
ui i (cid:80) |s |I We introduce a general Gaussian distribution (but not lim-
j∈I\{i} ij uj
itedto,otherdistributioncanbealsoappliedto.Itdependson
where ru(cid:48)j is rating score that user u gave to item j. rˆu(cid:48)i real-world context.) to density function p(∗) which naturally
denotes the estimation of user u’s preference on item i. r¯i leads to a sum-of-square-error.
is the mean value of all the ratings given to item i. I presents Morespecifically,assumethatanitem’ssimilarityvectorS
i
a set containing all the items. is independent from those of other items, and S is sampled
i
For further simplicity, Equation (1) is transformed into a from a mean-zero spherical Gaussian distribution. Thus we
vectorization form: have
(cid:80) M
rˆui = (cid:80)j∈I\{i}|ssijr|Iuj = |SSiR|Iu−− (2) p(S|αS)=(cid:89)N(Si|0,αS−1I) (5)
j∈I\{i} ij uj i u i=1
where rˆ = rˆ(cid:48) −r¯ and r = (r(cid:48) −r¯)I . S ∈ R1×M where N(x|µ,α−1) denotes the Gaussian distribution for x
ui ui i ui ui i ui i
with mean µ and precision α. We also assume that ratings are
denotes similarity vector corresponding to item i and R ∈
u
RN×1 represents rating vector of user u. The multiplication independent with each other. Combine with Equation (2), we
S R denotestheinnerproductofthetwovectors.I ∈BN×1 have following
i u u
itshaatndionedsicantootrcvoencttaoirnofanusietermu.wThheicshymisbboeli·n−gmpreeadnisctaedv.ecFtoorr p(R>0|S,R−,αR)=(cid:89)M (cid:89)N [N(rui||SSiR|Iu−−,αR−1)]Iui (6)
example, with regard to Equation (2), R− denotes a vector i=1u=1 i u
u
Fig.3:Multi-layersimilarity descriptor.Each
layer models an influence generated from Fig. 4: Graphical model of MPNBM. Note
features. that only black solid arrows (→) denote the
dependencyflowinthegraphicalmodel.
IV. MULTI-LAYERSimilarityDESCRIPTOR
where S =(cid:80)T φ(t)(Ω(t)◦Γ(t)). Follow the same assump-
i t=1 i i
In Section III, we introduced a general probabilistic tionsinSectionIII,wedefinetheprioroflayert(Ω(t)◦Γ(t))
(Bayesian) NBM framework which is simple and straight- i
as
forward. However, like other similarity computation methods, M
PNBM falls short in feature representation which extremely p(Ω(t)◦Γ(t)|α )=(cid:89)N(Ω(t)◦Γ(t)|0,α−1I) (9)
t i i t
limits the accuracy improvement. In this section, we present i=1
a multi-layer similarity descriptor (MLSD, shown in Fig.3 Andwehavetheconditionaldistributionoverobservedratings
) which is able to model and learn the joint influence of defined as
various features (e.g. ratings, text, genre, time). MLSD is
p(R>0|Ω(t),Γ(t),R−,α )=
mathematically defined as R
(cid:89)M (cid:89)N ((cid:80)T φ(t)Ω(t)◦Γ(t))R−
N[(r | t=1 u ,α−1)]Iui
S =(cid:88)T φ(t)(Ω(t)◦Γ(t)) (7) i=1u=1 ui ((cid:80)Tt=1φ(t)Ω(t)◦Γ(t))Iu− R
(10)
t=1
whereΓ(t) ∈RM×M denotesthesimilaritybasisatt-thlayer. V. MAXIMUMAPOSTERIOR
Ω(t) ∈ RM×M is Γ(t)’s constraint matrix which presents an
PNBM is a specific case of MPNBM, which only has one
influenceofobservedfeatures.Forexample,itcanpresentthe
layer with constraint-matrix set to 1. In this section, we take
similarity of text description ( or time closeness ) between
MPNBM as example to present how we optimize similarity
any two-item. Note that different influences may be generated
via maximizing a posterior.
from the same feature. φ(t) denotes the importance of the
The log of Bayesian network defined in Equation (8) is
feature-influence modeled at layer t. T is the number of
given by
layers (influence) employed to model similarity. In this paper,
we don’t require (cid:80)T φ(t) = 1, since we always have a logp(R−,r ,Γ(t),Ω(t),α ,α )∝
t=1 u ui i i t R
Enoqrumatailoinzat(i2o)n. fAac◦torB|Sdie|nIuo−tesinptohient-pwreisdeictpioronduecqtuaotpioenra,tiio.en. logp(r |S ,R−,α )+(cid:88)T p(Ω(t)◦Γ(t)|α ) (11)
ui i u R i i t
(Hadamard product) on matrices A and B, e.g. t=1
(cid:18) (cid:19) (cid:18) (cid:19) (cid:18) (cid:19) In fact, it defines the posterior distribution over similarity.
a a b b a b a b
11 12 ◦ 11 12 = 11 11 12 12 Combine it with Equation (9) and Equation (10), we have
a a b b a b a b
21 22 21 22 21 11 22 22
MLSD can be smoothly integrated into PNBM, shown in −logp(R−,R,Γ(t),Ω(t),α ,α )∝
i i t R
Fig. 4, named MPNBM. The Bayesian network is mathemat-
ically describe as αR (cid:88)M (cid:88)N (r − SiRu− )2+ αt (cid:88)T (cid:88)M (||Ω(t)◦Γ(t)|| )
2 ui |S |I− 2 i i 2
i=1u=1 i u t=1 i=1
p(Ru−,rpu(ri,uΓi|(iSt)i,,ΩR(iu−t),,ααRt,)α(cid:89)TR)p∝(Ω(it)◦Γ(it)|αt) (8) +M2(cid:88)i=T1log√α2tπ +log√α2Rπ (cid:88)iM=1u(cid:88)N=1Iui
(12)
t=1
TABLEI:DataSets
Maximizing the above Bayesian network distribution with
hyper-parameters being kept fixed is equivalent to minimizing dataset user# item# ratings# scales density
an error function defined as ML-20M 138,493 26,744 20,000,263 [0.5,5] 0.54%
ML-10M 69,878 1,0677 10,000,054 [0.5,5] 1.34%
Netflix 32,682 13,139 3,967,477 [1,5] 0.92%
E = 1(cid:88)M (cid:88)N (r − SiRu− )2+(cid:88)T (cid:88)M (λ ||Ω(t)◦Γ(t)|| ) Yahoo-R4 7,637 3,791 207,854 [1,5] 0.72%
2 ui |S |I− t i i 2
i=1u=1 i u t=1 i=1
(13)
[2]suchthateachuserrated50-1500movies,andeachmovie
where λt = 2ααtR is the regularization parameter for layer t. is rated by 5-1800 users. Yahoo-R4 is a subset of the movie-
A simple linear Gaussian model sometimes makes predic-
rating data set provided by the Yahoo Labs Webscope Team
tion value fall out of the range of valid rating values. In order
[4] such that each movie are at least rated by 5 users.
toforcethepredicationvaluestofallintovalidrange,wepass
• We use ML-10M, Netflix and Yahoo-R4 to compare the
thelinear-Gaussianmodelthroughhyperbolictangentfunction
h(x)= ex−e−x which makes prediction values be in range of models’ accuracy.
ex+e−x • We also compare each model’s accuracy on data sets
[-1,1]. We map the centralized ratings to range [-1, 1] with
with different densities. In order to avoid the inherent
Equation (14).
differences of data sets from different originations, we
x− maxx+minx extract 10 subsets from one single data set (ML-20M)
t(x)= 2 (14)
maxx− maxx+2minx based on the users’ rating number. Precisely, each subset
has similar amount of users and items, the number of
where max and min are the max and min value of
x x users and items are in range [10000, 15000] and [8000,
ratings, respectively. Since the ratings are centralized by their
20000] respectively. The density of each data set is from
corresponding mean value, we always have max > 0 and
x 0.28% to 2.67%.
min < 0. As a result, the range of valid rating value align
x
with the estimation produced by our models. B. Models for Comparison
The conditional distribution of observed ratings becomes
In this paper, the following models are compared:
p(R>0|S,R−,α )=(cid:89)M (cid:89)N [N(r |h(SiRu− ),α−1)]Iui • RegSim: Regression on similarity [18], a representative
R ui |S |I− R work which learns similarity via a regression method.
i=1u=1 i u (15) • SLIM: Sparse linear methods [14], a regression model
fortop-N recommendationonbinarydataset.Weextend
We adopt stochastic gradient descent (SGD) as learning
it to a arbitrary real-value prediction model by placing a
algorithm to train latent factors, shown in Algorithm 1.
jointly Gaussian-Laplace prior on similarity vectors. It
Algorithm 1 Training via Stochastic Gradient Descent has a very similar error function as SLIM,
PInrietliiamliiznaatiroyn::rastiimngilamriatytrixbaRsi,serΓr(otr),fuinncfltuioenn.ce constraint- E =12 (cid:88)N (cid:88)M (rui− |SSiR|Iu−−)2Iui
matrix Ω(t), influence importance factor φ(t), learning rate u=1i=1 i u (16)
βΓ,(t)r)e.gular parameter λt. Note that Si = (cid:80)Tt=1φ(t)(Ω(it) ◦ + λ2S (cid:88)M ||Si||2−λSµ(cid:88)M ||Si||1
i
• Training: i=1 i=1
for k =1 to K do where µ is a non-zero mean value of the Gaussian prior.
• For each layer (t), point-wisely update the similarity • PCC: NBMusingPearsoncorrelationassimilarity[16].
basis Γ(t): • COS: NBM using Cosine correlation as similarity [9].
ij
[Γ(ijt)]new =[Γ(ijt)]old−βeui∂∂Γrˆ(iujti) −βλt(Ω(ijt)◦Γ(ijt)) •• PMMPFN:BMPr:obaIbnilitshtiics mpaaptreixr,fwacetoreixzpatliooint [in3fl].uence from
• where e =rˆ −r .
ui ui ui ratings as an instance to demonstrate MLSD’s ability of
end for
modeling various features, thus to improve accuracy. We
Prediction: prediction using Equation (1) with top-200 the
use a 3-layer similarity descriptor in which layer-1 treats
most similar neighbors.
latent influence equally with constraint-matrix set to 1;
Layer-2 adopts Pearson correlation as constraint-matrix
that stresses the influence from those items which either
VI. EXPERIMENTS
have significant positive correlation or strong negative
A. Description of Data Sets
correlation with the item under predication; Layer-3 em-
In the experiments, we evaluate our models and state-of-art ploys Jaccard index to form a constraint-matrix that am-
methods over three different data sets, summarized in Table I. plifies the influence from those items which have similar
ML-20M, ML-10M are data sets provided by MovieLens rating history, alleviates the divergence from infrequent-
[1]. Netflix is a subset sampled from Netflix Prize data set rated items and frequent-rated items. Time Complexity.
Fig.5:RMSEevaluationonML-10M,Netflix,Yahoo-R4.TheY-axisdisplaysRMSEvalueandtheX-axis
showsthenumberofepochs(iterations)inthetraining.
Fig.6:RMSEevaluationwithdifferentdensity.Leftpanel:Basicinformationofthe10subsetsextracted
fromML-20M.Rightpanel:TheRMSEevaluationoneachsubset,theY-axisdisplaysRMSEvalue.The
X-axisofbothpanelsdisplaysthenumberofrateditemsperuserineachsubset.
The computational time is mainly taken by updating machine with 16 GB memory.
similarity. At a single epoch, approximately T ·L·#
u
similarities are updated, where L is the size of training
C. Parameters Setting
set, # is the average rating number per users and T is
u
thenumberofinfluencelayers.Intuitively,asingleepoch For RegSim, MPNBM, Tanh-MPNBM and SLIM, we em-
takes about 4, 260, 340 seconds on Yahoo-R4, Netflix, pirically choose parameters for each model after a grid search
ML-10M respectively. in which β ∈ {0.05,0.1,0.2,0.3,0.4,0.5},λ = λ = λ ∈
1 2 3
• Tanh-MPNBM: The model which we pass MPNBM {0.01,0.02,0.03,0.04,0.05,0.06,0.1}.Thefinallychosenpa-
through hyperbolic tangent function ( detailed in Section rametersaresummarizedinTableII(⊥indicatesamodeldoes
V ). not have such a parameter).
Experiment setting. All models are implemented with Mat- For PMF, we choose latent feature dimension D = 10
lab,andrunonasinglecoreofaIntel(R)Xeon(R)3.50GHz and the momentum of mini-batch SGD η = 0.8. Regularized
TABLE II: Parameters Setting for RegSim,
MPNBM,Tanh-MPNBM,SLIM.
β (λ1,λ2,λ3) (φ(1),φ(2),φ(3))
RegSim 0.1 (0.01,⊥,⊥) (⊥,⊥,⊥)
MPNBM 0.2 (0.05,0.05,0.05) (3,1,1)
Tanh-MPNBM 0.4 (0.05,0.05,0.05) (3,1,1)
SLIM 0.4 (0.02,⊥,⊥) (⊥,⊥,⊥)
parameters (λ ,λ for user latent factors and latent item
P Q
factors respectively) and learning rate β are set to
• For Yahoo-R4, λP =λQ =0.05 and β =0.0005.
• For Netflix, λP =λQ =0.002 and β =0.0002.
• For ML-10M, λP =λQ =0.02 and β =0.0002.
• ForthefirsttwoML-20Msubsetsshownintheleftpanel
of Fig. 6, λ =λ =0.01 and β =0.0002.
P Q
• For the other eight ML-20M subsets shown in the left
panel of Fig. 6, λ =λ =0.02 and β =0.0002.
P Q
For PCC and COS, we use top-200 the most similar Fig.7:Tanh-MPNBM:Comparisonbetween
neighbors for prediction. twostrategiesofsettingφ.
D. Comparison Results
During the test, we randomly divide each data set into strategies to select parameters φ for each layer, 1) we con-
training set (85%), validation set (5%) and testing set (10%). sistently choose φ(1) = 3, φ(2) = φ(3) = 1 for all the three
We adopt RMSE for evaluation. We repeat the experiments 5 data sets, named TMPN-V1; 2) letting φ(t) ∈ {1,1.5,2}, we
times. assignhighervaluetotheφwhichcorrespondingΩhaslower
RMSE,namedTMPN-V2.ThecomparisonisshowninFig.7,
TABLE III: Accuracy Comparison (The
and 1) the accuracy is not significantly influenced, MPNBM
smaller RMSE, the better accuracy for
(Tanh-MPNBM)isabletobalancetheinfluenceautomatically.
recommendation.). MPNB, TMPN, denote
2) Assigning proper weight to φ according to the RMSE of Ω
MPNBMandTanh-MPNBMrespectively.
results in faster convergence.
Yahoo-R4 Netflix ML-10M 2) Stability: Model based approach may easily over fit
RMSE INC% RMSE INC% RMSE INC% when increasing the number of parameters under training.
RegSim 0.9723 0 0.8713 0 0.8034 0 The system can be beneficent from the stability of algorithms
MPNB 0.9641 0.84 0.8425 3.31 0.7941 1.16
which is defined by
TMPN 0.9629 0.97 0.8363 4.02 0.7955 0.98
SLIM 0.9725 -0.02 0.8731 -0.21 0.8004 0.37 • converge speed: the first epoch where a model converges
PMF 1.0608 -9.1 0.8826 -1.3 0.7957 0.96 to the local best solution, denoted as (cid:15).
PCC 0.9813 -0.93 0.8620 1.07 0.8121 -1.08 • abilityofmodelstomaintaintheirbeststatus:thenumber
COS 0.9722 0.01 0.8605 1.24 0.8362 -4.08 of epochs that a model stays in the local best solution,
denoted as ζ.
1) Accuracy: The comparison is performed over : Table IV shows the values of (cid:15) and ζ of each model over
• Accuracy on different data sets. different data sets. In the comparison of stability, we treat
• Accuracy on different density.
TABLEIV:StabilityComparison
Fig. 5 presents the detail of training on different data sets.
Table III records the final accuracy comparison (the training Yahoo-R4 Netflix ML-10M
process is conducted by validation set). RegSim is selected as (cid:15) ζ (cid:15) ζ (cid:15) ζ
RegSim 86 102 134 ≥67 96 91
baseline model, the accuracy improvement of each model is
MPNBM 84 59 * 141 ≥60
displayed in the INC % column. Tanh-MPNBM 158 ≥43 166 ≥35 115 ≥86
Fig. 6 shows the accuracy comparison on data sets with SLIM 39 51 186 ≥15 150 ≥51
PMF 87 6 64 12 93 34
different density. MPNBM and Tanh-MPNBM consistently
outperformoutperformstate-of-artmodels,especiallyonthose
RMSE values x =x , if |x −x |≤0.0001. Note that with
extremely sparse data sets ( which have serious cold start 1 2 1 2
regard to a model which does not over fit after 200 epochs
problem). For simplicity, we don’t draw SLIM on the graph,
(valueofζ prefixedwith≥),ifthelowestRMSEvalueappears
since SLIM has similar accuracy with RegSim.
at least 10 epochs, it is seen as the local best solution. * in
We are also interested in that how the layer importance- Table IV means a model does not converge after 200 epochs
factor φ affects the MPNBM (Tanh-MPNBM). We use two on a data set. e.g, MPNBM does not converge on Netflix data
set, also shown in Fig. 5. The experimental results show that [12] Y.Koren,R.Bell,andC.Volinsky. Matrixfactorizationtechniquesfor
MPNBMandTanh-MPNBMstayinthelocalbestsolutionfor recommendersystems. Computer,(8):30–37,2009.
[13] Z.Liu,Y.-X.Wang,andA.Smola.Fastdifferentiallyprivatematrixfac-
many(>40)epochswhichisbetterthanPMF.Withregardto
torization.InProceedingsofthe9thACMConferenceonRecommender
convergespeed,asshowninTableIV,itseemsthatsometimes Systems,pages171–178.ACM,2015.
MPNBM and Tanh-MPNBM do not converge as fast as PMF. [14] X. Ning and G. Karypis. Slim: Sparse linear methods for top-n
recommender systems. In Data Mining (ICDM), 2011 IEEE 11th
Infact,theyachieveaconsiderableaccuracyatamuchearlier
InternationalConferenceon,pages497–506.IEEE,2011.
epoch. [15] S.Rendle,C.Freudenthaler,Z.Gantner,andL.Schmidt-Thieme. Bpr:
Bayesianpersonalizedrankingfromimplicitfeedback. InProceedings
VII. CONCLUSION of the twenty-fifth conference on uncertainty in artificial intelligence,
pages452–461.AUAIPress,2009.
In this paper, we have presented a probabilistic framework
[16] P.Resnick,N.Iacovou,M.Suchak,P.Bergstrom,andJ.Riedl. Grou-
of NBM family, and introduced a multi-layer similarity de- plens: an open architecture for collaborative filtering of netnews. In
scriptorunderPNBMwhichiscapableofmodelingandlearn- Proceedings of the 1994 ACM conference on Computer supported
cooperativework,pages175–186.ACM,1994.
ing the joint influence of various features. Our experiments
[17] X. Su and T. M. Khoshgoftaar. A survey of collaborative filtering
show that MPNBM and Tanh-MPNBM allow accurate and techniques. Advancesinartificialintelligence,2009:4,2009.
stable estimation of user preferences. [18] A. To¨scher, M. Jahrer, and R. Legenstein. Improved neighborhood-
basedalgorithmsforlarge-scalerecommendersystems. InProceedings
Privacy is a serious problem to recommender systems.
of the 2nd KDD Workshop on Large-Scale Recommender Systems and
Nowadays, applying differential privacy to recommendation theNetflixPrizeCompetition,page4.ACM,2008.
algorithms attracts great attention. A common approach is [19] Y.-X. Wang, S. E. Fienberg, and A. Smola. Privacy for free: Pos-
terior sampling and stochastic gradient monte carlo. arXiv preprint
adding noise to data set. Recently, people find that sampling
arXiv:1502.07645,2015.
from a posterior distribution achieves some extent of differ- [20] J. Yang, Z. Sun, A. Bozzon, and J. Zhang. Learning hierarchical
ential privacy “for free” [19], and this idea has been already feature influence for recommendation by recursive regularization. In
Proceedingsof10thACMConferenceonRecommenderSystems.(RecSys
successfully applied to probabilistic matrix factorization [13].
2016).ACM,2016.
Following the same idea, our models can also provide such [21] K.Yu,A.Schwaighofer,V.Tresp,X.Xu,andH.-P.Kriegel.Probabilistic
kind of “free privacy”. We leave a detailed investigation as memory-basedcollaborativefiltering.KnowledgeandDataEngineering,
IEEETransactionson,16(1):56–69,2004.
future work.
[22] Y. Zheng, B. Mobasher, and R. Burke. Incorporating context cor-
relation into context-aware matrix factorization. In Proceedings of
VIII. ACKNOWLEDGMENTS the 2015 International Conference on Constraints and Preferences for
ConfigurationandRecommendationandIntelligentTechniquesforWeb
Both authors are supported by a CORE (junior track) grant
Personalization-Volume1440,pages21–27.CEUR-WS.org,2015.
fromtheNationalResearchFund,Luxembourg.QiangTangis
alsopartially supported byaninternal projectfromUniversity
of Luxembourg.
REFERENCES
[1] Movielens,http://grouplens.org/datasets/movielens/.
[2] Netflixprize,https://www.netflix.com.
[3] http://www.utstat.toronto.edu/∼rsalakhu/BPMF.html.
[4] Yahoo!: Webscope movie data set (version 1.0),
http://research.yahoo.com/.
[5] P. Adamopoulos and A. Tuzhilin. On over-specialization and concen-
tration bias of recommendations: Probabilistic neighborhood selection
in collaborative filtering systems. In Proceedings of the 8th ACM
ConferenceonRecommendersystems,pages153–160.ACM,2014.
[6] R.Bell,Y.Koren,andC.Volinsky. Modelingrelationshipsatmultiple
scalestoimproveaccuracyoflargerecommendersystems. InProceed-
ingsofthe13thACMSIGKDDinternationalconferenceonKnowledge
discoveryanddatamining,pages95–104.ACM,2007.
[7] R. M. Bell and Y. Koren. Scalable collaborative filtering with jointly
derived neighborhood interpolation weights. In Data Mining, 2007.
ICDM 2007. Seventh IEEE InternationalConference on, pages 43–52.
IEEE,2007.
[8] A. Defazio and T. Caetano. A graphical model formulation of col-
laborativefilteringneighbourhoodmethodswithfastmaximumentropy
training. arXivpreprintarXiv:1206.4622,2012.
[9] C.DesrosiersandG.Karypis.Acomprehensivesurveyofneighborhood-
based recommendation methods. In Recommender systems handbook,
pages107–144.Springer,2011.
[10] G. Guo, J. Zhang, and N. Yorke-Smith. A novel bayesian similarity
measureforrecommendersystems. InProceedingsoftheTwenty-Third
international joint conference on Artificial Intelligence, pages 2619–
2625.AAAIPress,2013.
[11] Y. Koren. Factorization meets the neighborhood: a multifaceted col-
laborative filtering model. In Proceedings of the 14th ACM SIGKDD
international conference on Knowledge discovery and data mining,
pages426–434.ACM,2008.
