Graphical)Models - University of Massachusetts Amherst

33 Pages·2011·1.29 MB·English

Graphical
 Models
Lecture
 12:

Belief
 Update
 Message
 Passing
Andrew
 McCallum
[email protected]
Thanks
 to
 Noah
 Smith
 and

 Carlos
 Guestrin
 for
 slide
 materials. 1
Today’s
 Plan
• Quick
 Review:

 Sum
 Product
 Message
 Passing
 (also

known
 as
 “Shafer-­‐Shenoy”)
• Today:

 Sum
 Product
 Divide
 Message
 Passing
 (also

known
 as
 “belief
 update”
 message
 passing
 “Lauritzen-­‐
Spiegelhalter”
 and
 “belief
 propagaFon”)
• MathemaFcally
 equivalent,
 but
 different
 intuiFons.
• Moving
 toward
 approximate
 inference.
ϕ ϕ
Quick
 Review F A
ϕ
Flu All.
• {H,
 R,
 S,
 A,
 F} FAS
S.I.
ϕ ϕ
SR SH
ϕ
SR
ψ R.N. H.
SR
SR
τ
S
ϕ
F
ϕ
A ν
ϕ ϕ FAS
SH FAS
ψ
SH
HS SAF
ν (C ) = φ
j j
τ
S ψ φ Φ
FAS ￿∈ j
Message
 Passing
 (One
 Root)
• Input:

 clique
 tree
 T,
 factors
 Φ,
 root
 C
r
• For
 each
 clique
 C ,
 calculate
 ν

i i
• While
 C
 is
 sWll
 waiWng
 on
 incoming
 messages:
r
– Choose
 a
 C
 that
 has
 received
 all
 of
 its
 incoming

i δ = ν δ
i j i k i
→ →
messages. C S k Neighbors j
i￿\ i,j ∈ ￿ i\{ }
– Calculate
 and
 send
 the

message
 from
 C
 to
 C
i upstream-­‐neighbor(i)β = ν δ
r r k r
→
k Neighbors
∈ ￿ r
= φ
X C φ Φ
￿\ i ￿∈
= Z P(C )
r
·
Different
 Roots;
 Same
 Messages
δ
SAF→SR
β
SR
δ
SR
SR→SAF
HS SAF root
β
β
SAF
HS δ
HS→SAF
δ
SAF→HS
Sum-­‐Product
 Message
 Passing
• Each
 clique
 tree
 vertex
 C
 passes
 messages
 to

i
each
 of
 its
 neighbors
 once
 it’s
 ready
 to
 do
 so.
• At
 the
 end,
 for
 all
 C :
i
β = ν δ
i i k i
→
k Neighbors
￿ i
∈
– This
 is
 the
 unnormalized
 marginal
 for
 C .
i
Calibrated
 Clique
 Tree
• Two
 adjacent
 cliques
 C
 and
 C
 are
 calibrated

i j
when:
β = β
i j
C S C S
i￿i,j j￿i,j
\ \
= µ (S )
i,j i,j
β β
i j
μ
C i,j C
i j
S
 =
i,j
C
 ∩
 C
i j
Calibrated
 Clique
 Tree
 as
 a

• Original
 (unnormalized)
 factor
 model
 and

calibrated
 clique
 tree
 represent
 the
 same

(unnormalized)
 measure:
clique
 beliefs
β
C
C Vertices( )
￿
φ = ∈ T
µ
φ Φ S
￿
∈
S Edges( )
￿
∈ T sepset
 beliefs
unnormalized
 Gibbs
distribuWon
 from
 original
factors
 Φ
Inventory
 of
 Factors
• original
 factors
 ϕ
• iniWal
 potenWals
 ν
• messages
 δ
• intermediate
 factors
 ψ
 (no
 longer
 explicit)
• clique
 beliefs
 β
• sepset
 beliefs
 μ
Inventory
 of
 Factors
• original
 factors
 ϕ
• iniWal
 potenWals
 ν
• messages
 δ
• intermediate
 factors
 ψ
 (no
 longer
 explicit)
• clique
 beliefs
 β
• sepset
 beliefs
 μ New
 algorithm

collapses

everything
 into

beliefs!