Another introduction to Gaussian Processes

152 Pages·2017·4.03 MB·English

by  Richard Wilkinson

Another introduction to Gaussian Processes
Richard Wilkinson
SchoolofMathsandStatistics
UniversityofSheffield
GP summer school
September 2017
A stochastic process is a collection of random variables indexed by some
variable x
∈ X
f = f(x) : x
{ ∈ X}
Usually f(x) R and = Rn i.e. f can be thought of as a function of
∈ X
location x.
f is an infinite dimensional process.
Thankfully we only need consider the finite dimensional distributions
(FDDs), i.e., for all x ,...x and for all n N
1 n
∈
P(f(x ) y ,...,f(x ) y )
1 1 n n
≤ ≤
as these uniquely determine the law of f.
A Gaussian process is a stochastic process with Gaussian FDDs, i.e.,
(f(x ),...,f(x )) N (µ,Σ)
1 n n
∼
Why would we want to use this very restricted class of model?
Why use Gaussian processes?
Usually f(x) R and = Rn i.e. f can be thought of as a function of
∈ X
location x.
f is an infinite dimensional process.
Thankfully we only need consider the finite dimensional distributions
(FDDs), i.e., for all x ,...x and for all n N
1 n
∈
P(f(x ) y ,...,f(x ) y )
1 1 n n
≤ ≤
as these uniquely determine the law of f.
A Gaussian process is a stochastic process with Gaussian FDDs, i.e.,
(f(x ),...,f(x )) N (µ,Σ)
1 n n
∼
Why would we want to use this very restricted class of model?
Why use Gaussian processes?
A stochastic process is a collection of random variables indexed by some
variable x
∈ X
f = f(x) : x
{ ∈ X}
f is an infinite dimensional process.
Thankfully we only need consider the finite dimensional distributions
(FDDs), i.e., for all x ,...x and for all n N
1 n
∈
P(f(x ) y ,...,f(x ) y )
1 1 n n
≤ ≤
as these uniquely determine the law of f.
A Gaussian process is a stochastic process with Gaussian FDDs, i.e.,
(f(x ),...,f(x )) N (µ,Σ)
1 n n
∼
Why would we want to use this very restricted class of model?
Why use Gaussian processes?
A stochastic process is a collection of random variables indexed by some
variable x
∈ X
f = f(x) : x
{ ∈ X}
Usually f(x) R and = Rn i.e. f can be thought of as a function of
∈ X
location x.
Thankfully we only need consider the finite dimensional distributions
(FDDs), i.e., for all x ,...x and for all n N
1 n
∈
P(f(x ) y ,...,f(x ) y )
1 1 n n
≤ ≤
as these uniquely determine the law of f.
A Gaussian process is a stochastic process with Gaussian FDDs, i.e.,
(f(x ),...,f(x )) N (µ,Σ)
1 n n
∼
Why would we want to use this very restricted class of model?
Why use Gaussian processes?
A stochastic process is a collection of random variables indexed by some
variable x
∈ X
f = f(x) : x
{ ∈ X}
Usually f(x) R and = Rn i.e. f can be thought of as a function of
∈ X
location x.
f is an infinite dimensional process.
A Gaussian process is a stochastic process with Gaussian FDDs, i.e.,
(f(x ),...,f(x )) N (µ,Σ)
1 n n
∼
Why would we want to use this very restricted class of model?
Why use Gaussian processes?
A stochastic process is a collection of random variables indexed by some
variable x
∈ X
f = f(x) : x
{ ∈ X}
Usually f(x) R and = Rn i.e. f can be thought of as a function of
∈ X
location x.
f is an infinite dimensional process.
Thankfully we only need consider the finite dimensional distributions
(FDDs), i.e., for all x ,...x and for all n N
1 n
∈
P(f(x ) y ,...,f(x ) y )
1 1 n n
≤ ≤
as these uniquely determine the law of f.
Why would we want to use this very restricted class of model?
Why use Gaussian processes?
A stochastic process is a collection of random variables indexed by some
variable x
∈ X
f = f(x) : x
{ ∈ X}
Usually f(x) R and = Rn i.e. f can be thought of as a function of
∈ X
location x.
f is an infinite dimensional process.
Thankfully we only need consider the finite dimensional distributions
(FDDs), i.e., for all x ,...x and for all n N
1 n
∈
P(f(x ) y ,...,f(x ) y )
1 1 n n
≤ ≤
as these uniquely determine the law of f.
A Gaussian process is a stochastic process with Gaussian FDDs, i.e.,
(f(x ),...,f(x )) N (µ,Σ)
1 n n
∼
Why use Gaussian processes?
A stochastic process is a collection of random variables indexed by some
variable x
∈ X
f = f(x) : x
{ ∈ X}
Usually f(x) R and = Rn i.e. f can be thought of as a function of
∈ X
location x.
f is an infinite dimensional process.
Thankfully we only need consider the finite dimensional distributions
(FDDs), i.e., for all x ,...x and for all n N
1 n
∈
P(f(x ) y ,...,f(x ) y )
1 1 n n
≤ ≤
as these uniquely determine the law of f.
A Gaussian process is a stochastic process with Gaussian FDDs, i.e.,
(f(x ),...,f(x )) N (µ,Σ)
1 n n
∼
Why would we want to use this very restricted class of model?
Property 1: X N (µ,Σ) if and only if AX N (Aµ,AΣA ) for all
n p (cid:62)
∼ ∼
p n constant matrices A.
×
So sums of Gaussians are Gaussian, and marginal distributions of
multivariate Gaussians are still Gaussian.
Why use Gaussian processes?
Gaussian distributions have several properties that make them easy to
work with:
So sums of Gaussians are Gaussian, and marginal distributions of
multivariate Gaussians are still Gaussian.
Why use Gaussian processes?
Gaussian distributions have several properties that make them easy to
work with:
Property 1: X N (µ,Σ) if and only if AX N (Aµ,AΣA ) for all
n p (cid:62)
∼ ∼
p n constant matrices A.
×