Global Optimization Simulated Annealing and Tabu Search

24 Pages·2007·0.58 MB·English

Global Optimization
Simulated Annealing and Tabu Search
Doron Pearl
1
A Greedy Approach
Iteratively minimize current state, by replacing it by
(cid:122)
successor state that has lower value.
When successor is higher – return.
(cid:122)
2
A Greedy Approach
Problem
Most Minimization strategies find the
nearest local minimum.
barrier to local search
starting
point
descend
direction
local minimum
global minimum
3
Bouncing Ball
Goal: Find the lowest valley in a terrain.
Approach: A bouncing ball.
Process
At the beginning allow the ball to make high bounces.
1.
Slowly decreases the maximum bounce.
2.
Will jump barrier
starting
point
descend
direction
local minimum
4
global minimum
Physical Annealing Process
Gradual cooling of liquid …
(cid:122)
At high temperatures, molecules move freely
At low temperatures, molecules are "stuck“
if cooling is slow
(cid:122)
Low energy, organized
crystal lattice formed.
5
Simulated Annealing
A stochastic global optimization method that
(cid:122)
distinguishes between different local optima.
Derived its name from the annealing process used
(cid:122)
to re-crystallize metals.
Simulated annealing is summarized with the
(cid:122)
following idea:
“When optimizing a very large and complex system
(i.e., a system with many degrees of freedom) ,
instead of always going downhill, try to go downhill
most of the times” [Haykin, 1999]
6
Algorithm’s Sketch
Step 1: Initialize – Start with a random initial
placement. Initialize a very high
“temperature”.
Step 2: Move – Perturb the placement through a
defined move.
Step 3: Calculate score – calculate the change in the
score due to the move made.
Step 4: Choose – Depending on the change in score,
accept or reject the move. The probability of
acceptance depending on the current
“temperature”.
Step 5: Update and repeat– Update the temperature
value by lowering the temperature. Go back to
Step 2.
7 The process is done until “Freezing Point” is reached.
Basic Algorithm
1. Choose a random X, select the initial system
i
temperature, and outline the cooling (ie. annealing)
schedule.
2. Evaluate E(X) using a simulation model
i
3. Perturb X to obtain a neighboring Design Vector (X )
i i+1
4. Evaluate E(X ) using a simulation model
i+1
5. If E(X )< E(X), X is the new current solution
i+1 i i+1
6. If E(X )> E(X), then accept X as the new current
i+1 i i+1
solution with a probability e(-Δ /T) where Δ = E(X ) -E(X).
i+1 i
7. Reduce T according to the cooling schedule.
8. Terminate the algorithm.
8
T = System Temperature , E = Objective function
Rosenbrock Function
SA’s performance on the Rosenbrock function (2D):
f = [(1− x)2 +100× x − x 2]2
2 1
Objective reduction
Search Pattern
9
General SA Software Overview
Initial
configuration X
0 X
Simulated Annealing best
Options Algorithm
X
(e.g. annealing
history
schedule)
Evaluation Perturbation
function e(x) function p(x)
10