What is Simulated Annealing? - Department of Aerospace Engineering

19 Pages·2004·0.21 MB·English

An Introduction to Simulated Annealing
An Introduction to Simulated Annealing
Rajkumar Pant
Aerospace Engineering Department
IIT Bombay
PDF created with pdfFactory Pro trial version www.pdffactory.com
PRESENTATION OUTLINE
PRESENTATION OUTLINE
HISTORICAL BACKGROUND
(
GLOBAL OPTIMIZATION OF MULTI-MODAL FUNCTIONS
(
MARBLE IN CUBE ANALOGY
(
S A ALGORITHM
(
FEATURES OF S A
(
TUNING OF S A PARAMETERS
(
S A V/S CONVENTIONAL METHODS
(
S A FOR FUNCTIONS OF CONTINUOUS VARIABLES
(
APPLICATIONS OF S A
(
IMPROVEMENTS TO THE BASIC ALGORITHM
(
SIMANN S A ALGORITHM
(
PDF created with pdfFactory Pro trial version www.pdffactory.com
GLOBAL OPTIMIZATION OF MULTI-MODAL FUNCTIONS
( How to reach F, starting from any x ?
Greedy Algorithms ==> nearest local minimum
u
A ==> BC ==> DE ==> FG ==> H
s
multiple starts from various initial x values
s
E
A
C
G
f(x)
B
D
H
F
x
PDF created with pdfFactory Pro trial version www.pdffactory.com
NON-GREEDY ALGORITHMS
( Permit occasional uphill moves
sparingly, and in a controlled manner
u
( Large uphill moves
In the initial stages
u
–better domain exploration
Large changes in f(x)
u
–better chance of improvement
Once in a while
u
–to climb out of local minima
PDF created with pdfFactory Pro trial version www.pdffactory.com
HISTORICAL BACKGROUND
( Numerical simulation of AnnealingMetropolis et. al, 1953
p(dE) = e(- dE/kT)
T = temperature
p(d E) = probability of an increase in energy by d E
k = Boltzmann’sconstant
( Combinatorial OptimizationKirkpatrick et. al, 1980
Cerny, 1985
Thermodynamic SimulationCombinatorial optimization
System StatesFeasible solutions
EnergyCost
Change of stateNeighboring solution
TemperatureControl Parameter
Frozen stateHeuristic Solution
PDF created with pdfFactory Pro trial version www.pdffactory.com
MARBLE-IN-CUBE ANALOGY
How to take the marble to the lowest position in the cube ?
PDF created with pdfFactory Pro trial version www.pdffactory.com
SA ALGORITHM
Solution space X
Objective function f
Neighborhood structure N
Select Initial point s
0
Select Initial temperature T > 0
0
Select temperature reduction function a
Repeat
Repeat
Randomly select s ˛N(s )
0
d f = f(s) -f(s )
0
If d f < 0
then s = s
0
else
generate a random number r ˛(0,1)
if r < e(- d f / T ) then s = s
0 0
Until iteration_count = max. iteration
Set T = a (T)
Until stopping condition = TRUE
s is the approximation to the global minimum solution
0
PDF created with pdfFactory Pro trial version www.pdffactory.com
FEATURES OF SA
( Direct Method & Non-Greedy algorithm
Global optimization of multi-modal, discontinuous & noisy functions
u
( Mathematically proven to converge to global optimum
( Very simple architecture
( Parameters to be decided
Solution space X, Objective function f
u
user defined
s
Neighborhood structure N
u
should be adaptively modified
s
Initial point s
u
0
can be randomly selected
s
Initial temperature T & Temperature reduction function a
u
0
ensure proper “annealing”
s
Stopping Criteria
u
max. number of function evaluations
s
minimum improvement in f acceptable
s
PDF created with pdfFactory Pro trial version www.pdffactory.com
SA V/S CONVENTIONAL METHODS
(Very large number of function evaluations
nearly 1000 times more !
u
Exact optimal solution not reached in finite time
u
(Tuning of SA parameters required before starting
may take up 50% of the total time !
u
(Cannot implicitly handle constraints
Penalty Function approach
u
PDF created with pdfFactory Pro trial version www.pdffactory.com
Example of Penalty Function
Objective Function = F + Σ P
objt k
P = iv iact w constr
k k k k k
k = number of constraints
s
constr = numerical value of kth constraint
s
k
iv = 1 if constr > tol , = 0 otherwise
s
k k k
tol = tolerance on target value for kth constraint
s
k
iact = 1 if kth constraint is active, = 0 otherwise
s
k
w = weight on the value of kth constraint
s
k
PDF created with pdfFactory Pro trial version www.pdffactory.com