Genetic Programming II: Automatic Discovery of Reusable Programs (Complex Adaptive Systems)

768 Pages·1994·47.22 MB·English

by  John R. Koza

GeneticP rogrammingI I
Complex Adaptive Systems
Iotur H. Holland, Christopher Langton, and StewartW . Wilson, advisors
Adaptationin Naturala ndA rtificinlS ystemsA:n IntroductoryA nalysisw ith
Applicationsto BiologyC, ontrol,a ndA rtificial Intelligence
IokurH . Holland
Tbwarda Practiceo f AutonomousS ystemsP: roceedingosf theF irst European
Conferencoen Artificinl Life
edited by FranciscoJ .V arelaa nd Paul Bourgine
GmeticP rogrammingO: n theP rogrammingo f Computersb y Meanso f Natural
Selection
Iotrr R. Koza
FromA nimalst o Animats2 : Proceedingosf theS econdIn ternationaCl onference
on Simulationo f AdaptfueB ehnaior
edited by |ean-Arcady Meyeq,H erbert L. Roitblat, and StewartW . Wilson
IntelligentB ehnuioirn Animalsa nd Robots
David McFarland and ThomasB cisser
Adaancesin GeneticP rogramming
edited by Kenneth E. Kinnear,Jr.
GeneticP rogrammingII : AutomaticD iscoaeroy f ReusablPer ograms
Iohr, R. Koza
Also Available:
GeneticP rogrammingT: heM oaie
Iottn R. Koza and ]amesP .R ice
GeneticP rogrammingII VideotapeT:h eN ext Generation
Iotrr R. Koza and james P.R ice
Genetic Programming II
AutomaticD iscoveryo f
ReusableP rograms
]otur R. Koza
A Bradford Book
The MIT Press
Cambridge,M assachusetts
London, England
@1994M assachusettsIn stitute of Technology
All rights reserved. No part of this book may be reproduced in any form by any electronico r
mechanicalm eans( including photocopying, recording,o r information storageo r retrieval)
without permission in writing from the publisher.
Seti n Palatinob y ProteusT ypography,P alo Alto, California.
Printed and bound in the United Stateso f America.
The programs,p rocedures,a nd applicationsp resentedi n this book have been included for
their instructional value. The publisher and the author offer NO WARRANTY OF FITNESS
OR MERCHANTABILITY FOR ANY PARTICULARP URPOSEa nd do not accepta ny
liability with respectt o thesep rograms,p rocedures,a nd applications.
Library of CongressC ataloging-in-PublicationD ata
Library of CongressC atalog Card Number 94-76375
r
to my mother and father

CONTENTS
Preface xvii
Acknowledgments xix
1 Introduction 1
2 Background on Genetic Algorithms, LISP,
and Genetic Programming 21
3 HierarchicalProblem-Solving 45
4 Introduction to Automatically Defined Functions -
The TWo-BoxesP roblem 57
5 Problems that Straddle the Breakeven
Point for Computational Effort 109
6 Boolean Parity Functions t57
7 Determining the Architecture of the Program 20r
8 The Lawnmower Problem 225
9 The BumblebeeP roblem 275
10 The Increasing Benefits of ADFs as Problems are ScaledU p 30r
l'/.. Finding an Impulse ResponseF unction 307
12 Artificial Ant on the San Mateo Trail 349
L3 Obstacle-AvoidingRobot 365
14 The Minesweeper Problem 3n
L5 Automatic Discovery of Detectorsf or Letter Recognition 389
!6 Flushes and Four-of-a-Kinds in a Pinochle Deck 417
t7 Introduction to Biochemistry and Molecular Biology 429
18 Prediction of TransmembraneD omains in Proteins M5
19 Prediction of Omega Loops in Proteins 493
20 Lookahead Version of the ThansmembraneP roblem 505
21 Evolutionary Selection of the Architecture
of the Program 525
22 Evolution of Primitives and Sufficiency 5/3
23 Evolutionary Selection of Terminals 597
24 Evolution of Closure 60L
25 Simultaneous Evolution of Architecture,
Primitive Functions, Terminals, Sufficiency, and Closure 6Ll
26 The Role of Representationa nd the Lens Effect 619
27 Conclusion &3
Appendix A: List of Special Symbols il7
Appendix B: List of SpecialF unctions 651
Appendix C: List of Type Fonts 653
Appendix D: Default Parameters 555
Appendix E: Computer Implementation 661,
Appendix F: Annotated Bibliography of Genetic Programming 697
Appendix G: Electronic Mailing List and Public Repository 717
Bibliography 719
Index 735
Contents
Detailed Thble of Contents
1 Introduction 1
1.1 Overview 7
Background on Genetic Algorithms, LISR
and Genetic Programming 2\
2.1 Background on Genetic Algorithms 21.
2.2 Background on LISP 31,
2.3 Background on Genetic Programming 35
2.4 Sourceso f Additional Information 42
3 Hierarchical Problem-Solving 45
3.1 Hierarchical Decomposition 45
3.2 Recursive Application and Identical Reuse 47
3.3 ParameterizedR eusea nd Generalization 49
3.4 Abstraction 51
3.5 SOAR and Explanation-BasedG eneralization 53
4 Introduction to Automatically Defined Functions -
The Two-Boxes Problem 57
4.1 The Problem 57
4.2 PreparatoryStepsw ithoutADFs 60
4.3 ResultswithoutADFs 63
4.4 The Idea of Subroutines 67
4.5 The Idea of Automatically Defined Functions 73
4.6 PreparatoryStepsw ithADFs 81
4.7 Creation of the kritial Random Population 84
4.8 Structure-PreservingC rossovera nd Typing 85
4.9 ResultswithADFs 87
4.10 Comparison of the Structural Complexity of the Solutions 98
4.11 Comparison of Computational Effort 99
4.12 Summary 106
5 Problems that Shaddle the Breakeven Point
for Computational Effort 109
5.1 Sexticv ersusQ uintic Pollmomial 110
5.1.1 SexticP olynomial x6 -Zxa + x2 1L1,
5.1.1.1 PreparatoryS tepsw ithout ADFs 111
5.1.1.2 Resultsw ithoutADFs 1L2
5.1.1.3 PreparatoryS tepsw ith ADFs 1L3
5.1.1..4 Resultsw ith ADFs 11,4
5.1.1.5 Comparisonw ith and without ADFs 1r7
5.1.2 QuinticPollmomialx 5 -2x3 +x 118
5.1,.2.1,P reparatoryS tepsw ithout ADFs 118
Contents
5.1.2.2 Resultsw ithoutADFs r19
5.L.2.3 Resultsw ithADFs 120
5.1,.2.4 Comparisonw ith and without ADFs 120
5.2 The Boolean 6-Symmetry versus S-Symmetry 122
5.2.1, The Boolean 6-Symmetry Problem 122
5.2.1.1, PreparatoryS tepsw ithoutADFs 123
5.2.1,.2 Resultsw ithout ADFs 124
5.2.L.3 PreparatoryS tepsw ith ADFs 125
5.2.1,.4 Resultsw ithADFs 129
5.2.1.5 Comparisonw ith and withoutADFs 132
5.2.2 The BooleanS -Symmetry Problem 132
5.2.2.1 Resultsw ithoutADFs 132
5.2.2.2 Resultsw ith ADFs r32
5.2.2.3 Comparisonw ith and without ADFs r34
5.3 The Four-Sine versus Three-SineP roblems 134
5.3.1 The Four-SineP roblem- sin x * sinZx + sin 3x * sin 4x 135
5.3.1.1 PreparatoryS tepsw ithout ADFs 135
5.3.1,.2 Resultsw ithout ADFs 135
5.3.1.3 PreparatoryS tepsw ith ADFs r37
5.3.L.4 Resultsw ith ADFs 138
5.3.1,.5 Comparisonw ith and without ADFs 1"41
5.3.2 The Three-SineP roblem- sinx t sin 2x r sin3x 1,42
5.3.2.1. Resultsw ithoutADFs 1,42
5.3.2.2 Resultsw ith ADFs 144
5.3.2.3 Comparisonw ith and withoutADFs 144
5.4 Four Occurrencesv ersusT hree Occurrenceso f a ReusableC onstant 1,M
5.4.1 Three-TermExpressioxn / n+xz / n2 +Zrx 1.45
5.4.1,.L PreparatoryS tepsw ithout ADFs 1,46
5.4.1..2 Resultsw ithoutADFs 146
5.4.1.3 PreparatoryS tepsw ith ADFs 147
5.4.1.4 Resultsw ithADFs 1.48
5.4.1.5 Comparisonw ith and without ADFs 150
5.4.2 TheTWo-TermExpressioxn l n+*' ln2 151
5.4.2.1 Resultsw ithoutADFs 151
5.4.2.2 Resultsw ith ADFs 1.52
5.4.2.3 Comparisonw ith and without ADFs 153
5.5 Summary L53
6 Boolean Parity Functions r57
6.1. The Even-ParityP roblem 157
6.2 PreparatoryS tepsw ithoutADFs 158
6.3 Even-3-Parityw ithoutADFs 1.61,
6.4 Even-4-Parityw ithoutADFs L62
6.5 Even-S-ParitywithoutADFs 162
6.6 Even-6-ParitywithoutADFs L&
6.7 Multiple Function-Defining Branches L66
6.8 HierarchicalA utomatically Defined Functions L67
6.9 PreparatoryS tepswithADFs 170
6.10 Even-3-Parityw ithADFs 175
6.71 Even-4-ParitywithADFs 178
6.12 Even-S-ParitywithADFs 180
6.13 Even-6-ParityP roblemw ith ADFs 188
6.1,4 Summary for the Even-3-,4 -,5-, and 6-Parity problems 189
6.15 Scalingf or the Even-3-,4 -,5-, and 6-Parity problems 190
6.1.6 Higher-Order Even-Parityproblems t94
Contents