Linear Genetic Programming

322 Pages·2007·1.749 MB·English

by  Markus F. Brameier, Wolfgang Banzhaf

Markus Brameier and Wolfgang Banzhaf
Linear Genetic Programming
Genetic and Evolutionary Computation Series
Series Editors
David E. Goldberg
Consulting Editor
IlliGAL, Dept. of General Engineering
University of Illinois at Urbana-Champaign
Urbana, IL 61801 USA
Email: [email protected]
John R. Koza
Consulting Editor
Medical Informatics
Stanford University
Stanford, CA 94305-5479 USA
Email: [email protected]
Selected titles from this series:
Nikolay Y. Nikolaev, Hitoshi Iba
Adaptive Learning of Polynomial Networks, 2006
ISBN 978-0-387-31239-2
Tetsuya Higuchi, Yong Liu, Xin Yao
Evolvable Hardware, 2006
ISBN 978-0-387-24386-3
David E. Goldberg
The Design of Innovation: Lessons from and for Competent Genetic Algorithms, 2002
ISBN 978-1-4020-7098-3
John R. Koza, Martin A. Keane, Matthew J. Streeter, William Mydlowec, Jessen Yu, Guido Lanza
Genetic Programming IV: Routine Human-Computer Machine Intelligence
ISBN: 978-1-4020-7446-2 (hardcover), 2003; ISBN: 978-0-387-25067-0 (softcover), 2005
Carlos A. Coello Coello, David A. Van Veldhuizen, Gary B. Lamont
Evolutionary Algorithms for Solving Multi-Objective Problems, 2002
ISBN: 978-0-306-46762-2
Lee Spector
Automatic Quantum Computer Programming: A Genetic Programming Approach
ISBN: 978-1-4020-7894-1 (hardcover), 2004; ISBN 978-0-387-36496-4 (softcover), 2007
William B. Langdon
Genetic Programming and Data Structures: Genetic Programming + Data Structures = Automatic
Programming! 1998
ISBN: 978-0-7923-8135-8
For a complete listing of books in this series, go to http://www.springer.com
Markus Brameier
Wolfgang Banzhaf
Linear Genetic
Programming

1 3
Markus F. Brameier
Bioinformatics Research Center (BiRC)
University of Aarhus
Denmark
[email protected]
Wolfgang Banzhaf
Department of Computer Science
Memorial University of Newfoundland
St. John’s, NL
Canada
[email protected]
Library of Congress Control Number: 2006920909
ISBN-10: 0-387-31029-0 e-ISBN-10: 0-387-31030-4
ISBN-13: 978-0387-31029-9 e-ISBN-13: 978-0387-31030-5

© 2007 by Springer Science+Business Media, LLC
All rights reserved. This work may not be translated or copied in whole or in part
without the written permission of the publisher (Springer Science + Business
Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief
excerpts in connection with reviews or scholarly analysis. Use in connection with
any form of information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter
developed is forbidden.
The use in this publication of trade names, trademarks, service marks and similar
terms, even if they are not identified as such, is not to be taken as an expression
of opinion as to whether or not they are subject to proprietary rights.
Printed in the United States of America
9 8 7 6 5 4 3 2 1
springer.com
We dedicate this book
to our beloved parents.
Contents
Preface xi
About the Authors xv
1. INTRODUCTION 1
1.1 Evolutionary Algorithms 1
1.2 Genetic Programming 3
1.3 Linear Genetic Programming 6
1.4 Motivation 8
Part I Fundamental Analysis
2. BASIC CONCEPTS OF LINEAR GENETIC PROGRAMMING 13
2.1 Representation of Programs 13
2.2 Execution of Programs 25
2.3 Evolution of Programs 29
3. CHARACTERISTICS OF THE LINEAR REPRESENTATION 35
3.1 Effective Code and Noneffective Code 35
3.2 Structural Introns and Semantic Introns 37
3.3 Graph Interpretation 47
3.4 Analysis of Program Structure 56
3.5 Graph Evolution 60
3.6 Summary and Conclusion 61
viii Contents
4. A COMPARISON WITH NEURAL NETWORKS 63
4.1 Medical Data Mining 63
4.2 Benchmark Data sets 64
4.3 Experimental Setup 65
4.4 Experiments and Comparison 69
4.5 Summary and Conclusion 74
Part II Method Design
5. SEGMENT VARIATIONS 77
5.1 Variation Effects 78
5.2 Effective Variation and Evaluation 79
5.3 Variation Step Size 80
5.4 Causality 82
5.5 Selection of Variation Points 86
5.6 Characteristics of Variation Operators 87
5.7 Segment Variation Operators 89
5.8 Experimental Setup 99
5.9 Experiments 102
5.10 Summary and Conclusion 118
6. INSTRUCTION MUTATIONS 119
6.1 Minimum Mutation Step Size 119
6.2 Instruction Mutation Operators 121
6.3 Experimental Setup 129
6.4 Experiments 131
6.5 Summary and Conclusion 148
7. ANALYSIS OF CONTROL PARAMETERS 149
7.1 Number of Registers 149
7.2 Number of Output Registers 156
7.3 Rate of Constants 157
Contents ix
7.4 Population Size 159
7.5 Maximum Program Length 162
7.6 Initialization of Linear Programs 164
7.7 Constant Program Length 169
7.8 Summary and Conclusion 170
8. A COMPARISON WITH TREE-BASED GP 173
8.1 Tree-Based Genetic Programming 173
8.2 Benchmark Problems 177
8.3 Experimental Setup 181
8.4 Experiments and Comparison 185
8.5 Discussion 190
8.6 Summary and Conclusion 191
Part III Advanced Techniques and Phenomena
9. CONTROL OF DIVERSITY AND VARIATION STEP SIZE 195
9.1 Introduction 195
9.2 Structural Program Distance 197
9.3 Semantic Program Distance 200
9.4 Control of Diversity 201
9.5 Control of Variation Step Size 203
9.6 Experimental Setup 205
9.7 Experiments 206
9.8 Alternative Selection Criteria 222
9.9 Summary and Conclusion 223
10. CODE GROWTH AND NEUTRAL VARIATIONS 225
10.1 Code Growth in GP 226
10.2 Proposed Causes of Code Growth 227
10.3 Influence of Variation Step Size 229
10.4 Neutral Variations 230
x Contents
10.5 Conditional Reproduction and Variation 232
10.6 Experimental Setup 233
10.7 Experiments 233
10.8 Control of Code Growth 249
10.9 Summary and Conclusion 259
11. EVOLUTION OF PROGRAM TEAMS 261
11.1 Introduction 261
11.2 Team Evolution 262
11.3 Combination of Multiple Predictors 265
11.4 Experimental Setup 273
11.5 Experiments 276
11.6 Combination of Multiple Program Outputs 286
11.7 Summary and Conclusion 287
Epilogue 289
References 291
Index 303
Preface
This book is about linear genetic programming (LGP), a variant of GP
that evolves computer programs as sequences of instructions of an imper-
ative programming language. It is a comprehensive text with a strong
experimental basis and an in-depth focus on structural aspects of the lin-
ear program representation.
The three major objectives of this book are:
(cid:2) To discuss linear genetic programming in a broader context and to
contrast it with tree-based genetic programming.
(cid:2) To develop advanced methods and efficient genetic operators for the
imperative representation to produce both better and shorter program
solutions.
(cid:2) To give a better understanding for the intricate effects of operators on
evolutionary processes and emergent phenomena in linear GP.
PartIofthebookisdedicatedtolayingafoundationforbasicunderstand-
ing as well as to providing methods for analysis. The first two chapters
give an introduction to evolutionary computation, genetic programming,
and linear GP.
Chapter 3 presents efficient algorithms for analyzing the imperative and
functional structure of linear genetic programs during runtime. The spe-
cial program representation used in this book can be transformed into
directed acyclic graphs (DAGs). So-called structurally noneffective code
can be identified that is disconnected from the effective data flow and
independent of program semantics. Other important parameters of linear
programs include the number of effective registers at a certain program