Genetic Programming Theory and Practice XIV

233 Pages·2018·6.782 MB·English

by  Rick Riolo, Bill Worzel, Brian Goldman, Bill Tozier

Genetic and Evolutionary Computation
Rick Riolo · Bill Worzel · Brian Goldman 
Bill Tozier Editors
Genetic
Programming
Theory and
Practice XIV
Genetic and Evolutionary Computation
SeriesEditors:
DavidE.Goldberg,ThreeJoyAssociates,Inc.,Urbana,IL,USA
JohnR.Koza,StanfordUniversity,LosAltos,CA,USA
Moreinformationaboutthisseriesathttp://www.springer.com/series/7373
Rick Riolo • Bill Worzel (cid:129) Brian Goldman
Bill Tozier
Editors
Genetic Programming
Theory and Practice XIV
123
Editors
RickRiolo BillWorzel
CenterfortheStudyofComplexSys EvolutionEnterprises
UniversityofMichigan AnnArbor,MI,USA
AnnArbor,MI,USA
BillTozier
BrianGoldman AnnArbor
ColoradoStateUniversity MI,USA
FortCollins,CO,USA
ISSN1932-0167
GeneticandEvolutionaryComputation
ISBN978-3-319-97087-5 ISBN978-3-319-97088-2 (eBook)
https://doi.org/10.1007/978-3-319-97088-2
LibraryofCongressControlNumber:2018957658
©SpringerNatureSwitzerlandAG2018
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Thisisthewrittenrecordofthe14thmeetingoftheannualGeneticProgramming
Theory and Practice Workshop, which was hosted by the Center for the Study of
ComplexSystemsattheUniversityofMichigan,inAnnArbor,May19–21,2016.
It is, as a matter of course, woefully incomplete. It can serve only as a
fragmentary record of our meeting. I say this not as an apology, but rather as a
sortofgentlewarning,ofasortIrarelyseemtoseeinthesecollections.
Letmetrytoexplain.ThecentralandexplicitfocusoftheGPTPWorkshophas
alwaysbeentheconversationsthatarefosteredatthemeetingitself.Theseconver-
sations happen among the invited keynote speakers, the technical participants, the
students,andsponsorswhospendsomewhatmorethan3daysinaroomtogether.
Thetechnicalworkthatinvitedspeakershavepreparedbeforetheyarrive,andwhich
theybringwiththemtopresentinsessionanddiscuss,isreallyjusttheprovocation
or seed of the “real workshop.” As a result, the proceedings volume from each
workshop should only ever be read as a record of where some subset of attendees
startedindividually,notasapositioninwhichweendedupasagroupbytheend.
Therealresultswillonlygraduallyappearinthesubsequentliterature,asacloudof
worksthatmaynotevenexplicitlyrefertothismeeting,andinthesubtlychanged
directionsofongoingresearchprogramsinyearstocome.
In other words, think of these chapters as the inputs to a years-long dynamical
process,notasthefinaloutputofthatprocess.
Further,thisvolumedoesnotevenmanagetocoverallofthepresentation-driven
portionofourdiscussions.Itisasadbutunavoidabletruthofmodernworklifethat
onlyafractionofourmostinfluentialparticipantshavethesparetimetoproducea
chapterforyoutoread.Ourthreeinvitedkeynotespeakers,forexample,inevitably
haveastronginfluenceonourcollectiveattention,settingthetoneandfocusofthe
entireweekassoonastheybeginspeaking...buttheyarerarelyabletomaketime
to provide written contributions here. And because we so actively seek out both
industrial and academic speakers, we often hear at least one or two presentations
that treat contemporary professional work from an industrial domain. Proprietary
and ongoing projects cannot always be translated successfully into an academic-
stylepaper.
v
vi Preface
Since this was our 14th meeting, you should suspect that even calling these
chaptersthe“inputs”ofalong-termprocessmissestheprecedinghistory.Ifwecan
onlydetectthe“outputs”intheworkshop’sfuture-facinglight-cone—byseeingthe
consequences of the week’s conversations and insights in the attendees’ actions—
thensurelyalmostallof2016sworkshopwasinturninfluencedbyearliersessions.
YouwillfindchaptersheredescribingtheongoingeffortsofMichaelAffenzeller’s
HEAL group, as they implement, explore, and distribute ideas and algorithms in
theirHeuristicLabsystem,whichhaveinseveralcasesarisendirectlyfromearlier
GPTPworkshops.MichaelKornscarriesforwardhismanyyears’effortinsupport
ofindustrial-strengthsymbolicregressionsystems.Thereareatleastthreechapters
herefromLeeSpector,NicMcPhee,ThomasHelmuth,andcolleagues,whichhave
arisenfrommanyyearsoffruitfulexplorationofthePushlanguageforGP,andso
on.IdoubtasinglechapterlacksastronglinkfromsomeearlierGPTPconversation.
Therefore, think of this book as a collection of blurred snapshots, taken from
the center of the web of conversations that make up the “real” GPTP Workshop.
The historian interested in the context from which they have arisen should review
the prior 13 volumes in the series at least as diligently as she reviews the more
mundanepublishedconferenceliteratureinadjacentyears.
Assume that we took the time to speak to one another—substantively, but not
constantly—aboutthematerialpresentedinthisvolume.Butinthemomentandin
the room together, our conversations were much more along the lines of, “That’s
fascinating,buthaveyouconsidered...,”or“Iwonderifyou’venoticedthatwhat
you did in Section 3 could be related to what Smith did in her work on...,” and
so forth. We do not just clap and grumble at one another—we discuss. When it
works, this workshop is a generative process, not a ritualized presentation. Most
ofthe“work”itdoesinthefieldwillhavehappenedaftertheattendeesarrived,in
the lunch breaks, the hallway conversations, and the notes we have scribbled on a
napkininthepubafterward,orinournotebooksinhotelroomsafterquietthoughtful
dinners.
Inthefollowing14chapters,youwillfindthesubsetofcontributionsfrominvited
speakerswhowereabletoprovidethem.Theyhavedoneagoodjobcontextualizing
and framing their work, subject to the caveats I have spelled out above. So rather
thansimplyrevisitingeachoneinturn,letmetrytofillinthegapsbetweenthema
bit,beginningwithourthreekeynotespeakers.
Dr.JoannaMasel,fromEcologyandEvolutionaryBiologyattheUniversityof
Arizona, was the first of our invited keynote speakers. She spoke on “Evolution
ofmolecularerrorrates,andtheconsequencesforrobustness,evolvability,andthe
de novo emergence of new protein-coding genes from junk DNA.” She pointedly
reminded the audience—we are for the most part computer science folks—of the
deepfundamentaldifferencesbetweenbiologicaland“computational”evolutionary
processes.Inthecourseofherpresentation,shedidanexcellentjobconveyingboth
the fascinating complexity of biological evolution (and evolutionary biology, the
discipline), and also the potential shortcomings of our greatly simplified mental
models in evolutionary computing. Throughout the workshop, she was able to
Preface vii
helpfullyremindmanyspeakersandparticipantsofthewaysinwhichsuchanoverly
glibsimplemodelmightleadoneintoaproverbialditch.
Dr. Stephanie Forrest, from the University of New Mexico and the Santa Fe
Institute (a colleague of several attendees), spoke in her keynote lecture about
“Software: Evolution, Robustness, and Diversity (also, the Mutation Cliff).” She
spoke about her group’s and colleagues’ ongoing research into the complexities
of real (which is to say: human-written) software systems and in particular their
work in the new field of genetic improvement and automated software repair. As
withDr.Masel’seffort,weintheaudiencewerefrequentlyandpointedlyreminded
of the “simplifying assumptions” our work often makes for the sake of being
tractable. Once again, there are many ways in which the consequences from our
overly simplified framing notions can stumble, when faced with the externalities
of the real world. In particular, she spoke of her own practical and philosophical
explorationsofwhat“fitness”mightmeaninthecontextofproducingwell-repaired
softwareoriginallyproducedbyhumanprogrammers.Asmynoteshaveit,“What
isareasonablewayofquantifyingtheperformanceofabrokenorrepairedpieceof
low-levelsoftwareinfrastructure?”WetendinthefieldofGPnottothinkveryoften
about bug reports, side effects, and other matters that live past the interface with
software development, deployment, and usability...but our more advanced work
inevitablybumpsupagainstit.
On the third day of our meeting, Dr. Cosma Shalizi from the Statistics Depart-
mentatCarnegieMellonUniversityandtheSantaFeInstitutespokeon“Bayesian
Learning, Evolutionary Search, and Information Theory.” And boy did he. He
pointed out remarkable (butas farasIamaware, previously unremarked) similar-
itiesinthedeepstructureofBayesianlearningrepresentationsandalgorithms,the
replicator equation and other core dynamical models from theoretical population
biology, and the ways in which information (in Shannon’s sense) is “handled” by
these processes. In other words, in a bit more than an hour, he stitched together
three increasingly independent disciplinary approaches to learning and dynamical
systemsmodelsanddescribedastrongframeworkforexploringwhatitmightmean
forevolutionto“learn.”Thisdelightfulpresentation—whichseveralofushopehe
willhavepublishedsomewheresoon—broughtinframeworksfromdisciplinesthat
willcertainlybenefitfromanontologicalreconciliationlikethis.Inotherwords,we
lookforwardtoreadinghispaperonthematteratleastasmuchasyoudo.
Intermingledwiththethreekeynote speakers’talks,andthepresentations from
theinvitedspeakerswhosechaptersfollow,therewerealsoinnumerablebreakfast,
lunch,anddinnermeetings(alladhocofcourse).Watchfortheireffectsinthefuture
workoftheattendees,asyoureadtheworkstheybroughttothetabletobeginour
conversationswithoneanother.
AnnArbor,MI,USA WilliamTozier
FortCollins,CO,USA BrianGoldman
AnnArbor,MI,USA RickRiolo
AnnArbor,MI,USA WilliamP.Worzel
December2017
Acknowledgments
Wefeelobligednotonlytothanktheparticipantsinthe2016GPTPWorkshopbutto
emphasizehowproudwearetohaveplayedourlittleroleinbringingthemtogether.
Werealize(becausetheysayitoverandover,yearafteryear)thatthegueststhink
thisisanenjoyableandinspiringmeeting,butthepleasureisliterallyours—atleast
inhindsight.Ifyoulisten,youwillnoticeouranecdotesalltalkaboutthewaywe
watchedyoutalkwithoneanothersoproductivelyandpleasantly,notjustwhatyou
saidbuthowyouweretogether.Sothankyouallforbeinginterestingpeoplewho
cannowandthenlistentooneanother.
The keynote speakers are not just punctuation marks or particularly honored
membersofourlittlecommunity,buthavebeentaskedwithacrucialroleofjostling
our attention in a productive way and steering us away from our discipline-killing
tendencyforonlyevertalkingtooneanother.Ourspeakersin2016didanadmirable
job in this, and we appreciate their company at least as much as the insights they
broughtfromadjacentdisciplines.
We could not undertake these ambitious workshops without the generous
sponsorships, with whose help we are increasingly able to bring new colleagues
intothefold.In2016,theseincluded:
(cid:129) The Center for the Study of Complex Systems at the University of Michigan
andespeciallyCarlSimonandCharlesDoering,thechampionsoftheworkshop
series
(cid:129) JohnKoza
(cid:129) JasonMoore
(cid:129) BabakHodjatatSentient
(cid:129) MichaelKornsandGildaCabral
(cid:129) MarkKotanchekatEvolvedAnalytics
(cid:129) The Heuristic and Evolutionary Algorithms Laboratory at the Upper Austria
UniversityofAppliedScience
Finally, there are those without whose help we would not have been able to
manage: Linda Wood and Mita Gibson at the Center for the Study of Complex
Systems.
Thankyouallformakingthispossible,andapleasure.
ix
Contents
1 Similarity-BasedAnalysisofPopulationDynamicsinGenetic
ProgrammingPerformingSymbolicRegression........................ 1
Stephan M. Winkler, Michael Affenzeller, Bogdan Burlacu,
GabrielKronberger,MichaelKommenda,andPhilippFleck
2 An Investigation of Hybrid Structural and Behavioral
DiversityMethodsinGeneticProgramming ............................ 19
ArmandR.BurksandWilliamF.Punch
3 InvestigatingMulti-PopulationCompetitiveCoevolutionfor
AnticipationofTaxEvasion............................................... 35
ErikHemberg,JacobRosen,andUna-MayO’Reilly
4 EvolvingArtificialGeneralIntelligenceforVideo
GameControllers........................................................... 53
ItayAzaria,AchiyaElyasaf,andMosheSipper
5 ADetailedAnalysisofaPushGPRun ................................... 65
NicholasFreitagMcPhee,MitchellD.Finzel,MaggieM.Casale,
ThomasHelmuth,andLeeSpector
6 LinearGenomesforStructuredPrograms .............................. 85
Thomas Helmuth, Lee Spector, Nicholas Freitag McPhee,
andSaulShanabrook
7 Neutrality,Robustness,andEvolvabilityinGenetic
Programming ............................................................... 101
TingHuandWolfgangBanzhaf
8 LocalSearchisUnderusedinGeneticProgramming .................. 119
Leonardo Trujillo, Emigdio Z-Flores, Perla S. Juárez-Smith,
PierrickLegrand,SaraSilva,MauroCastelli,LeonardoVanneschi,
OliverSchütze,andLuisMuñoz
xi