Intelligence Emerging: Adaptivity and Search in Evolving Neural Systems

499 Pages·2015·7.85 MB·English

by  Keith L. Downing

IntelligenceEmerging

IntelligenceEmerging
AdaptivityandSearchinEvolvingNeuralSystems
KeithL.Downing
TheMITPress
Cambridge,Massachusetts
London,England
(cid:13)c 2015MassachusettsInstituteofTechnology
Allrightsreserved.Nopartofthisbookmaybereproducedinanyformbyanyelectronicormechanical
means(includingphotocopying,recording,orinformationstorageandretrieval)withoutpermission
inwritingfromthepublisher.
MITPressbooksmaybepurchasedatspecialquantitydiscountsforbusinessorsalespromotionaluse.
Forinformation,[email protected]
This book was set in Palatino by diacriTech, Chennai. Printed and bound in the United States of
America.
LibraryofCongressCataloging-in-PublicationData
Downing,KeithL.
Intelligenceemerging:adaptivityandsearchinevolvingneuralsystems/KeithL.Downing.
pagescm
Includesbibliographicalreferencesandindex.
ISBN978-0-262-02913-1(hardcover:alk.paper)1.Neuralnetworks(Computerscience)
2.Machinelearning.3.Experientiallearning.4.Geneticalgorithms.
5.Neuralnetworks(Computerscience)6.Adaptivecomputingsystems.I.Title.
QA76.87.D692015
006.3’2—dc23
2014045996
10 9 8 7 6 5 4 3 2 1
ToNancy,Marvin,Ma˚lfrid,Neva,Asta,andJasem

Contents
Preface xiii
Acknowledgments xix
1 Introduction 1
1.1 ExplorationandEmergence 1
1.2 Intelligence 4
1.3 Adaptivity 6
1.4 IAm,ThereforeIThink 8
1.5 EvolvingNeuralNetworks 12
1.6 DecipheringKnowledgeandGaugingComplexity 13
1.7 Simon’sGreatAnt 14
1.8 SearchinCarbonandSilicon 17
1.9 TravelingLight 19
2 Emergence 21
2.1 ComplexAdaptiveSystems 21
2.2 ChaosandComplexity 25
2.3 LevelsofEmergence 29
2.3.1 Deacon’sAutogen 32
2.4 EmergenceandSearch 36
2.5 EmergentIntelligence 39
3 Search:TheCoreofAI 43
3.1 DesignSearch 43
3.2 ClassicAISearchTechniques 44
3.2.1 Best-FirstSearch 48
3.2.2 SteinerTreesUsingBest-FirstSearch 52
3.2.3 SearchamongWholeSolutions 54
3.2.4 LocalSearchforSteinerTrees 56
3.3 SteinerBrains 61
3.4 BiologicalSearchandProblemSolving 63
viii Contents
4 RepresentationsforSearchandEmergence 67
4.1 KnowledgeRepresentations 67
4.2 ModelsofIntelligence 68
4.3 RepresentationandReasoning 71
4.4 Bio-InspiredRepresentationsforLocalSearch 74
4.5 ARepresentationPuzzle 77
4.6 ACommonFrameworkforEmergentAI 80
4.7 RepresentationsforEvolvingArtificialNeuralNetworks 82
4.8 RepresentationalTrade-offs 84
5 EvolutionaryAlgorithms 85
5.1 DarwinianEvolution 85
5.2 ArtificialEvolutioninaComputer 87
5.3 SpyversusSpy 92
5.4 TheMainClassesofEvolutionaryAlgorithms 94
5.4.1 EvolutionaryStrategies 94
5.4.2 EvolutionaryProgramming 95
5.4.3 GeneticAlgorithms 96
5.4.4 GeneticProgramming 97
5.4.5 ConvergenceofEATypes 98
5.5 LetEvolutionFigureItOut 98
6 ArtificialNeuralNetworks 101
6.1 TheMentalMatrix 101
6.2 ThePhysiologyofInformationTransmission 102
6.3 ThePhysiologicalBasisofLearning 106
6.3.1 BasicBiochemistryofSynapticChange 106
6.3.2 Long-TermPotentiation 110
6.4 AbstractionsandModels 111
6.4.1 SpatialAbstractions 112
6.4.2 FunctionalAbstractions 115
6.4.3 TemporalAbstractions 121
6.4.4 TheSpikeResponseModel(SRM) 124
6.4.5 TheIzhikevichModel 126
6.4.6 Continuous-TimeRecurrentNeuralNetworks 126
6.5 NetPicking 129
7 KnowledgeRepresentationinNeuralNetworks 131
7.1 Activity,Connectivity,andCorrelation 131
7.2 RepresentationsinFiringPatterns 133
Contents ix
7.2.1 TheSyntaxofNeuralInformation 134
7.2.2 PatternCompletioninDistributedMemories 135
7.2.3 SequentialPatternRetrieval 136
7.2.4 ConnectingPatterns:TheSemanticsofNeuralInformation 137
7.2.5 HierarchicalClusterAnalysis 142
7.3 RepresentationsinSynapticMatrices 144
7.3.1 PrincipalComponentAnalysis 150
7.4 NeuroarchitecturestoRealizeEffectiveDistributedCoding 155
7.4.1 PatternProcessingwithDenseDistributedCodes 158
7.4.2 Cortical-HippocampalInteractions 162
7.4.3 Third-OrderEmergenceintheBrain 163
7.5 TheFallacyofUntetheredKnowledge 163
8 SearchandRepresentationinEvolutionaryAlgorithms 165
8.1 SearchasResourceAllocation 165
8.2 EvolutionarySearch 166
8.3 ExplorationversusExploitationinEvolutionarySearch 171
8.4 RepresentationsforEvolutionaryAlgorithms 175
8.4.1 Data-OrientedGenotypes 178
8.4.2 Program-OrientedGenotypes 192
8.5 Bio-AI’sHeadDesigner 197
9 EvolutionandDevelopmentoftheBrain 199
9.1 NeuronsinaHaystack 199
9.2 TheNeuromericModelandHoxGenes 200
9.3 NeuralDarwinismandDisplacementTheory 203
9.4 FacilitatedVariation 206
9.4.1 Modularity 207
9.4.2 WeakLinkage 208
9.4.3 ExploratoryGrowth 211
9.4.4 EmergingTopologicalMaps 214
9.4.5 DeconstrainingEvolution 217
9.5 RepresentationandSearchinNeuralEvolutionandDevelopment 219
9.6 TheBio-Inspiration 221
9.6.1 GeneticRegulatoryNetworks 221
9.6.2 ConservedCoreProcesses 223
9.6.3 WeakLinkageandTags 224
9.6.4 InteractingAdaptiveMechanisms 226
9.6.5 SolvingNetworkProblemswithFruitFlies 227
9.7 AppreciatingAdd-HoxDesign 229