Table Of ContentT H E F R O N T I E R S C O L L E C T I O N
SINGULARITY HYPOTHESES SINGULARITY HY
POTHESES SINGULARITY HYPOTHESES SINGU
LARITY HYPOTHESES SINGULARITY HYPOTHES
ES SINGULARITY HYPOTHESES SINGULARITY
HYPOTHESES SINGULARITY HYPOTHESES SIN
GULARITY HYPOTHESES SINGULARITY HYPO
THESES SINGULARITY HYPOTHESES SINGULA
RITY HYPOTHESES SINGULARITY HYPOTHESES
SINGULARITY HYPOTHESES SINGULARITY HY
POTHESES SINGULARITY HYPOTHESES SINGU
LARITY HYPOTHESES SINGULARITY HYPOTHES
ES SINGULARITY HYPOTHESES SINGULARITY
HYPOTHESES SINGULARITY HYPOTHESES SIN
GULARITY HYPOTHESES SINGULARITY HYPO
THESES SINGULARITY HYPOTHESES SINGULA
RITY HYPOTHESES SINGULARITY HYPOTHESES
SINGULARITY HYPOTHESES SINGULARITY HY
POTHESES SINGULARITY HYPOTHESES SINGULA
SINGULARITY HYPOTHESES SIN
GULARITY HYPOTHESES SINGU
LARITY HYPOTHESES SINGULA
RITY HYPOTHESES SINGULARITY
Amnon H. Eden · James H. Moor
HYPOTHESES SINGULARITY HYPO
THESES SINGULARITY HYPOTHESES
Johnny H. Søraker · Eric SStINeGUinLAhRIaTYr tH YPOTHESES SIN
(Eds.)
GULARITY HYPOTHESES SINGU
LA RITY HYPOTHESES SINGULA
RITY HYPOTHESES SINGULARITY
HY POTHESES SINGULARITY HY
SI NGU L A R I T Y
PO THESES SINGULARITY HYPOTHESES
SINGULARITY HYPOTHESES SIN
GULARITY HYPOTHESES SINGU
LARITY HYPOTHESES SINGULA
H Y POT H E SE S
RITY HYPOTHESES SINGULARITY
HYPOTHESES SINGULARITY HY
POTHESES SINGULARITY HYPO
A Scientific and
Philosophical Assessment
THE FRONTIERS COLLECTION
Series Editors
Avshalom C. Elitzur
Unit of Interdisciplinary Studies, Bar-Ilan University, 52900, Ramat-Gan, Israel
e-mail: [email protected]
Laura Mersini-Houghton
DepartmentofPhysics,UniversityofNorthCarolina,ChapelHill,NC27599-3255
USA
e-mail: [email protected]
Maximilian Schlosshauer
Department of Physics, University of Portland,
5000 North Willamette Boulevard Portland, OR 97203, USA
e-mail: [email protected]
Mark P. Silverman
Department of Physics, Trinity College, Hartford, CT 06106, USA
e-mail: [email protected]
Jack A. Tuszynski
Department of Physics, University of Alberta, Edmonton, AB T6G 1Z2, Canada
e-mail: [email protected]
Rudy Vaas
Center for Philosophy and Foundations of Science, University of Giessen, 35394,
Giessen, Germany
e-mail: [email protected]
H. Dieter Zeh
Gaiberger Straße 38, 69151, Waldhilsbach, Germany
e-mail: [email protected]
For furthervolumes:
http://www.springer.com/series/5342
THE FRONTIERS COLLECTION
Series Editors
A. C. Elitzur L. Mersini-Houghton M. Schlosshauer
M. P. Silverman J. A. Tuszynski R. Vaas H. D. Zeh
The books in this collection are devoted to challenging and open problems at the
forefrontofmodernscience,includingrelatedphilosophicaldebates.Incontrastto
typical research monographs, however, they strive to present their topics in a
manner accessible also to scientifically literate non-specialists wishing to gain
insightintothedeeperimplicationsandfascinatingquestionsinvolved.Takenasa
whole,theseriesreflectstheneedforafundamentalandinterdisciplinaryapproach
tomodernscience.Furthermore,itisintendedtoencourageactivescientistsinall
areas toponderoverimportant andperhapscontroversialissuesbeyondtheirown
speciality. Extending from quantum physics and relativity to entropy, conscious-
ness and complex systems—the Frontiers Collection will inspire readers to push
back the frontiers of their own knowledge.
Forafulllistofpublishedtitles,pleaseseebackofbookorspringer.com/series/5342
Amnon H. Eden James H. Moor
•
Johnny H. Søraker Eric Steinhart
•
Editors
Singularity Hypotheses
A Scientific and Philosophical Assessment
123
Editors
AmnonH.Eden JohnnyH.Søraker
School ofComputer Science Department of Philosophy
and ElectronicEngineering Universityof Twente
Universityof Essex Enschede
Colchester The Netherlands
UK
Eric Steinhart
James H.Moor Department of Philosophy
Dartmouth College William Paterson University
Hanover Wayne
USA USA
ISSN 1612-3018
ISBN 978-3-642-32559-5 ISBN 978-3-642-32560-1 (eBook)
DOI 10.1007/978-3-642-32560-1
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2012946755
(cid:2)Springer-VerlagBerlinHeidelberg2012
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor
informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodology now known or hereafter developed. Exempted from this legal reservation are brief
excerpts in connection with reviews or scholarly analysis or material supplied specifically for the
purposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthe
work. Duplication of this publication or parts thereof is permitted only under the provisions of
theCopyrightLawofthePublisher’slocation,initscurrentversion,andpermissionforusemustalways
beobtainedfromSpringer.PermissionsforusemaybeobtainedthroughRightsLinkattheCopyright
ClearanceCenter.ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
To Saul, With love,
—Aba
Contents
1 Singularity Hypotheses: An Overview. . . . . . . . . . . . . . . . . . . . . . 1
Amnon H. Eden, Eric Steinhart, David Pearce and James H. Moor
Part I A Singularity of Artificial Superintelligence
2 Intelligence Explosion: Evidence and Import. . . . . . . . . . . . . . . . . 15
Luke Muehlhauser and Anna Salamon
2A Robin Hanson on Muehlhauser and Salamon’s
‘‘Intelligence Explosion: Evidence and Import’’ . . . . . . . . . . . . 41
3 The Threat of a Reward-Driven Adversarial Artificial General
Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Itamar Arel
3A William J. Rapaport on Arel’s ‘‘The Threat of a Reward-Driven
Adversarial Artificial General Intelligence’’ . . . . . . . . . . . . . . . 59
4 New Millennium AI and the Convergence of History:
Update of 2012 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Jürgen Schmidhuber
4A Aaron Sloman on Schmidhuber’s ‘‘New Millennium AI
and the Convergence of History 2012’’ . . . . . . . . . . . . . . . . . . 79
4B Selmer Bringsjord, Alexander Bringsjord and Paul Bello
on Schmidhuber’s ‘‘New Millennium AI and the Convergence
of History 2012’’. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5 Why an Intelligence Explosion is Probable . . . . . . . . . . . . . . . . . . 83
Richard Loosemore and Ben Goertzel
5A Peter Bishop’s on Loosemore and Goertzel’s
‘‘Why an Intelligence Explosion is Probable’’. . . . . . . . . . . . . . 97
vii
viii Contents
Part II Concerns About Artificial Superintelligence
6 The Singularity and Machine Ethics. . . . . . . . . . . . . . . . . . . . . . . 101
Luke Muehlhauser and Louie Helm
6A Jordi Vallverdú on Muehlhauser and Helm’s
‘‘the Singularity and Machine Ethics’’ . . . . . . . . . . . . . . . . . . . 127
7 Artificial General Intelligence and the Human Mental Model . . . . 129
Roman V. Yampolskiy and Joshua Fox
8 Some Economic Incentives Facing a Business that Might Bring
About a Technological Singularity . . . . . . . . . . . . . . . . . . . . . . . . 147
James D. Miller
8A Robin Hanson on Miller’s ‘‘Some Economic
Incentives Facing a Business that Might Bring
About a Technological Singularity’’. . . . . . . . . . . . . . . . . . . . . 159
9 Rational Artificial Intelligence for the Greater Good. . . . . . . . . . . 161
Steve Omohundro
9A Colin Allen and Wendell Wallach on Omohundro’s
‘‘Rationally-Shaped Artificial Intelligence’’. . . . . . . . . . . . . . . . 177
10 Friendly Artificial Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
Eliezer Yudkowsky
10A ColinAllenonYudkowsky’s‘‘FriendlyArtificialIntelligence’’. . . 195
Part III A Singularity of Posthuman Superintelligence
11 The Biointelligence Explosion. . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
David Pearce
11A Illah R. Nourbakhsh on Pearce’s ‘‘The Biointelligence
Explosion’’. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
12 Embracing Competitive Balance: The Case for
Substrate-Independent Minds and Whole Brain Emulation . . . . . . 241
Randal A. Koene
12A Philip Rubin on Koene’s ‘‘Embracing Competitive Balance:
The Case For Substrate-Independent Minds and Whole
Brain Emulation’’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
13 Brain Versus Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Dennis Bray
13A Randal Koene on Bray’s ‘‘Brain Versus Machine’’ . . . . . . . . . . 279
Contents ix
14 The Disconnection Thesis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
David Roden
Part IV Skepticism
15 Interim Report from the Panel Chairs: AAAI Presidential
Panel on Long-Term AI Futures. . . . . . . . . . . . . . . . . . . . . . . . . . 301
Eric Horvitz and Bart Selman
15A Itamar Arel on Horwitz’s ‘‘AAAI Presidential Panel
on Long Term AI Futures’’. . . . . . . . . . . . . . . . . . . . . . . . . . . 307
15B Vernor Vinge on Horvitz’s ‘‘AAAI Presidential Panel
on Long-Term AI Futures’’. . . . . . . . . . . . . . . . . . . . . . . . . . . 309
16 Why the Singularity Cannot Happen . . . . . . . . . . . . . . . . . . . . . . 311
Theodore Modis
16A Vernor Vinge on Modis’ ‘‘Why the Singularity
Cannot Happen’’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
16B Ray Kurzweil on Modis’ ‘‘Why the Singularity
Cannot Happen’’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
17 The Slowdown Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
Alessio Plebe and Pietro Perconti
17A Eliezer Yudkowsky on Plebe & Perconti’s
‘‘The Slowdown Hypothesis’’ . . . . . . . . . . . . . . . . . . . . . . . . . 363
18 Software Immortals: Science or Faith?. . . . . . . . . . . . . . . . . . . . . 367
Diane Proudfoot
18A Francis Heylighen on Proudfoot’s ‘‘Software Immortals:
Science or Faith?’’. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
19 Belief in The Singularity is Fideistic. . . . . . . . . . . . . . . . . . . . . . . 395
Selmer Bringsjord, Alexander Bringsjord and Paul Bello
19A Vernor Vinge on Bringsjord et al.’s ‘‘Belief in the Singularity
is Fideistic’’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
19B Michael Anissimov on Bringsjord et al.’s ‘‘Belief in
The Singularity is Fideistic’’. . . . . . . . . . . . . . . . . . . . . . . . . . 411
20 A Singular Universe of Many Singularities: Cultural
Evolution in a Cosmic Context. . . . . . . . . . . . . . . . . . . . . . . . . . . 413
Eric J. Chaisson
20A Theodore Modis on Chaisson’s ‘‘A Singular Universe
of Many Singularities: Cultural Evolution
in a Cosmic Context’’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
Chapter 1
Singularity Hypotheses: An Overview
Introduction to: Singularity Hypotheses:
A Scientific and Philosophical Assessment
Amnon H. Eden, Eric Steinhart, David Pearce and James H. Moor
Questions
Bill Joy in a widely read but controversial article claimed that the most powerful
21st century technologies are threatening to make humans an endangered species
(Joy 2000). Indeed, a growing number of scientists, philosophers and forecasters
insist that the accelerating progress in disruptive technologies such as artificial
intelligence, robotics, genetic engineering, and nanotechnology may lead to what
theyrefertoasthetechnologicalsingularity:aneventorphasethatwillradically
change human civilization, and perhaps even human nature itself, before the
middle of the 21st century (Paul and Cox 1996; Broderick 2001; Garreau 2005,
Kurzweil 2005).
Singularity hypotheses refer to either one of two distinct and very different
scenarios. The first (Vinge 1993; Bostrom to appear) postulates the emergence of
artificialsuperintelligentagents—software-basedsyntheticminds—asthe‘singular’
outcomeofacceleratingprogressincomputingtechnology.Thissingularityresults
A.H.Eden(&)
SchoolofComputerScienceand,ElectronicEngineering,
UniversityofEssex,Colchester,CO43SQ,UK
e-mail:[email protected]
E.Steinhart
DepartmentofPhilosophy,WilliamPatersonUniversity,300PomptonRoad,Wayne,NJ
07470,USA
e-mail:[email protected]
D.Pearce
knightsbridgeOnline,7LowerRockGardens,Brighton,USA
e-mail:[email protected]
J.H.Moor
DepartmentofPhilosophy,DartmouthCollege,6035Thornton,Hanover,NH03755,USA
e-mail:[email protected]
A.H.Edenetal.(eds.),SingularityHypotheses,TheFrontiersCollection, 1
DOI:10.1007/978-3-642-32560-1_1,(cid:2)Springer-VerlagBerlinHeidelberg2012
