Table Of ContentI
N
AT “Digital Soul’s thoughtful questions stimulate a fascinating narrative.”
NE
—Science Books & Films
DL
L
HI
G
U
E
M
N
AT INTELLIGENT MACHINES
N
M
SCIENCE / TECHNOLOGY V AND HUMAN VALUES
A
A
C
L
UH
“a book certain to spark sharp debate.” EIN
—Booklist SE
S
Should the day come when intelligent machines not only make computations but also
think and experience emotions as humans do, how will we distinguish the “human” from
the “machine”?
This introduction to artificial intelligence—and to its potentially profound social, moral,
and ethical implications—is designed for readers with little or no technical back-
ground. In accessible, focused, engaging discussions, physicist and award-winning
science writer Thomas Georges explores the fundamental issues: What is consciousness?
Can computers be conscious? If machines could think and even feel, would they then be
entitled to “human” rights? Will machines and people merge into a biomechanical race?
Should we worry that super-intelligent machines might take over the world?
Even now we continue to put increasingly sophisticated machines in control of critical
aspects of our lives in ways that may hold unforeseen consequences for the human race.
Digital Soul challenges all of us, before it’s too late, to think carefully and rationally about the
kind of world we will want to live in—with intelligent machines ever closerby our sides.
G
THOMAS M. GEORGES is a former research scientist at the National Bureau of E
O
Standards, the Institute for Telecommunication Sciences, and the National Oceanic and
R
Atmospheric Administration. He lives in Boulder, Colorado.
G
A CHOICEOutstanding Academic Title E
T H O M A S M . G E O R G E S
S
US $16.95 / CAN $23.95
Westview Press
www.westviewpress.com
digital soul
d i g i t a l s o u l
Intelligent Machines and Human Values
T H O M A S M . G E O R G E S
A Member of the Perseus Books Group
All rights reserved.Printed in the United States ofAmerica.No part ofthis publica-
tion may be reproduced or transmitted in any form or by any means,electronic or
mechanical,including photocopy,recording,or any information storage and retrieval
system,without permission in writing from the publisher.
Copyright © 2003 by Westview Press,A Member ofthe Perseus Books Group.
Hardcover edition published in 2003 in the United States by Westview Press,
A Member ofthe Perseus Books Group,and in the United Kingdom by Westview
Press,12 Hid’s Copse Road,Cumnor Hill,Oxford OX2 9JJ.Paperback edition
published by Westview Press in 2004.
Find us on the world wide web at www.westviewpress.com
Westview Press books are available at special discounts for bulk purchases in the
United States by corporations,institutions,and other organizations.For more
information,please contact the Special Markets Department at the Perseus Books
Group,11 Cambridge Center,Cambridge,MA 02142,or call (617) 252-5298,
(800) 255-1514 or email [email protected].
A Cataloging-in-Publication data record for this book is available from the
Library ofCongress.
Paperback:ISBN 0-8133-4266-X Hardcover:ISBN 0-8133-4057-8
The paper used in this publication meets the requirements ofthe American National
Standard for Permanence ofPaper for Printed Library Materials Z39.48–1984.
contents
Preface vii
1 Artificial Intelligence—That’s the Fake Kind,Right? 1
2 What Makes Computers So Smart? 9
3 What Do You Mean,Smarter than Us? 19
4 Machines Who Think 29
5 Let the Android Do It 47
6 What Is Intelligence? 57
7 What Is Consciousness? 75
8 Can Computers Have Emotions? 99
9 Can Your PC Become Neurotic? 115
10 The Moral Mind 123
11 Moral Problems with Intelligent Artifacts 135
v
vi Contents
12 The Moral Machine 151
13 Global Network to Global Mind 165
14 Will Machines Take Over? 173
15 Why Not Just Pull the Plug? 183
16 Cultures in Collision 189
17 Beyond Human Dignity 197
18 Extinction or Immortality? 209
19 The Enemy Within 217
20 Electronic Democracy 227
21 Rethinking the Covenant Between Science and Society 235
22 What About God? 257
Notes 271
About the Author 277
Index 279
preface
Most people have a short list of books that they discovered at pivotal
points in their life and that changed the way they think in crucial ways.
In a sense, this book began in the 1960s, when I picked up a copy of
Dean Wooldridge’s book The Machinery of the Brain.1 In this and his
two succeeding books,The Machinery ofLifeand Mechanical Man: The
Physical Basis of Intelligent Life, Wooldridge laid out an astounding
idea: that human beings function entirely according to the laws of
physics.2 This idea was both amazing and disturbing to me because it
upset a lifetime of Catholic doctrine about an immortal soul and the
spiritual nature of the human mind. In place of mysticism, it offered
the possibility that the whole of human experience is not only under-
standable but reproducible.Could it possibly be true? I had to find out
more.
My curiosity led me to books such as Huston Smith’s The Religions
of Man, B. F. Skinner’s Beyond Freedom and Dignity, Edward O. Wil-
son’s On Human Nature, Pamela McCorduck’s Machines Who Think,
Douglas Hofstadter’s Gödel, Escher, Bach, and Robert Wright’s Moral
Animal as well as the 1988 Public Broadcasting Service series Joseph
Campbell and the Power of Myth.3
I found out that the idea of man as a mechanism is an ancient one,
but that this idea has been eclipsed by mystical and spiritual views of
man promoted by the world’s religions. I learned that science is dis-
vii
viii Preface
covering that the most complex human behaviors,including ethical
and moral reasoning,are rooted in basic biological imperatives and
that brain research is revealing more and more mental functions,
like emotions and consciousness, as the workings of a wonderfully
complex information processor.
Adding to the evidence for physical explanations of intelligent
behavior are the great strides that have been made since the 1950s
toward developing machines with artificial intelligence, or AI. As a
result,machines that can learn are taking over more and more func-
tions that we have always assumed to be uniquely human, such as
playing championship chess. Given the present rate of growth of
computing power, some scientists seriously predict that machines
will become more intelligent and aware than we are in just thirty to
forty years—outperforming humans in every important way, and
perhaps even forming a global brain!
What are we to make of all this? Although we are still far from
creating an artificial human being,it might be wise to start thinking
about how the future intelligent machines could change the way we
see ourselves and even alter humankind’s future.A world populated
with intelligent artifacts that have minds and feelings of their own
raises many tricky questions,such as these:
• How would humans react on discovering that the club of sen-
tient beings is not as exclusive as they thought?
• How would that knowledge change our moral and ethical val-
ues?
• How would it affect our notions of freedom and dignity?
• How would it affect our beliefs in God?
• Can computers be conscious?
• Can they have emotions?
• If so,what are their rights and responsibilities?
• If we make something that is indistinguishable from a person,
should we treat it like one?
Preface ix
• Should we be worried that superintelligent machines will
somehow take over the world?
• Are intelligent machines the next step in evolution?
• Will humans and machines somehow merge?
• Will humans become extinct? Or immortal?
• And most important ... How much control do we have over
the process?
This book is not for experts in computers or artificial intelli-
gence. My audience is ordinary people who are curious enough to
ask questions like these,and who want to be able to make informed
decisions about the course of AI’s development.It is even for those
who think that all this is rubbish—that no matter how “smart”ma-
chines get,they will never truly think,have a soul,or be self-aware
in the same sense we are.Whatever your leanings, it should do no
harm to explore, poke around, ask questions, and try to find out
what makes machines so smart, what their inherent limitations
might be,and where the boundary between human and artificial in-
telligence might lie. If these questions make you vaguely uncom-
fortable,it may be because they challenge the very foundations ofall
our social,legal,and religious institutions.
Because there are already lots of books about the mechanics of
AI, as well as works of fiction and nonfiction that speculate about
future worlds inhabited by thinking machines,it is reasonable to ask
what new territory I hope to cover here. We will explore an unfa-
miliar land along the boundary between science and human values.
There,we will seek out the logical structure ofhuman feelings,con-
sciousness, and morality that would make it possible for machines
to possess them as well. Then we will delve into the social, moral,
ethical, and religious consequences of creating thinking and feeling
artifacts.
What sort ofconsequences? As any science-fiction fan knows,the
big problem with intelligent machines,going back as far as Franken-
