Table Of Contentrev-panu.qxp 8/17/01 9:46 AM Page 992
Book Review
Exploring Randomness
and
The Unknowable
Reviewed by Panu Raatikainen
Exploring Randomness What Gödel proved in 1931 is that in any finitely
Gregory Chaitin presented system of mathematical axioms there are
Springer-Verlag, 2000 sentences that are true but that cannot be proved
ISBN 1-85233-417-7 to be true in the system. Church showed in 1936
176 pages, $34.95 that there is no general mechanical method for
deciding whether a given sentence is logically valid
The Unknowable or not and, similarly, that there is no method for
Gregory Chaitin deciding whether a given sentence is a theorem of
Springer-Verlag, 1999 a given axiomatized mathematical theory. Such an
ISBN 9-814-02172-5 impossibility proof required an exact mathemati-
122 pages, $29.00 cal substitute for the informal, intuitive notion of
a mechanical procedure; Church used his own λ-
In the early twentieth century two extremely in- definable functions. Turing arrived independently
fluential research programs aimed to establish at the same results at the same time. Moreover, he
solid foundations for mathematics with the help gave a superior philosophical explication of the con-
of new formal logic. The logicism of Gottlob Frege cept of mechanical procedure in terms of abstract
and Bertrand Russell claimed that all mathemat- imaginary machines, known today as Turing ma-
ics can be shown to be reducible to logic. David chines; this advance made it possible to prove ab-
Hilbert and his school in turn intended to demon-
solute unsolvability results and to develop Gödel’s
strate, using logical formalization, that the use of
incompleteness theorem in its full generality. This
infinistic, set-theoretical methods in mathemat-
identification of the intuitive notion of mechanical
ics—viewed with suspicion by many—can never
method and an exact mathematical notion is usu-
lead to finitistically meaningful but false state-
ally called Church’s thesis or, more properly, the
ments and is thus safe. This came to be known as
Church-Turing thesis. It is the fundamental basis
Hilbert’s program.
of all proofs of absolute unsolvability.
These grand aims were shown to be impossible
One of the greatest achievements of modern
by applying the exact methods of logic to itself:
mathematical logic was certainly the proof by Yuri
the limitative results of Kurt Gödel, Alonzo
Matiyasevich in 1970, based on earlier work by
Church, and Alan Turing in the 1930s revolution-
Julia Robinson, Martin Davis, and Hilary Putnam,
ized the whole understanding of logic and
that the tenth problem of Hilbert’s famous list of
mathematics (the key papers are reprinted in [5]).
open mathematical problems from 1900 is in fact
unsolvable; i.e., there is no general method for de-
Panu Raatikainen is a fellow in the Helsinki Collegium for
Advanced Study and a docent of theoretical philosophy ciding whether a given Diophantine equation has
at the University of Helsinki. His e-mail address is a solution or not [11]. This result implies that in
[email protected]. any axiomatized theory there exist Diophantine
992 NOTICESOFTHEAMS VOLUME48, NUMBER9
rev-panu.qxp 8/17/01 9:46 AM Page 993
equations that have no solution but cannot be a comprehensive survey of the theory and its
proved in the theory to have no solution. applications).
However, it is now a widespread view, espe- Chaitin was active in developing this approach
cially in computer science circles, that certain into a systematic theory (although one should
variants of incompleteness and unsolvability not ignore the important contributions by many
results by the American computer scientist Gregory others). From the 1970s onwards Chaitin’s inter-
Chaitin are the last word in this field. These vari- est has focused more and more on incompleteness
ants are claimed to both explain the true reason and unsolvability phenomena related to the notion
for Gödel’s incompleteness theorem and to be the of program-size complexity. Indeed, he now
ultimate, or the strongest possible, incomplete- says that “the most fundamental application” of
ness results. Chaitin’s results emerge from the the theory is in “the new light that it sheds on
theory of algorithmic complexity or program-size the incompleteness phenomenon” (The Unknow-
complexity (also known as “Kolmogorov com- able, pp. 86–7).
plexity”); Chaitin himself was, in fact, one of the It was known from the beginning that program-
founders of the theory. size complexity is unsolvable. Chaitin, however,
The classical work on unsolvability dealt solely made in the early 1970s an interesting observation:
with solvability in principle: one abstracted from Although there are strings with arbitrarily large
the practical limits of space and time and required program-size complexity, for any mathematical
only finiteness. From the late 1950s onward, how- axiom system there is a finite limit c such that in
ever, more and more attention has been paid to dif- that system one cannot prove that any particular
ferent kinds of complexity questions—at least in string has a program-size complexity larger than
part because of the emergence of computing ma- c [1]. Later Chaitin attempted to extend his “in-
chines and the practical resource problems that ac- formation-theoretic” approach to incompleteness
companied them. In logic and computer science theorems in order to obtain “the strongest possi-
various different notions of complexity have been ble version of Gödel’s incompleteness theorem” ([3],
studied intensively. First, computational complex- p. v). For this purpose he has defined a specific
itymeasures the complexity of a problem in terms infinite “random” sequence Ω.
of resources, such as space and time, required to As was noted, one of the major sources that
solve the problem relative to a given machine originally motivated the development of the
model of computation. Second, descriptive com- theory of program-size complexity, especially in
plexity analyzes the complexity of a problem in Kolmogorov’s case, was a problem in the theory
terms of logical resources, such as the number of of probability, viz. that of giving a precise and
variables, the kinds of quantifiers, or the length of plausible definition for the notion of a random
a formula required to define the problem. And fi- string. The problem is related to the paradox of
nally, by the algorithmic complexity, or the pro- randomness, which may be explained as follows:
gram-size complexity(or Kolmogorov complexity), Assume we are given two binary strings of 20
of a number or a string, one means the size of the digits each, and we are informed that they were
shortest program that computes as output that both obtained by flipping a coin. Let these two
number or string. strings be:
x=00000000000000000000
Theory of Algorithmic Complexity
The basic idea of the theory of algorithmic com- and
plexity was suggested in the 1960s independently
y =01001110100111101000.
by Ray J. Solomonoff, Andrei N. Kolmogorov, and
Gregory Chaitin. Solomonoff used it in his com- Now according to the standard theory of
putational approach to scientific inference, probability, these strings are equally probable.
Kolmogorov aimed initially to give a satisfactory And yet intuitively one tends to think that x
definition for the problematic notion of a random cannot possibly be a randomly generated string—
sequence in probability theory, and Chaitin there is too much regularity in it—whereas y
first studied the program-size complexity of appears to be genuinely irregular and random
Turing machines for its own sake. Kolmogorov and may well be the result of a toss of a coin. The
went on to suggest that this notion also provides algorithmic theory of randomness explicates this
a good explication of the concept of the informa- idea of regularity with the help of Turing machine
tion content of a string of symbols. Later Chaitin programs. One considers a finite string as regular,
followed him in this interpretation. Consequently, or nonrandom, if it can be generated by a simple
the name “algorithmic information content” has program, i.e., if its program-size complexity is
frequently been used for program-size complex- considerably smaller than its length. Accordingly,
ity, and the whole field of study is very often a finite string is defined to be random if its
called “algorithmic information theory” ([10] is program-size complexity is roughly equal to its
OCTOBER2001 NOTICESOFTHEAMS 993
rev-panu.qxp 8/17/01 9:46 AM Page 994
length, i.e., if it cannot be compressed to a shorter language provides exactly what is needed, so he
program. (Note that this notion is relative to a invented a new version of LISP. Chaitin begins The
chosen programming language or coding system; Unknowableby saying that what is new in the book
a finite string may be random in one but nonran- is the following: “I compare and contrast Gödel’s,
dom in another.) Turing’s and my work in a very simple and straight-
Extending this approach to infinite strings forward manner using LISP” (p. v). According to
turned out to be, however, more difficult than was Chaitin this book is a “prequel” to his previous
thought. Kolmogorov’s first book, The Limits of Mathematics(Springer-Verlag,
idea was that an infinite string 1998), and is an easier introduction to his work
be considered random if all of on incompleteness. In Exploring Randomness
its finite initial segments are Chaitin in turn writes that “[t]he purpose of this
random. But Per Martin-Löf book is to show how to program the proofs of the
showed that this definition does main theorems about program-size complexity,
not work and then gave a more so that we can see the algorithms in these proofs
satisfactory definition in mea- running on the computer” (p. 29).
sure-theoretic terms. In 1975 Such an approach may be attractive for pro-
Chaitin presented a definition gramming enthusiasts and engineers, but for the
(which is equivalent to Martin- rest of us its value is less clear. It is quite doubt-
Löf’s definition) in terms of ful whether it manages to increase the under-
program-size complexity: An standing of the basic notions and results and
infinite string is defined to be whether it really makes the fundamental issues,
random if the program-size which are rather theoretical and conceptual, more
complexity of an initial segment accessible. All these can be, and have been, ex-
of length ndoes not drop arbi- plained quite easily and elegantly in terms of sim-
trarily far below n [2]. One ply describable Turing machines, and it is ques-
should add that it is not indis- tionable that it is really easier to understand them
putable that this really provides by first learning Chaitin’s specially modified LISP
in all respects an unproblematic and then programming them. And after all, the key
explication of the notion of randomness. point here is that there is no program for decid-
Also in 1975 Chaitin presented for the first ing the basic properties—that they are not pro-
time his (since then much-advertised) number Ω: grammable. In The Unknowable (p. 27) Chaitin
it is the halting probability of the universal says that “[r]eaders who hate computer program-
Turing machine U, i.e., the probability that Uhalts ming should skip directly to Chapter VI”—that is,
when its binary input is chosen randomly bit by should skip half of the book (and similarly with Ex-
bit, such as by flipping a coin. The infinite string ploring Randomness). What is left is two popular
Ωis, according to the above definition, random [2]. surveys of the field.
Somewhat analogously to his earlier incomplete- What is totally missing from Chaitin’s accounts
ness result, Chaitin has demonstrated that no is the link between his particular programming
axiomatic mathematical theory enables one to language and the intuitive notion of mechanical
determine infinitely many digits of Ω ([3], [4]; procedure, that is, an analogue of the Church-
cf. [6], [8]). Turing thesis. Turing’s conceptual analysis of what
a mechanical procedure is and the resulting
Chaitin’s New Approach via LISP Programs Church-Turing thesis are indispensable for the
This is the general theoretical background of the proper understanding of the fundamental
books under review. How do these two new books unsolvability results: only the thesis gives them
by Chaitin relate to these older works? In terms of their absolute character (in contradistinction to
results there is hardly anything new compared to unsolvability by some fixed, restricted methods).
the older work by Chaitin and others reviewed Consequently, without some extra knowledge,
above. Rather, these books aim to popularize that the theoretical relevance of the limitations of the
work. In The Unknowablethe emphasis is on the LISP programs that Chaitin demonstrates may
incompleteness phenomena related to program- remain unclear to the reader: one may wonder
size complexity. Exploring Randomness aims to whether perhaps some other programming
explain the program-size complexity approach language would do better. This is a serious
to randomness. weakness of these presentations as first intro-
What is new is Chaitin’s approach via LISP- ductions to the unsolvability phenomena.
programming; he has “translated” his own
earlier work, which was in terms of abstract, Problematic Philosophical Conclusions
idealized Turing machines, into actual programs The most controversial parts of Chaitin’s work
in the LISP computer language. Well, not exactly: are certainly the highly ambitious philosophical
according to Chaitin, no existing programming conclusions he has drawn from his mathematical
994 NOTICESOFTHEAMS VOLUME48, NUMBER9
rev-panu.qxp 8/17/01 9:46 AM Page 995
work. Recall that according to Chaitin the most but their relevance for the foundations of mathe-
fundamental application of the theory is in the matics has been greatly exaggerated.
new light that it sheds on the incompleteness Further, Chaitin has often stated that he has
phenomenon. He writes: “Gödel and Turing were shown that mathematical truth is random: “But the
only the tip of the iceberg. AIT [algorithmic infor- bits of this number Ω, whether they’re 0 or 1, are
mation theory] provides a much deeper analysis mathematical truths that are true by accident!
of the limits of the formal axiomatic method. It …they’re true by no reason…there is no reason that
provides a deeper source of incompleteness, a individual bits are 0 or 1!” (Ex-
more natural explanation for the reason that no ploring Randomness, pp. 23–4)
finite set of axioms is complete” (Exploring This is false too. The individual
Randomness, p. 163). bits of Ω are 0 or 1 depending
But why does Chaitin think so? It is because on whether certain Turing ma-
he interprets his own variants of incompleteness chines halt or not—that is the
reason. It is an objective matter
theorems as follows: “The general flavor of my
of fact; the truth here is com-
work is like this. You compare the complexity of
pletely determined, and
the axioms with the complexity of the result you’re
Chaitin’s interpretation of the
trying to derive, and if the result is more complex
situation is quite misleading.
than the axioms, then you can’t get it from those
Chaitin also claims that Ω is
axioms” (The Unknowable, p. 24). Or, in other
“maximally unknowable” and
words: “my approach makes incompleteness more
that in his setting one gets in-
natural, because you see how what you can do
completeness and unsolvability
depends on the axioms. The more complex the
“in the worst possible way” (Ex-
axioms, the better you can do” (The Unknowable,
ploring Randomness, p. 19). But
p. 26).
contrary to what Chaitin’s own
But appearances notwithstanding, this is sim-
interpretations suggest, his re-
ply wrong. In fact, there is no direct dependence
sults can in fact be derived as
between the complexity of an axiom system and
quite easy corollaries of Tur-
its power to prove theorems. On the one hand,
ing’s classical unsolvability result and are not es-
there are extremely complex systems of axioms
sentially stronger than it. Moreover, there are many
that are very weak and enable one to prove only
incompleteness and unsolvability results in the
trivial theorems. Consider, for example, an enor-
literature of mathematical logic that are in various
mously complex finite collection of axioms with the
ways stronger than Chaitin’s results; many of them
form n<n+1; even the simple theory consisting
also have a much more natural mathematical
of the single generalization “for all x, x<x+1” can content [13].
prove more. On the other hand, there exist very sim- Chaitin, however, seems to be quite indifferent
ple and compact axiom systems that are to all such criticism. Instead of trying to seriously
sufficient for the development of all known answer it in any way, he sweeps all such problems
mathematics (e.g., the axioms of set theory) and under the carpet with rather cheap rhetoric: “AIT
that can in particular decide many more cases of is tremendously revolutionary; it is a major para-
program-size complexity than some extremely digm shift, which is why so many people find the
complex but weak axiom systems (such as the one philosophical conclusions that I draw from my
above). Moreover, it is possible for two theories to theory to be either incomprehensible or unpalat-
differ considerably in strength or complexity but able” (Exploring Randomness, p. 161). Or: “AIT is
nevertheless be able to decide exactly the same a drastic paradigm shift, and as such, obeys Max
facts about program-size complexity and have the Planck’s dictum that major new scientific ideas
same Chaitinian finite limit c [12]. Analogously, never convince their opponents, but instead are
Chaitin’s claim with respect to Ω that “an N-bit adopted naturally by a new generation that grows
formal axiomatic system can determine at most up with them and takes them for granted and that
N bits of Ω” (The Unknowable, p. 90) is again not have no personal stake nor have built careers on
true for related reasons [13]. older, obsolete viewpoints” (Exploring Random-
It has been shown conclusively (see [9], [12], [13]) ness, p. 163).
that Chaitin’s philosophical interpretations of his But wouldn’t it be conceivable that the true rea-
work are unfounded and false; they are based on son for some resistance is not dogmatic prejudice
various fatal confusions. And thus we have all the but that his conclusions are untenable because
more reason for doubting the claim that his they are very weakly justified and even contradict
approach can explain the true source of the various logico-mathematical facts? Creative and
incompleteness and unsolvability theorems. As original as Chaitin has been, it is sometimes quite
his philosophical interpretations fall, so does this disturbing that he seems totally ignorant of large
claim. Chaitin’s findings are not without interest, parts of mathematical logic relevant to the issues
OCTOBER2001 NOTICESOFTHEAMS 995
rev-panu.qxp 8/17/01 9:46 AM Page 996
he is dealing with. It is regrettable that Chaitin does of these matters, it is questionable whether these
not respond to criticism of his work but simply books are really a good place to start.
evades difficult questions and keeps on writing as (There is an errata for Exploring Randomness
if they did not exist. Chaitin’s own attitude begins on Chaitin’s home page: http://www.cs.
to resemble the dogmatism he accuses his oppo- auckland.ac.nz/CDMTCS/chaitin/).
nents of.
References
At worst, Chaitin’s claims are nearly megalo-
maniacal. What else can one think of statements [1] GREGORYJ. CHAITIN, Randomness and mathematical
proof, Sci. Amer.232:5 (1975), 47–52.
such as the following?: “AIT will lead to the major
breakthrough of 21st century mathematics, which [2] ——— , A theory of program size complexity for-
mally identical to information theory, J. Assoc. Com-
will be information-theoretic and complexity based
put. Mach.22(1975), 329–340.
characterizations and analyses of what is life, what [3] ———, Algorithmic Information Theory, Cambridge
is mind, what is intelligence, what is consciousness, University Press, Cambridge, 1987.
of why life has to appear spontaneously and then [4] ———, Randomness in arithmetic, Sci. Amer.259:1
to evolve” (Exploring Randomness, p. 163). (1988), 80–85.
[5] MARTIN DAVIS (ed.), The Undecidable, Raven Press,
Problems of Popularization New York, 1965.
The historical surveys of the theory of program- [6] JEAN-PAUL DELAHAYE, Chaitin’s equation: An exten-
sion of Gödel’s theorem, Notices Amer. Math. Soc.
size complexity that Chaitin gives are sometimes
36:8 (1989), 984–987.
rather distorted. Some have even complained that
[7] PETERGACS, Review of Gregory J. Chaitin, Algorith-
Chaitin is “rewriting the history of the field” and mic Information Theory, J. Symbolic Logic54(1989),
“presenting himself as the sole inventor of its main 624–627.
concepts and results” [7]. This complaint also fits [8] MARTINGARDNER, Mathematical games: The random
to a considerable degree the present books. They number Omega bids fair to hold the mysteries of the
are quite idiosyncratic. universe, Sci. Amer.241:5 (1979), 20–34.
The style of these books is very loose and pop- [9] MICHIELVANLAMBALGEN, Algorithmic information the-
ory, J. Symbolic Logic54(1989), 1389–1400.
ular. Large parts of the text are directly transcribed
[10] MING LI and PAUL VITANYI, An Introduction to Kol-
from oral lectures and include asides like “Thanks
mogorov Complexity and Its Applications, Springer-
very much, Manuel! It’s a great pleasure to be Verlag, New York, 1993.
here!” It is perhaps a matter of taste whether one [11] YURI V. MATIYASEVICH, Hilbert’s Tenth Problem, MIT
finds this entertaining or annoying. Personally, I Press, Cambridge, MA, 1993.
don’t think that this is a proper style for books in [12] PANURAATIKAINEN, On interpreting Chaitin’s incom-
pleteness theorem, J. Philos. Logic 27 (1998),
a scientific series. It certainly does not decrease the
269–586.
sloppiness of the text.
The books bring together popular and intro- [13] ——— , Algorithmic information theory and unde-
cidability, Synthese123(2000), 217–225.
ductory talks given on different occasions, and
some of this material has already appeared else-
where. There is also a lot of redundancy and over-
lap between the books. Both books (as well as the
earlier The Limits of Mathematics) start with a
loose and not very reliable historical survey—
Chaitin himself calls it “a cartoon summary”—
beginning with Cantor’s set theory, going through
Gödel’s and Turing’s path-breaking results, and
culminating, unsurprisingly, in Chaitin’s own work.
All three books then present an introduction to
LISP and Chaitin’s modification of it. Both The Un-
knowable and Exploring Randomness contain a
more theoretical section on algorithmic informa-
tion theory and randomness. And finally, both
books end with rather speculative remarks on the
future of mathematics. Would it have been better
to do some editing and publish just one book
instead of three?
To a considerable degree, Exploring Randomness
and The Unknowable just recycle the same old
ideas. Consequently, for those with some knowl-
edge of this field, these books do not offer anything
really new. For those with no previous knowledge
996 NOTICESOFTHEAMS VOLUME48, NUMBER9
