Table Of Content·
Guerino Mazzola Gérard Milmeister
Jody Weissmann
Comprehensive Mathematics
for Computer Scientists 2
Calculus and ODEs, Splines, Probability,
Fourier and Wavelet Theory,
Fractals and Neural Networks,
Categories and Lambda Calculus
With 114 Figures
1 3
GuerinoMazzola
GérardMilmeister
JodyWeissmann
DepartmentofInformatics
UniversityofZurich
Winterthurerstr.190
8057Zurich,Switzerland
ThetexthasbeencreatedusingLATEX2ε.Thegraphicsweredrawnusingtheopen
sourceillustratingsoftwareDiaandInkscape,withalittlehelpfromMathematica.
ThemaintexthasbeensetintheY&YLucidaBrighttypefamily,theheadingin
BitstreamZapfHumanist601.
LibraryofCongressControlNumber:2004102307
MathematicsSubjectClassification(1998):00A06
ISBN3-540-20861-5SpringerBerlinHeidelbergNewYork
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthe
materialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,
recitation,broadcasting,reproductiononmicrofilmorinanyotherway,andstorageindata
banks.Duplicationofthispublicationorpartsthereofispermittedonlyundertheprovisions
oftheGermanCopyrightLawofSeptember9,1965,initscurrentversion,andpermission
forusemustalwaysbeobtainedfromSpringer.Violationsareliableforprosecutionunder
theGermanCopyrightLaw.
SpringerisapartofSpringerScience+BusinessMedia
springeronline.com
©Springer-VerlagBerlinHeidelberg2005
PrintedinGermany
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
Coverdesign:ErichKirchner,Heidelberg
Typesetting:Camerareadybytheauthors
Production:LE-TEXJelonek,Schmidt&V¨ocklerGbR,Leipzig
Printedonacid-freepaper 40/3142YL-543210
Preface
Thissecondvolumeofacomprehensivetourthroughmathematicalcore
subjects for computer scientists completes the first volume in two re-
gards:
Part III first adds topology, differential, and integral calculus to the top-
ics of sets, graphs, algebra, formal logic, machines, and linear geometry,
of volume 1. With this spectrum of fundamentals in mathematical edu-
cation, young professionals should be able to successfully attack more
involvedsubjects,whichmayberelevanttothecomputationalsciences.
Inasecondregard,theendofpartIIIandpartIVaddaselectionofmore
advanced topics. In view of the overwhelming variety of mathematical
approaches in the computational sciences, any selection, even the most
empirical, requires a methodological justification. Our primary criterion
has been the search for harmonization and optimization of thematic di-
versityandlogicalcoherence.Thisiswhywehave,forinstance,bundled
suchseeminglydistantsubjectsasrecursiveconstructions,ordinarydif-
ferential equations, and fractals under the unifying perspective of con-
tractiontheory.
For the same reason, the entry point to part IV is category theory. The
reader will recognize that a huge number of classical results presented
in volume 1 are perfect illustrations of the categorical point of view,
which will definitely dominate the language of mathematics and theo-
retical computer science of the decades to come. Categories are advan-
tageous or even mandatory for a thorough understandingof higher sub-
jects, such as splines, fractals, neural networks, and λ-calculus. Even for
the specialist, our presentationmay here and there offer a fresh view on
classicalsubjects.Forexample,thesystematicusageofcategoricallimits
VI Preface
inneuralnetworkshasenabledanoriginalformalrestatementofHebbian
learning,perceptronconvergence,andtheback-propagationalgorithm.
However, a secondary, but no less relevant selection criterion has been
applied. It concerns the delimitation from subjects which may be very
important for certain computational sciences, but which seem to be nei-
thermathematicallynorconceptuallyofgerminalpower.Inthisspirit,we
havealsorefrainedfromwritingapropercourseintheoreticalcomputer
science or in statistics. Such an enterprise would anyway have exceeded
by far the volume of such a work and should be the subject of a specific
educationincomputerscienceorappliedmathematics.Nonetheless,the
readerwillfindsomeinterfacestothesetopicsnotonlyinvolume1,but
alsoinvolume2,e.g.,inthechaptersonprobabilitytheory,insplinethe-
ory, and in the final chapter on λ-calculus, which also relates to partial
recursivefunctionsandtoλ-calculusasaprogramminglanguage.
We should not conclude this preface without recalling the insight that
there is no valid science without a thorough mathematical culture. One
ofthemostintriguingillustrationsofthisuniversal,butoftensurprising
presence of mathematics is the theory of Lie derivatives and Lie brack-
ets, which the beginner might reject as “abstract nonsense”: It turns out
(using the main theorem of ordinary differential equations) that the Lie
bracketoftwovectorfieldsisdirectlyresponsibleforthecontrolofcom-
plex robot motion, or, still more down to earth: to everyday’s sideward
parkingproblem.Wewishthatthereadermayalwayskeepinmindthese
universal tools of thought while guiding the universal machine, which is
thecomputer,tointelligentandsuccessfulapplications.
Zurich, GuerinoMazzola
August2004 GérardMilmeister
JodyWeissmann
Contents
III Topology and Calculus 1
27 LimitsandTopology 3
27.1Introduction............................................. 3
27.2TopologiesonRealVectorSpaces........................ 4
27.3Continuity............................................... 14
27.4Series ................................................... 21
27.5Euler’sFormulaforPolyhedraandKuratowski’sTheorem 30
28 Differentiability 37
28.1Introduction............................................. 37
28.2Differentiation .......................................... 39
28.3Taylor’sFormula ........................................ 53
29 InverseandImplicitFunctions 59
29.1Introduction............................................. 59
29.2TheInverseFunctionTheorem........................... 60
29.3TheImplicitFunctionTheorem .......................... 64
30 Integration 73
30.1Introduction............................................. 73
30.2PartitionsandtheIntegral ............................... 74
30.3MeasureandIntegrability................................ 81
31 TheFundamentalTheoremofCalculusandFubini’sTheorem 87
31.1Introduction............................................. 87
31.2TheFundamentalTheoremofCalculus .................. 88
31.3Fubini’sTheoremonIteratedIntegration................. 92
32 VectorFields 97
32.1Introduction............................................. 97
32.2VectorFields ............................................ 98
VIII Contents
33 Fixpoints 105
33.1Introduction............................................. 105
33.2Contractions ............................................ 105
34 MainTheoremofODEs 113
34.1Introduction............................................. 113
34.2ConservativeandTime-DependentOrdinary
DifferentialEquations:TheLocalSetup .................. 114
34.3TheFundamentalTheorem:LocalVersion................ 115
34.4TheSpecialCaseofaLinearODE ........................ 117
34.5TheFundamentalTheorem:GlobalVersion .............. 119
35 ThirdAdvancedTopic 125
35.1Introduction............................................. 125
35.2NumericsofODEs....................................... 125
35.3TheEulerMethod ....................................... 129
35.4Runge-KuttaMethods.................................... 131
IV Selected Higher Subjects 137
36 Categories 139
36.1Introduction............................................. 139
36.2WhatCategoriesAre..................................... 140
36.3Examples................................................ 143
36.4FunctorsandNaturalTransformations................... 147
36.5LimitsandColimits...................................... 153
36.6Adjunction.............................................. 159
37 Splines 161
37.1Introduction............................................. 161
37.2PreliminariesonSimplexes .............................. 161
37.3WhatareSplines?........................................ 164
37.4LagrangeInterpolation .................................. 168
37.5BézierCurves ........................................... 171
37.6TensorProductSplines .................................. 176
37.7B-Splines ................................................ 179
38 FourierTheory 183
38.1Introduction............................................. 183
38.2SpacesofPeriodicFunctions............................. 185
38.3Orthogonality ........................................... 188
Contents IX
38.4Fourier’sTheorem....................................... 191
38.5RestatementinTermsoftheSineandCosineFunctions.. 194
38.6FiniteFourierSeriesandFastFourierTransform ......... 200
38.7FastFourierTransform(FFT) ............................ 204
38.8TheFourierTransform .................................. 209
39 Wavelets 215
39.1Introduction............................................. 215
39.2TheHilbertSpaceL2(R) ................................. 217
39.3FramesandOrthonormalWaveletBases ................. 221
39.4TheFastHaarWaveletTransform........................ 225
40 Fractals 231
40.1Introduction............................................. 231
40.2Hausdorff-MetricSpaces................................. 232
40.3ContractionsonHausdorff-MetricSpaces ................ 236
40.4FractalDimension ....................................... 242
41 NeuralNetworks 253
41.1Introduction............................................. 253
41.2FormalNeurons ......................................... 254
41.3NeuralNetworks ........................................ 264
41.4Multi-LayeredPerceptrons ............................... 269
41.5TheBack-PropagationAlgorithm......................... 272
42 ProbabilityTheory 279
42.1Introduction............................................. 279
42.2EventSpacesandRandomVariables ..................... 279
42.3ProbabilitySpaces ....................................... 283
42.4DistributionFunctions................................... 290
42.5ExpectationandVariance................................ 299
42.6IndependenceandtheCentralLimitTheorem............ 306
42.7ARemarkonInferentialStatistics ....................... 310
43 LambdaCalculus 313
43.1Introduction............................................. 313
43.2TheLambdaLanguage................................... 314
43.3Substitution............................................. 316
43.4Alpha-Equivalence....................................... 318
43.5Beta-Reduction .......................................... 320
43.6Theλ-CalculusasaProgrammingLanguage.............. 326
X Contents
43.7RecursiveFunctions ..................................... 328
43.8RepresentationofPartialRecursiveFunctions............ 331
A FurtherReading 335
B Bibliography 337
Index 341
III
PART
Topology and Calculus
27
CHAPTER
Limits and Topology
27.1 Introduction
Thischapteropensalineofmathematicalthoughtandmethodswhichis
quitedifferentfrompurelyset-theoretical,algebraicandformallylogical
approaches: topology and calculus. Generally speaking this perspective
is about the “logic of space”, which in fact explains the Greek etymol-
ogy of the word “topology”, which is “logos of topos”, i.e., the theory
of space. The “logos” is this: We learned that a classical type of logical
algebras, the Boolean algebras, are exemplified by the power sets 2a of
given sets a, together with the logical operations induced by union, in-
tersection and complementation of subsets of a (see volume 1, chapter
3).Thelogicwhichisaddressedbytopologyisamorerefinedone,andit
appears in the context of convergent sequences of real numbers, which
wehavealready studiedinvolume1,section9.3,toconstructimportant
operations such as the n-th root of a positive real number. In this con-
text, not every subset of R is equally interesting. One rather focuses on
subsets C ⊂ R which are “closed” with respect to convergent sequences,
i.e., if we are given a convergent sequence (c ) having all its members
i i
ci ∈ C, then l = limi→∞ci must also be an element of C. This is a useful
property,sincemathematicalobjectsareoftenconstructedthroughlimit
processes, and one wants to be sure that the limit is contained in the
samesetthattheconvergentserieswasinitiallydefinedin.
Actually, for many purposes, one is better off with sets complementary
to closed sets, and these are called open sets. Intuitively, an open set
