Computational Thinking: First Algorithms, Then Code

176 Pages·2018·2.11 MB·English

by  Paolo Ferragina

Paolo Ferragina · Fabrizio Luccio
Computational
Thinking
First Algorithms, Then Code
Computational Thinking
Paolo Ferragina • Fabrizio Luccio
Computational Thinking
First Algorithms, Then Code
Paolo Ferragina Fabrizio Luccio
Dipartimento di Informatica Dipartimento di Informatica
University of Pisa University of Pisa
Pisa, Italy Pisa, Italy
ISBN 978-3-319-97939-7 ISBN 978-3-319-97940-3 (eBook)
https://doi.org/10.1007/978-3-319-97940-3

Library of Congress Control Number: 2018952631

© Springer Nature Switzerland AG 2018
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or
information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt
from the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained herein
or for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
Themainpurposeofthistextistointroducereaderstoalgorithmsandtheircoding
forsolvingproblemsindifferentfields,someofwhichareofmajorimportancein
today’sworld.Thepresentationisaccessibletoanyonewithelementarymathemat-
ical knowledge and is made assimple as possible, without giving up the scientific
rigorthatisindispensablefortransforminggeneralideasintoexecutablealgorithms.
Inthecourseofthediscussion,however,arithmeticnotionsthatareparticularlyrel-
evantfortheconstructionofalgorithmsarereferredtoasneeded.
Tomakeitasusableaspossible,thetextisdividedintosubstantiallyindependent
chapters that can be read separately, leaving the reader the freedom to choose the
topicsheorsheprefers.Inafewcasesmorechallengingsideargumentshavebeen
reportedwhichcanbeomittedwithoutaffectingtheunderstandingofthemaintext.
In addition, readers interested in implementing the algorithms described in the
textonacomputermay,evenwithoutanyknowledgeof“programming”,examine
theiroperationandperhapstrytobuildtheirownprogramsthroughaveryaccessi-
blewebsitecreatedforthispurpose.1 Itcontainsthecodingofthesealgorithmsin
Pythonandprovideseverythingareaderwouldneedtoexecutethemand,perhaps,
constructnewexamples.
Pisa, PaoloFerragina
February2018 FabrizioLuccio
1https://github.com/ComputationalThinking-Springer/FirstAlgorithmsThenCode
v
Contents
1 ABriefHistoricalOutline....................................... 1
2 AProblemwithWhichtoBegin ................................. 5
3 AlgorithmsandCoding ......................................... 11
3.1 ThePseudocode............................................ 12
3.2 AMoreUsefulProgram..................................... 16
3.3 InteractingwiththeMachine ................................. 18
3.4 BinarySearch ............................................. 19
4 TheTournament ............................................... 25
4.1 ALowerBoundtotheProblemofMaximum ................... 28
4.2 TheKnockoutStage ........................................ 29
4.3 TheProblemoftheRunner-up................................ 32
5 AFinancialProblem ........................................... 35
5.1 A“Cubic”Algorithm ....................................... 38
5.2 A“Quadratic”Algorithm.................................... 39
5.3 A“Linear”Algorithm....................................... 41
5.4 AlgorithmEfficiencyVersusHardwareEfficiency ............... 44
5.5 SomeInterestingVariantsoftheProblem....................... 45
6 SecretMessages................................................ 49
6.1 Modularity ................................................ 51
6.2 ExponentialGrowth ........................................ 53
6.3 LeonBattistaAlberti’sDisk.................................. 56
6.4 Today’sCiphers............................................ 59
6.5 DispositionsofCharacters ................................... 60
6.6 TheConstructionofKeys.................................... 62
vii
viii Contents
7 PuttingThingsinOrder ........................................ 67
7.1 TheSortingProblem........................................ 69
7.2 AMoreEfficientAlgorithm.................................. 75
7.3 ALowerBound............................................ 80
7.4 SortingIntegers ............................................ 83
7.5 SortingBigData ........................................... 86
7.6 Multi-wayMerge(Sort) ..................................... 90
8 “Easy”and“Difficult”Problems................................. 95
8.1 WalkingThroughaCity ..................................... 96
8.2 P=NP? ...................................................107
9 SearchEngines ................................................111
9.1 FromWordstoHyperlinks...................................112
9.2 FourthGeneration ..........................................115
9.3 TheStructure ..............................................117
9.4 HowtoSolveaQuery.......................................119
9.5 StillanImprovement........................................125
9.6 AConcludingRemark ......................................127
10 DataCompression..............................................129
10.1 StatisticalCompressors......................................132
10.2 Dictionary-BasedCompressors ...............................137
10.3 Sort-BasedCompressors.....................................141
10.4 ConcludingRemarks........................................147
11 Recursion .....................................................149
11.1 AVeryInstructiveGame ....................................151
11.2 ComparisonBetweenRecursionandIteration...................155
11.3 OtherExamples............................................158
11.4 AFamousRandomizedAlgorithm ............................163
11.5 ToConclude,LetUsBeaLittleCautious ......................168
Index .............................................................173
Chapter 1
A Brief Historical Outline
Computation,inthesenseofcomputingnumbers,hasexistedsinceancienttimes.In
apapyruswrittenaround1650B.C.,thescribeAhmesdescribedtoEgyptianshow
to multiply by doubles, halves, and additions according to a method known long
before his writing. We begin with a modern version of this method, as an uncon-
ventionalintroductiontoourwork.InAhmes’spapyrustheproductfactorsaretwo
positiveintegers,saya,b.Thecalculationisbasedonthefollowingconsiderations:
1. ifa=1,a·b=b;
2. ifaiseven,a·b=(a/2)·(2b);and
3. ifaisoddandgreaterthan1,a·b=((a−1)/2)·(2b)+b.
Theproduct p=a·bisobtainedbyiteratingsteps2and3,whicheverapplies.At
eachiterationthevalueofthefirstfactorofthemultiplicationdecreases(abecomes
a/2or(a−1)/2),andtheiterationsgoonuntilthevalueofthisfactorbecomes1.
Atthispointstep1appliesandtheprocessends.Forexample,tocalculate p=6·7
proceedasfollows:
• 6iseven(step2),so p=(6/2)·(7·2)=3·14 (nowa=3andb=14);
• 3isoddandgreaterthan1(step3),sop=((3−1)/2)·(2·14)+14=2·14+14;
• 2iseven(step2),so p=(2/2)·(14·2)+14=1·28+14;
• thefirstfactorhasbecome1(step1),so p=28+14=42.
ThemethoddescribedbyAhmesiswhatisnowcalledan“algorithm,”atermun-
known to most people until a decade ago and very fashionable nowadays, even
though those who use it may not know what it exactly means. Its “realization” is
anexampleof“coding.”Itwasdoneinthepastbymeansofanabacus,whichmade
iteasytodouble,halve,andadd,andisexecutedtodaybytheelectroniccircuitsof
acomputer.
Algorithms and coding are the absolute protagonists of this book. The former
establish the logical rules to be used for solving problems, and the sequence in
which these rules must be applied. The latter commands a particular computing
devicetoexecutethestepsofthatsequence.Inourcase,thereferencedevicewill
be a computer and the coding will consist of a “program,” but the device may be
© Springer Nature Switzerland AG 2018 1
P. Ferragina, F. Luccio, Computational Thinking,
https://doi.org/10.1007/978-3-319-97940-3_1
2 1 ABriefHistoricalOutline
differentandtheborderbetweenalgorithmsandcodingmaynotbesharplydefined.
Computationalthinkingisbasedontheunderstandingoftheseconcepts.
The history of science, especially the history of mathematics, is full of com-
puting processes conceived before the concept of an algorithm had been precisely
defined.Inallthesestudies,whethertheyhadapracticalpurposeorwereabstract
speculation,arithmeticandgeometrywerealmosttheonlyprotagonists.Nowadays,
thesituationhaschangedsubstantially.Eventhosewhouseasmartphonetomake
phone calls, listen to music, take pictures, or access the Internet, put into action a
completely invisible but very complicated “computation” that allows them to op-
erateonsounds,pictures,orsequencesofcharacters.Thiscomputationondataof
any kind is essential in the present world, even though numbers still play an im-
portantrole.Infact,nootherfieldofknowledgedevelopedwiththeextraordinary
speed that has characterized the science of computation. The origins are found in
the studies of mathematics and mathematical logic developed between the end of
thenineteenthandthebeginningofthetwentiethcentury,andthekeyturningpoints
occurred in 1936, with the formal definition of algorithms and computable func-
tions, and in 1971, with the discovery of problems that are intractable in practice.
Theseweretwomomentsinwhichfundamentaldefinitionswerelaiddownandthe
limitations of computation were specified. Let us summarize some phases of this
initialdevelopment,whichconstitutetheculturalbasisofourfield.
At the end of the nineteenth century, the German mathematician Georg Cantor
formulatedhistheoryofinfinitesets,initiallyrejectedbymany,andthenassumed
asoneofthebasicsofmathematics.Inparticular,Cantorprovedthattherecannotbe
a one-to-one correspondence between integers and real numbers: intuitively, there
areinfinitely“fewer”integersthanrealnumbers.Withinthistheoryitcaneasilybe
shown that the sequences of finite (but a priori unlimited) length constructed with
anarbitrarybutfinitealphabetarenumerable,thatis,theycanbeputinone-to-one
correspondence with the integers. An obvious example is given by the sequences
constructedwiththedigits0to9thatdescribetheintegersthemselves,eachofwhich
has a finite length, without, however, the possibility of fixing a limit to the length
ofthesequencerepresentinganinteger.Similarly,alltheinfinite“words”thatcan
beconstructedwitharbitrarysequencesofthelettersofouralphabetarenumerable,
leavingasidewhetherornottheyaremeaningful.Whereas,thesequencesofinfinite
length are not numerable, that is, there are (many) more of them than there are
integers.Inparticular,realnumberstypicallyrequireaninfinitenumberofdigitsto
berepresented,sotheyarenotnumerable,asCantorshowed.
Allofthishasanimmediateconsequenceontheuseofalgorithmsforcalculating
mathematical functions. In fact, even before formally defining them, we may note
that algorithms must be described with finite sequences of characters in a natural
orartificiallanguage.Thesesequencescanbeofanylengthand,accordingtowhat
we said above, the algorithms must be numerable. By contrast, functions such as
y= f(x),i.e.lawsthatestablishacorrespondencebetweeninfinitevaluesofxand
thecorrespondingvaluesofy,arenotnumerablebecausethetwoinfinitesequences
forxandymustbefullyspecifiedifthereisnoruletoestablishthevalueofyforany
valueofx.Since,aspreviouslymentioned,thesequencesofinfinitelengtharenot
1 ABriefHistoricalOutline 3
numerable,thefunctionsarenotnumerableeither.Itfollowsthattherearefunctions
for which no computing algorithm can exist because all possible algorithms are
infinitely“fewer”thanallpossiblefunctions.Thatis,theremustbenon-computable
functionsandthesearetheabsolutemajorityofallpossiblefunctions.
Thisconclusionhadafundamentalimpactontheemergingtheoryofalgorithms.
In fact, you can calculate a function by specifying which rules to use among the
known ones. For example, to calculate y=x2 you can use Ahmes’s algorithm to
multiplyxbyitself,butitisobviouslynotpossibletofindanon-computablefunc-
tion without giving a general definition of an algorithm. In 1936, the American
logicianAlonsoChurchandtheEnglishmathematicianAlanTuringindependently
defined two computing models, respectively known as λ-calculus and the Turing
machine, which constitute two formal definitions of an algorithm. It quickly be-
cameapparentthatthetwomodelswereequivalentinthesensethattheywereable
tocalculateexactlythesamefunctions,andthereisstrongevidencethatinfactthey
can calculate all the functions that could be calculated with pen and paper. In the
samestudies,thefirstexamplesofnon-computablefunctionswerepresented.
Thetwomodelshaveservedasabasicreferenceforthedevelopmentofcomput-
ersandtheirprograms.Infact,thedefinitionofanalgorithmisfundamentaltothe
field of theoretical computer science but too subtle and complex for the purposes
ofthisbook.Itwillsufficetoindicateasimplerbutsubstantiallycorrectdefinition,
thatis:analgorithmisafinitesequenceofelementarystepsleadingtotheresolution
ofaproblem.Inaddition,analgorithmsatisfiesthefollowingproperties:(1)itcan
beusedondifferentinputsgeneratingcorrespondingoutputs,(2)eachstepallowsa
singleunambiguousinterpretationandisexecutableinafiniteamountoftime,and
(3)whatevertheinput,itsexecutioneventuallystops.
TheScienceofComputationisaspecificfieldofstudydevelopedfromChurch’s
andTuring’smodels.Itisaddressedtosolvingproblemsinthebroadsenseandnot
just calculating mathematical functions, but this distinction does not really exist.
Just as the calculation of a function y= f(x) transforms x values into y values, so
aproblemresolutionprocessesdatatogenerateresults.Thisamountstocalculating
afunctioninChurch’sandTuring’smodelsbecausedataandresultsaretreatedas
charactersequencesfromanarbitraryalphabet,andarithmeticdataprocessingdoes
not differ in principle from that of other genres. After this was made clear, partic-
ularattentionwaspaidtotheresourcesneededtomakeacalculation,inparticular
the time and the space required by an algorithm, leading to studies on computa-
tionalcomplexity. Atfirstglance,itmayseemcontradictorythatdizzyingadvances
in hardware have required increased attention to the quality of software, because
amorepowerfulcomputershouldcushionlowerefficiencyinitsuse.However,the
needtodealwithbiggerdatahasstimulatedthedevelopmentofefficientalgorithms,
partly because advances in hardware only marginally influence calculation time if
thequalityofthealgorithmsemployedislow,aswewillexplainbelow.In1971,the
CanadiancomputerscientistStephenCookandtheRussianmathematicianLeonid
Levin independently identified a family of problems that, while solvable by com-
putablefunctions,areinfactintractablebecausethenumberofoperationsrequired