Table Of Contentwww.allitebooks.com
The Leprechauns of Software
Engineering
How folklore turns into fact and what to
do about it
Laurent Bossavit
Thisbookisforsaleathttp://leanpub.com/leprechauns
Thisversionwaspublishedon2015-06-28
ThisisaLeanpubbook.Leanpubempowersauthorsand
publisherswiththeLeanPublishingprocess.LeanPublishingis
theactofpublishinganin-progressebookusinglightweight
toolsandmanyiterationstogetreaderfeedback,pivotuntil
youhavetherightbookandbuildtractiononceyoudo.
©2012-2015LaurentBossavit
www.allitebooks.com
Tweet This Book!
PleasehelpLaurentBossavitbyspreadingthewordaboutthis
bookonTwitter!
Thesuggestedhashtagforthisbookis#leprechauns.
Findoutwhatotherpeoplearesayingaboutthebookby
clickingonthislinktosearchforthishashtagonTwitter:
https://twitter.com/search?q=#leprechauns
www.allitebooks.com
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Chapter1:SoftwareEngineering’stelephonegame . . 3
Howwegotthere . . . . . . . . . . . . . . . . . . . . 3
Surfaceplausibility . . . . . . . . . . . . . . . . . . . 5
Leprechaunspotting . . . . . . . . . . . . . . . . . . 6
Whatyoucando . . . . . . . . . . . . . . . . . . . . 8
Chapter2:TheConeofUncertainty . . . . . . . . . . . 11
Howtofeelfoolishinfrontofaclass . . . . . . . . . 13
Makingsenseofthepicture . . . . . . . . . . . . . . 14
Gettingtothefacts . . . . . . . . . . . . . . . . . . . 15
Thetelephonegameinaction . . . . . . . . . . . . . 16
Controversy . . . . . . . . . . . . . . . . . . . . . . . 19
Whattomakeofallthis? . . . . . . . . . . . . . . . . 20
Chapter3:Whyyoushouldcareaboutempiricalresults 22
Theperilsofempiricalresearch . . . . . . . . . . . . 23
Disciplineenvy . . . . . . . . . . . . . . . . . . . . . 24
Scienceandreality . . . . . . . . . . . . . . . . . . . 26
Wheretogofromhere . . . . . . . . . . . . . . . . . 28
www.allitebooks.com
CONTENTS
Chapter4:Themessyworkingsofscientificdiscourse 31
Modalities . . . . . . . . . . . . . . . . . . . . . . . . 32
Citationasmodality . . . . . . . . . . . . . . . . . . 33
Theconstructionoffacts . . . . . . . . . . . . . . . . 34
Chapter5:Thehuntforthe10xfiles . . . . . . . . . . 36
Whyisthisimportant?Isn’titobvious? . . . . . . . . 36
Theimpressivelistofreferences . . . . . . . . . . . . 37
Theoriginalstudyandthe10xclaim . . . . . . . . . 38
Harshlycriticized . . . . . . . . . . . . . . . . . . . . 39
The10xfiles . . . . . . . . . . . . . . . . . . . . . . . 41
Goodstudy,badstudy . . . . . . . . . . . . . . . . . 43
Thewildgoosechase . . . . . . . . . . . . . . . . . . 44
Chapter6:Thevariableprogrammer . . . . . . . . . . 48
Gettingjusttheresultsyouwant . . . . . . . . . . . 48
Within-subjectvariability . . . . . . . . . . . . . . . 49
Rocketscience:theNASAdata . . . . . . . . . . . . . 50
Needleinahaystack . . . . . . . . . . . . . . . . . . 52
TheCOCOMOhaystacks . . . . . . . . . . . . . . . . 53
Environmentaleffects . . . . . . . . . . . . . . . . . 55
Summingup. . . . . . . . . . . . . . . . . . . . . . . 56
Interlude:HowToLie . . . . . . . . . . . . . . . . . . . 59
Chapter7:Who’safraidoftheBigBadWaterfall? . . 62
Thestandardstory . . . . . . . . . . . . . . . . . . . 63
Alternateendings . . . . . . . . . . . . . . . . . . . . 64
Justthefacts. . . . . . . . . . . . . . . . . . . . . . . 66
Nopaperisanisland . . . . . . . . . . . . . . . . . . 68
Latebloomer . . . . . . . . . . . . . . . . . . . . . . 72
Birthofamyth . . . . . . . . . . . . . . . . . . . . . 73
www.allitebooks.com
CONTENTS
Chapter8:Software’sperpetualcrisis . . . . . . . . . . 75
Chapter9:ALeprechaunhuntingtutorial . . . . . . . 78
Chapter10:Thecostofdefects:anillustratedhistory. 86
Origins . . . . . . . . . . . . . . . . . . . . . . . . . 86
Firstamendments . . . . . . . . . . . . . . . . . . . . 88
Where’sthedata? . . . . . . . . . . . . . . . . . . . . 90
Metamorphoses . . . . . . . . . . . . . . . . . . . . . 91
Changingthetopicaltogether . . . . . . . . . . . . . 96
Readingcurves . . . . . . . . . . . . . . . . . . . . . 98
Theory-ladendiagrams . . . . . . . . . . . . . . . . . 99
Boehm’sassent . . . . . . . . . . . . . . . . . . . . . 101
Chapter11:RocketscienceandFlaubertmath . . . . . 105
FlaubertandthemathofROI . . . . . . . . . . . . . 105
NASAIV&V’smath . . . . . . . . . . . . . . . . . . 107
Howoldisthecaptain? . . . . . . . . . . . . . . . . . 107
Eighy-three!Forsomevalueofeighty-three. . . . . . 109
Chapter12:Forsomevalueof56 . . . . . . . . . . . . 112
Wherebugscomefrom . . . . . . . . . . . . . . . . . 112
Samplesizeofone . . . . . . . . . . . . . . . . . . . 113
Poorrequirements . . . . . . . . . . . . . . . . . . . 114
Asoftwaretriumph . . . . . . . . . . . . . . . . . . . 115
Chapter13:Thecostofbadresearch . . . . . . . . . . 119
Uncriticalthinking . . . . . . . . . . . . . . . . . . . 119
Extraordinarilysuspectclaims . . . . . . . . . . . . . 120
Termsofinquiry . . . . . . . . . . . . . . . . . . . . 122
Researchstandards . . . . . . . . . . . . . . . . . . . 125
www.allitebooks.com
CONTENTS
Chapter14:Raisingthebar . . . . . . . . . . . . . . . . 127
Two modest proposals for publications on software
development . . . . . . . . . . . . . . . . . . . 127
Willyoutakethepledge? . . . . . . . . . . . . . . . . 127
Chapter15:Anewmodelofinquiry . . . . . . . . . . 129
TheOuroboroseffect:circularcausation . . . . . . . 130
FromRequirementsToNegotiation . . . . . . . . . . 131
Thecliffhanger . . . . . . . . . . . . . . . . . . . . . 132
AppendixA:bibliographicalanalysisofthe10xfiles . 135
Questionsofindirection . . . . . . . . . . . . . . . . 137
Summaryresults . . . . . . . . . . . . . . . . . . . . 138
Thequestforprimarysources . . . . . . . . . . . . . 140
Abetterlist:primarysourceswithempiricalevidence 141
Appendix B: bibliographical analysis for the “defect-
cost-increasecurve” . . . . . . . . . . . . . . . . . . 146
Theolderreferences . . . . . . . . . . . . . . . . . . 146
Thenewerreferences . . . . . . . . . . . . . . . . . . 159
AppendixC-Conceptionsandinventionofwaterfall 166
Inventionofwaterfall . . . . . . . . . . . . . . . . . . 166
Conceptionsofwaterfall(articlesbetween1970and1989)166
www.allitebooks.com
Preface
Thisbookisaworkinprogress.
I don’t know when, if ever, it will be finished. This is the new
normal for books, apparently. It has been the new normal for
softwareforalongtimenow.
Thesamplecontainstwochaptersinadditiontothispreface,and
shouldgiveyouanideaofthetopicandmywritingstyle.
I’mstillwritingfurthermaterial.Ifyoupurchasethebook,you
are entitled to free updates with all the new material that goes
intothisbook,atnoextracharge.
I have no schedule to finish the book, nor a very clear idea of
what it will be like when it’s done. Probably not too different
from what it is now, but assume that any of the book is up for
revision.
I’mwritingthisbookbecauseImust,notasmarketingmaterial
or anything else. These are topics that I find myself obsessed
with, and I find that the only way I get them out of my system
is to write about them. I write about them in various places,
and you can probably get your hand on nearly everything I’ve
writtenforfree.
TurningthesewritingsintoabookonlymeansthatI’mmaking
moreofanefforttoweaveacoherentstoryoutofwhatlearnings
I’vebeenabletoglean.
www.allitebooks.com
Preface 2
Bypurchasingthebook-atanypriceandinanystate-youare
supportingmeinthisefforttounderstandbetterwhatweknow
aboutsoftwareengineeringandwhywethinkweknowit.
I appreciate your feedback in any form: cold hard cash, a pat
on the back, a piece of gentle or harsh criticism. You’re even
welcometotellmewhatanidiotIam(andImayignoreyou).
Thankyou.
www.allitebooks.com
Chapter 1: Software
Engineering’s telephone
game
The software profession has a problem, widely recognized but
whichnobodyseemswillingtodomuchabout.Youcanthinkof
this problem as a variant of the well known “telephone game”,
wheresometrivialrumorisrepeatedfromonepersontothenext
until it has become distorted beyond recognition and blown up
outofallproportion.
Unfortunately, the objects of this telephone game are generally
consideredcornerstonetruthsofthediscipline,tothepointthat
theiracceptancenowhindersfurtherprogress.
It is not that these claims are outlandish in themselves; they
startedassomewhatreasonablehypotheses.Theproblemisthat
they have become entrenched as “fact” supposedly supported
by“research”,andattainedthiselevatedstatusinspiteofbeing
merelyanecdotal.
How we got there
One of the ways that anecdote persists is by dressing itself up
inthegarmentsofproperscholarship.Supposeyoucomeacross
thefollowingclaimforthefirsttime:
www.allitebooks.com
