Experiment!: Planning, Implementing and Interpreting

277 Pages·2003·3.38 MB·English

by  Oivind Andersson(auth.)

P1:OTA/XYZ P2:ABC
JWST195-fm JWST195-Andersson May23,2012 14:13 Printer:Yettocome Trim:246mm×189mm
Experiment!
P1:OTA/XYZ P2:ABC
JWST195-fm JWST195-Andersson May23,2012 14:13 Printer:Yettocome Trim:246mm×189mm
Experiment!
Planning, Implementing and Interpreting
¨
Oivind Andersson
Lund University, Sweden
A John Wiley & Sons, Ltd., Publication
P1:OTA/XYZ P2:ABC
JWST195-fm JWST195-Andersson May23,2012 14:13 Printer:Yettocome Trim:246mm×189mm
Thiseditionfirstpublished2012
©2012JohnWiley&Sons,Ltd
Registeredoffice
JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UnitedKingdom
Fordetailsofourglobaleditorialoffices,forcustomerservicesandforinformationabouthowtoapplyfor
permissiontoreusethecopyrightmaterialinthisbookpleaseseeourwebsiteatwww.wiley.com.
Therightoftheauthortobeidentifiedastheauthorofthisworkhasbeenassertedinaccordancewiththe
Copyright,DesignsandPatentsAct1988.
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,in
anyformorbyanymeans,electronic,mechanical,photocopying,recordingorotherwise,exceptaspermittedby
theUKCopyright,DesignsandPatentsAct1988,withoutthepriorpermissionofthepublisher.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbe
availableinelectronicbooks.
Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Allbrandnames
andproductnamesusedinthisbookaretradenames,servicemarks,trademarksorregisteredtrademarksoftheir
respectiveowners.Thepublisherisnotassociatedwithanyproductorvendormentionedinthisbook.This
publicationisdesignedtoprovideaccurateandauthoritativeinformationinregardtothesubjectmattercovered.
Itissoldontheunderstandingthatthepublisherisnotengagedinrenderingprofessionalservices.Ifprofessional
adviceorotherexpertassistanceisrequired,theservicesofacompetentprofessionalshouldbesought.
Thepublisherandtheauthormakenorepresentationsorwarrantieswithrespecttotheaccuracyorcompleteness
ofthecontentsofthisworkandspecificallydisclaimallwarranties,includingwithoutlimitationanyimplied
warrantiesoffitnessforaparticularpurpose.Thisworkissoldwiththeunderstandingthatthepublisherisnot
engagedinrenderingprofessionalservices.Theadviceandstrategiescontainedhereinmaynotbesuitablefor
everysituation.Inviewofongoingresearch,equipmentmodifications,changesingovernmentalregulations,and
theconstantflowofinformationrelatingtotheuseofexperimentalreagents,equipment,anddevices,thereader
isurgedtoreviewandevaluatetheinformationprovidedinthepackageinsertorinstructionsforeachchemical,
pieceofequipment,reagent,ordevicefor,amongotherthings,anychangesintheinstructionsorindicationof
usageandforaddedwarningsandprecautions.Thefactthatanorganizationorwebsiteisreferredtointhis
workasacitationand/orapotentialsourceoffurtherinformationdoesnotmeanthattheauthororthepublisher
endorsestheinformationtheorganizationorWebsitemayprovideorrecommendationsitmaymake.Further,
readersshouldbeawarethatInternetwebsiteslistedinthisworkmayhavechangedordisappearedbetween
whenthisworkwaswrittenandwhenitisread.Nowarrantymaybecreatedorextendedbyanypromotional
statementsforthiswork.Neitherthepublishernortheauthorshallbeliableforanydamagesarisingherefrom.
LibraryofCongressCataloging-in-PublicationData
Andersson,O¨ivind,1970–
Experiment!:planning,implementing,andinterpreting/O¨ivindAndersson.
p. cm.
ISBN978-0-470-68826-7(cloth)–ISBN978-0-470-68825-0(pbk.)
1.Research. 2.Experimentaldesign 3.Science–Methodology. I.Title.
Q180.A1A6252012
001.4(cid:2)34–dc23
2012008855
AcataloguerecordforthisbookisavailablefromtheBritishLibrary.
PrintISBN:Cloth:9780470688267Paper:9780470688250
Typesetin10/12ptTimesbyAptaraInc.,NewDelhi,India
InstructorscanaccessPowerPointfilesoftheillustrationspresentedwithinthistext,forteaching,at:
http://booksupport.wiley.com
P1:OTA/XYZ P2:ABC
JWST195-fm JWST195-Andersson May23,2012 14:13 Printer:Yettocome Trim:246mm×189mm
ForCarl-Johan,
anavidexperimenter
P1:OTA/XYZ P2:ABC
JWST195-fm JWST195-Andersson May23,2012 14:13 Printer:Yettocome Trim:246mm×189mm
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
PartOne UnderstandingtheWorld
1 You,theDiscoverer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 VenturingintotheUnknown. . . . . . . . . . . . . . . . . . . . . . 4
1.2 EmbarkingonaPh.D.. . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 TheArtofDiscovery. . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 AboutthisBook. . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 HowtoUsethisBook. . . . . . . . . . . . . . . . . . . . . . . . . 8
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 WhatisScience?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1 CharacteristicsoftheScientificApproach. . . . . . . . . . . . . . . . 11
2.2 TheInductiveMethod. . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 TheHypothetico-DeductiveMethod . . . . . . . . . . . . . . . . . . 16
2.4 ConsequencesofFalsification . . . . . . . . . . . . . . . . . . . . . 19
2.5 TheRoleofConfirmation . . . . . . . . . . . . . . . . . . . . . . . 21
2.6 PerceptionisPersonal. . . . . . . . . . . . . . . . . . . . . . . . . 23
2.7 TheScientificCommunity. . . . . . . . . . . . . . . . . . . . . . . 29
2.8 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3 Science’sChildhood. . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.1 Infancy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2 IonianDawn. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3 DivineMathematics. . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.4 Adolescence–Revolution!. . . . . . . . . . . . . . . . . . . . . . . 41
3.5 TheChildrenoftheRevolution. . . . . . . . . . . . . . . . . . . . . 47
3.6 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4 ScienceInclinedtoExperiment . . . . . . . . . . . . . . . . . . . . . . 53
4.1 Galileo’sImportantExperiment . . . . . . . . . . . . . . . . . . . . 54
4.2 ExperimentorHoax? . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.3 ReconstructingtheExperiment. . . . . . . . . . . . . . . . . . . . . 58
4.4 GettingtheSwingofThings. . . . . . . . . . . . . . . . . . . . . . 60
P1:OTA/XYZ P2:ABC
JWST195-fm JWST195-Andersson May23,2012 14:13 Printer:Yettocome Trim:246mm×189mm
viii Contents
4.5 TheMessagefromthePlane. . . . . . . . . . . . . . . . . . . . . . 62
4.6 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5 Scientists,EngineersandOtherPoets . . . . . . . . . . . . . . . . . . . 65
5.1 ResearchandDevelopment. . . . . . . . . . . . . . . . . . . . . . . 65
5.2 CharacteristicsofResearch. . . . . . . . . . . . . . . . . . . . . . . 68
5.3 BuildingTheories. . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.4 TheRelationshipbetweenTheoryandReality. . . . . . . . . . . . . . 75
5.5 Creativity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.6 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
PartTwo InterferingwiththeWorld
6 Experiment!. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.1 WhatisanExperiment?. . . . . . . . . . . . . . . . . . . . . . . . 83
6.2 Questions,AnswersandExperiments. . . . . . . . . . . . . . . . . . 85
6.3 AGalleryofExperiments . . . . . . . . . . . . . . . . . . . . . . . 88
6.4 ReflectionsontheExhibition. . . . . . . . . . . . . . . . . . . . . . 108
6.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7 BasicStatistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
7.1 TheRoleofStatisticsinDataAnalysis. . . . . . . . . . . . . . . . . 113
7.2 PopulationsandSamples. . . . . . . . . . . . . . . . . . . . . . . . 115
7.3 DescriptiveStatistics. . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.4 ProbabilityDistribution . . . . . . . . . . . . . . . . . . . . . . . . 122
7.5 TheCentralLimitEffect. . . . . . . . . . . . . . . . . . . . . . . . 126
7.6 NormalProbabilityPlots. . . . . . . . . . . . . . . . . . . . . . . . 129
7.7 ConfidenceIntervals. . . . . . . . . . . . . . . . . . . . . . . . . . 132
7.8 Thet-Distribution. . . . . . . . . . . . . . . . . . . . . . . . . . . 134
7.9 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
8 StatisticsforExperiments. . . . . . . . . . . . . . . . . . . . . . . . . 139
8.1 ATeatimeExperiment. . . . . . . . . . . . . . . . . . . . . . . . . 139
8.2 TheImportanceofRandomization. . . . . . . . . . . . . . . . . . . 141
8.3 One-SidedandTwo-SidedTests . . . . . . . . . . . . . . . . . . . . 142
8.4 Thet-TestforOneSample. . . . . . . . . . . . . . . . . . . . . . . 143
8.5 ThePowerofaTest. . . . . . . . . . . . . . . . . . . . . . . . . . 148
8.6 ComparingTwoSamples. . . . . . . . . . . . . . . . . . . . . . . . 150
8.7 AnalysisofVariance(ANOVA). . . . . . . . . . . . . . . . . . . . . 155
8.8 AMeasurementSystemAnalysis. . . . . . . . . . . . . . . . . . . . 159
8.9 OtherUsefulHypothesisTests. . . . . . . . . . . . . . . . . . . . . 163
P1:OTA/XYZ P2:ABC
JWST195-fm JWST195-Andersson May23,2012 14:13 Printer:Yettocome Trim:246mm×189mm
Contents ix
8.10 Interpretingp-Values. . . . . . . . . . . . . . . . . . . . . . . . . . 164
8.11 Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
8.12 RegressionModeling . . . . . . . . . . . . . . . . . . . . . . . . . 167
8.13 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
9 ExperimentalDesign. . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
9.1 StatisticsandtheScientificMethod. . . . . . . . . . . . . . . . . . . 175
9.2 DesignswithOneCategoricalFactor. . . . . . . . . . . . . . . . . . 176
9.3 SeveralCategoricalFactors:theFullFactorialDesign. . . . . . . . . . 178
9.4 AreInteractionsImportant? . . . . . . . . . . . . . . . . . . . . . . 186
9.5 FactorScreening:FractionalFactorialDesigns . . . . . . . . . . . . . 187
9.6 DeterminingtheConfoundingPattern. . . . . . . . . . . . . . . . . . 188
9.7 DesignResolution. . . . . . . . . . . . . . . . . . . . . . . . . . . 190
9.8 WorkingwithScreeningDesigns. . . . . . . . . . . . . . . . . . . . 191
9.9 ContinuousFactors:RegressionandResponseSurfaceMethods. . . . . 195
9.10 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
10 PhaseI:Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
10.1 TheThreePhasesofResearch. . . . . . . . . . . . . . . . . . . . . 211
10.2 Experiment1:VisualOrientationinaBeetle . . . . . . . . . . . . . . 213
10.3 Experiment2:Lift-OffLengthinaDieselEngine. . . . . . . . . . . . 216
10.4 FindingOutWhatisNotKnown. . . . . . . . . . . . . . . . . . . . 218
10.5 DeterminingtheScope. . . . . . . . . . . . . . . . . . . . . . . . . 221
10.6 ToolsforGeneratingHypotheses. . . . . . . . . . . . . . . . . . . . 222
10.7 ThoughtExperiments . . . . . . . . . . . . . . . . . . . . . . . . . 227
10.8 PlanningChecklist. . . . . . . . . . . . . . . . . . . . . . . . . . . 229
10.9 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
11 PhaseII:DataCollection. . . . . . . . . . . . . . . . . . . . . . . . . . 235
11.1 GeneratingUnderstandingfromData. . . . . . . . . . . . . . . . . . 235
11.2 MeasurementUncertainty. . . . . . . . . . . . . . . . . . . . . . . 236
11.3 DevelopingaMeasurementSystem. . . . . . . . . . . . . . . . . . . 238
11.4 MeasurementSystemAnalysis. . . . . . . . . . . . . . . . . . . . . 244
11.5 TheDataCollectionPlan. . . . . . . . . . . . . . . . . . . . . . . . 248
11.6 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
12 PhaseIII:AnalysisandSynthesis . . . . . . . . . . . . . . . . . . . . . 253
12.1 TurningDataintoInformation. . . . . . . . . . . . . . . . . . . . . 253
12.2 GraphicalAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . . . 256
12.3 MathematicalAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . 259
12.4 WritingaScientificPaper . . . . . . . . . . . . . . . . . . . . . . . 260
P1:OTA/XYZ P2:ABC
JWST195-fm JWST195-Andersson May23,2012 14:13 Printer:Yettocome Trim:246mm×189mm
x Contents
12.5 WritingaPh.D.Thesis. . . . . . . . . . . . . . . . . . . . . . . . . 264
12.6 Farewell. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
12.7 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
StandardNormalProbabilities. . . . . . . . . . . . . . . . . . . . . . . . 269
ProbabilityPointsforthet-Distribution. . . . . . . . . . . . . . . . . . . . 270
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
P1:OTA/XYZ P2:ABC
JWST195-bpreface JWST195-Andersson May14,2012 12:14 Printer:Yettocome Trim:246mm×189mm
Preface
Today,externallyfundedresearchistheruleratherthantheexception.Theproblemstate-
ments and hypotheses of a research project have already been formulated by a senior
researcher whenfilingthefundingapplication.Whentheresearchstudententersthepic-
ture a project plan has already been made. All that remains is to produce results in the
laboratory. This situation enhances the impression that many students have, that experi-
mentsaremainlyabouttakingmeasurements–abeliefarisingduringtraditional,structured
laboratoryclassesthatarebasedondemonstrationsratherthanrealexperiments.
Experiencedexperimentersknowthatthemaineffortsinasuccessfulexperimentliein
meticulousplanningandanalysis,andthatbothofthesemustbefirmlyrootedinascientific
approach. Entering the research process at such a late stage, students may be delayed in
realizing this crucial point. The art of experimentation is often learned by doing, so an
intuitiveunderstandingoftheexperimentalmethodusuallyevolvesgraduallyduringyears
oftrialanderror.Theaimofthisbookistospeedupthejourney.
Research education is often practically oriented, focusing on techniques and concepts
that are used in specific research areas. In order to become an independent researcher it
isimportanttoalsoacquiremoregeneralresearchcompetencies.Researchstudentsmust
adopt a scientific mindset, learn how to plan meaningful experiments, and understand
the fundamentals of collecting and interpreting data. This book focuses on these general
researchskills.Itisdirectedtoanyoneengagedinexperiments,andespeciallyPh.D.and
master’sstudentsjuststartingtodeveloptheirownexperiments.
Thereareseveralbooksthatdiscussthescientificmethodfromaphilosophicalstandpoint.
Therearealsogoodbookscoveringresearchmethodsfromastatisticalpointofview.Why
doweneedanotherbookonscientificmethod?Thereasonisthatscientistsneedtoemploy
several types of research competency in parallel, and it is unfortunate that these tend to
be taught separately. The philosophy of science is frequently introduced using historical
examplesincourseswheremoretimeisspentcriticizingideasthanexplaininghowtoapply
them. It is too common to teach statistics as a pure study of numbers without explaining
hownumbersmaybeturnedintomeaningfulscientificknowledge. Thelackofacontext
relevant to working scientists makes it difficult for research students to incorporate such
knowledgeintotheirotherresearchcompetencies.Thisbookaimstomergetheseaspects
ofresearchintoapracticallyworkablepackage.
The book is organized in two parts. The first part gives a general introduction to the
scientific approach while the second describes general methods and tools with a focus
on experiments. Towards the end of the book a methodology is presented, which leads
the reader through the three phases of an experiment: “Planning”, “Data Collection”,
and “Analysis and Synthesis”. The first phase puts the discussion about scientific and
experimental method into a practical context, treating how research problems may be
identifiedandhowtoapproachthembyconceivingmeaningfulexperiments.Thefollowing
twophasescontinuetobuildontheseideas,butincorporatestatisticaltechniquesintothe
process. In the data collection phase, a measurement system is devised, analyzed and
P1:OTA/XYZ P2:ABC
JWST195-bpreface JWST195-Andersson May14,2012 12:14 Printer:Yettocome Trim:246mm×189mm
xii Preface
improvedtoobtaindataofsufficientquality.Inthelastphase,therawdataareturnedinto
meaningful information, analyzed with graphical and mathematical tools, and connected
backtotheresearchquestion.
Itismyexperiencethatthismethodologyhelpsstudentsconnectthesomewhatabstract
concepts from statistical theory and the philosophy of science to their scientific praxis.
I am not an advocate of cookbook recipes for research and the methodology is not to
be viewed that way. The idea is rather that readers, by applying these simple tools to
their own research problems, will reflect on and learn to understand key elements of the
experimentalmethod.
Many people have kindly helped me in the preparation of this book. My friend and
colleague Rolf Egnell deserves a very special thank you for taking the time to read and
commentonthemanuscriptinitsentirety.Hehashadtheadmirableabilitytolookbeyond
the depths of his knowledge and read the text with the eyes of a layman. His valuable
commentshavesignificantlyimprovedthereadabilityofthistext.IamalsogratefultoTim
Davis for his comments on the statistics material, and to Staffan Ulfstrand for providing
feedback on parts where evolutionary biology is discussed. Advice and help of various
kinds were willingly given by Malte Andersson, Marie Dacke, Leif Lo¨nnblad, Carl-Erik
Magnusson,Cle´mentChartier,JohanZetterberg,andTonyGreenfield.Ialsowanttothank
Sarah Tilley at Wiley in Chichester, UK, for her support and encouraging remarks along
theway.IgratefullyacknowledgethesepeoplewhilestressingthatIam,ofcourse,solely
responsibleforanymistakesorotherauthorialshortcomingstillpresentinthetext.
Whenwritingthis,otherscometomindthathavehelpedmeovertheyears.Manyhave
had a hand in teaching me to see the world around me and to think independently, but
spacepreventsmefrommentioningthemall.Iespeciallyowethankstotheteachersand
staffattheDepartmentsofPhysicsandTheoreticalPhysicsinLund,whoalwayspresented
scientific ideas with a perfect blend of playfulness and intellectual sharpness. I am also
gratefultomymotherforherunconditionalsupportinalllife’sendeavors.Asachildshe
instilled in me that I succeeded with everything I tried my hand at, even though I often
didnot.
Aboveall,mydeepestandmostheartfeltgratitudegoestoGunnel,theloveofmylife,
and Carl-Johan, our son, for their sympathy, patience and laughter. You are the twin star
aroundwhichmyworldblissfullyrevolves.