Table Of ContentP1:IML/SBA P2:IML/SBA QC:IML/SBA T1:IML
Riegelman-FM Riegelman-1490G Riegelman-v9.cls September1,2004 22:56
Studying a Study and Testing a Test
How to Read the Medical Evidence
FifthEdition
RichardK.Riegelman,m.d.,m.p.h.,phh.d.
ProfessorofEpidemiology-Biostatistics,Medicine,andHealthPolicy
andFoundingDean
TheGeorgeWashingtonUniversity
SchoolofPublicHealthandHealthServices
Washington,D.C.
i
P1:IML/SBA P2:IML/SBA QC:IML/SBA T1:IML
Riegelman-FM Riegelman-1490G Riegelman-v9.cls September1,2004 22:56
AcquisitionsEditor:DanetteSomers
ManagingEditor:MichelleM.LaPlante
ProjectManager:BridgettDougherty
SeniorManufacturingManager:BenRivera
MarketingManager:KathyNeely
CoverDesigner:HollyMcLaughlin
Compositor:TechBooks
Printer:EdwardsBrothers
(cid:2)C 2005byRichardK.Riegelman
PublishedbyLippincottWilliams&Wilkins
530WalnutStreet
Philadelphia,PA19106USA
LWW.com
Allrightsreserved.Thisbookisprotectedbycopyright.Nopartofthisbookmaybe
reproducedinanyformorbyanymeans,includingphotocopying,orutilizedbyany
informationstorageandretrievalsystemwithoutwrittenpermissionfromthecopyright
owner,exceptforbriefquotationsembodiedincriticalarticlesandreviews.Materials
appearinginthisbookpreparedbyindividualsaspartoftheirofficia dutiesasU.S.
governmentemployeesarenotcoveredbytheabove-mentionedcopyright.
PrintedintheUSA
LibraryofCongressCataloging-in-PublicationData
<COMP:setCIPhere>
Carehasbeentakentoconfir theaccuracyoftheinformationpresentedandtodescribe
generallyacceptedpractices.However,theauthors,editors,andpublisherarenot
responsibleforerrorsoromissionsorforanyconsequencesfromapplicationofthe
informationinthisbookandmakenowarranty,expressedorimplied,withrespecttothe
currency,completeness,oraccuracyofthecontentsofthepublication.Applicationofthis
informationinaparticularsituationremainstheprofessionalresponsibilityofthe
practitioner.
Theauthors,editors,andpublisherhaveexertedeveryefforttoensurethatdrugselection
anddosagesetforthinthistextareinaccordancewithcurrentrecommendationsand
practiceatthetimeofpublication.However,inviewofongoingresearch,changesin
governmentregulations,andtheconstantfl wofinformationrelatingtodrugtherapyand
drugreactions,thereaderisurgedtocheckthepackageinsertforeachdrugforanychange
inindicationsanddosageandforaddedwarningsandprecautions.Thisisparticularly
importantwhentherecommendedagentisaneworinfrequentlyemployeddrug.
SomedrugsandmedicaldevicespresentedinthispublicationhaveFoodandDrug
Administration(FDA)clearanceforlimiteduseinrestrictedresearchsettings.Itisthe
responsibilityofthehealthcareprovidertoascertaintheFDAstatusofeachdrugor
deviceplannedforuseintheirclinicalpractice.
10 9 8 7 6 5 4 3 2 1
ii
P1:IML/SBA P2:IML/SBA QC:IML/SBA T1:IML
Riegelman-FM Riegelman-1490G Riegelman-v9.cls September1,2004 22:56
Contents
Preface:PuttingProgressIntoPractice v
Acknowledgments vii
1. IntroductionandFlaw-CatchingExercise 1
SectionI.StudyingaStudy
2. TypesofStudiesandtheM.A.A.R.I.E.Framework 7
3. Method 16
4. Assignment 20
5. Assessment 25
6. Results 32
7. Interpretation 50
8. Extrapolation 58
9. RandomizedClinicalTrials 67
10. DatabaseResearch:NonconcurrentCohortStudies 89
11. Meta-analysis 99
12. QuestionstoAskandFlaw-CatchingExercises 116
SectionII.TestingaTest
13. Method 137
14. Assignment 143
15. Assessment 149
16. Results 157
17. Interpretation 164
18. Extrapolation 172
19. Screening 175
20. QuestionstoAskandFlaw-CatchingExercises 184
iii
P1:IML/SBA P2:IML/SBA QC:IML/SBA T1:IML
Riegelman-FM Riegelman-1490G Riegelman-v9.cls September1,2004 22:56
iv Contents
SectionIII.RatingaRate
21. MethodandAssignment 193
22. Assessment 200
23. Results 204
24. InterpretationandExtrapolation 211
25. LifeExpectancy 220
26. QuestionstoAskandFlaw-CatchingExercises 227
SectionIV.ConsideringCostsandEvaluatingEffectiveness
27. IntroductionandMethod 235
28. Assignment 246
29. Assessment 251
30. Results 260
31. Interpretation 269
32. Extrapolation 276
33. QuestionstoAskandFlaw-CatchingExercises 280
SectionV.AGuidetotheGuidelines
34. Method 293
35. Assignment 297
36. Assessment 300
37. Results 302
38. Interpretation 308
39. ExtrapolationandQuestionstoAsk 317
SectionVI.SelectingaStatistic
40. BasicPrinciples 323
41. UnivariableAnalysis 334
42. BivariableAnalysis 341
43. MultivariableAnalysis 350
44. SelectingaStatisticFlowcharts 362
Glossary 369
SubjectIndex 395
P1:IML/SBA P2:IML/SBA QC:IML/SBA T1:IML
Riegelman-FM Riegelman-1490G Riegelman-v9.cls September1,2004 22:56
Preface
Putting Progress Into Practice
Thepracticeofmedicineischangingatunprecedentedspeed.Today’sreason-
able assumption is outdated by tomorrow’s evidence. A deluge of data faces
usasweconfronttheonslaughtofhealthresearchliterature.Howisthebusy
student,residentorpractitionertodealwiththisdilemma?Effcientlyreading
theresearchevidenceisthekeytosuccessfullyputtingprogressintopractice.
The 5th edition of Studying a Study and Testing a Test: How to Read
the Medical Evidence opens up the door to the practice of evidence-based
medicine. For the frst time, this new edition provides a unifying structure,
the M.A.A.R.I.E. framework, for reading the full range of research articles
encountered by students, residents and practitioners of medicine and public
health.TheM.A.A.R.I.E.frameworkisthekeytothestep-by-stepquestions-
to-askapproachtoeffcientlyreadingthemedicalevidence.
TheStudyingaStudyOnlineWebsiteisanintegralpartofthe5thedition.
Itcanbefoundonthewebatwww.StudyingaStudy.com.ThisWebsitepro-
vides practice using the M.A.A.R.I.E. framework, interactive faw-catching
exercises,thefowchartofstatisticsandexamplesofhowtoreadrealjournal
articles. The book and the Web site together are designed to be useful to the
individualreader,residents’JournalClub,aswellasstudentsandfacultyina
widevarietyofclinicalandpublichealthdisciplines.
Popularfeaturesfrompreviouseditionshavebeenexpandedinthe5thedition
andupdatesadded.AnewsectioncalledAGuidetotheGuidelinesexamines
evidence-based recommendations for practitioners. New faw-catching exer-
cisesappearthroughoutthebook.Thefowchartofstatisticshasbeenupdated
andmoreexamplesadded.
The Studying a Study and Testing a Test approach to reading the health
research literature aims to help students, residents, and practitioners practice
evidence-basedmedicinebuiltuponastrongfoundationofresearchevidence.
Theaimistohelpyoueffcientlyreviewjournalarticlesandfeelconfdentin
your ability to fnd the faws that so often occur. It is important, however, to
rememberthateveryfawisnotfatal.Thegoalistorecognizethelimitations
ofresearchandtakethemintoaccountasyouputtheevidenceintopractice.
Onefnalwarningbeforeyouproceed.Readingthehealthresearchliterature
canbehabitforming.Youmayevenfnditenjoyable.
v
P1:IML/SBA P2:IML/SBA QC:IML/SBA T1:IML
Riegelman-FM Riegelman-1490G Riegelman-v9.cls September1,2004 22:56
vi
P1:IML/SBA P2:IML/SBA QC:IML/SBA T1:IML
Riegelman-FM Riegelman-1490G Riegelman-v9.cls September1,2004 22:56
Acknowledgements
Thechallengesassociatedwithproducingthe5theditionofStudyingaStudyand
Testing a Test have been made easier by the support and encouragement I have
receivedfromstudents,colleagues,andofcoursethemanyreadersoftheprevious
editionsfromaroundtheworld.
ThiseditionrequiredwritinganewsectioncalledAGuidetotheGuidelinesas
wellasextensivelyrewritingtheTestingaTestandRatingaRatesections.Allof
thesebeneftedfromfeedbackfromstudentsattheGeorgeWashingtonUniversity
School of Public Health and Health Services as well as the School of Medicine
andHealthSciences.
The Studying a Study Online Web site is a joint project with my wife Linda.
She has a unique ability to understand what technology can add to the learning
process,anintuitivesenseofhowtomaketechnologywork,andthepatienceto
explainitalltome.Asthe“contentprovider”Iremainresponsibleforwhatever
goeswrongonthewebbutshedeservesfullcreditforwhatevergoesright.
ThestaffofLippincottWilliamsandWilkinsincludingtheireditorsandwebstaff
havebeenessentialtomovingthisprojecttocompletion.ThestaffofTechBooks
have made the editing of the 5th edition move very smoothly. I am especially
gratefultoLymanLyons,thecopyeditor,whohashelpedensurethateverysentence
makessenseandeverywordiscarefullychosen.
Writinga5theditionofabookisanespeciallyenjoyableexperienceifitmeans
thatyougettobuilduponthepastwhilelookingtothefuture.Thishasbeenthe
casewiththe5th editionofStudyingaStudyandTestingaTest.Ihopeyouwill
enjoyreadingitasmuchasIenjoyedwritingit.
RichardRiegelmanM.D.,M.P.H.,Ph.D.
WashingtonD.C.July2004
vii
P1:IML/SBA P2:IML/SBA QC:IML/SBA T1:IML
Riegelman-FM Riegelman-1490G Riegelman-v9.cls September1,2004 22:56
viii
P1:IML/FFX P2:IML/FFX QC:IML/FFX T1:IML
GB072-01 Riegelman-1490G Riegelman-v9.cls August30,2004 9:14
1 Introduction and
Flaw-Catching Exercise
Thetraditionalcourseinreadingthehealthliteratureconsistsof“Here’sTheNew
EnglandJournalofMedicine.Readit!”Thisapproachisanalogoustolearningto
swimbythetotalimmersionmethod.Somepersonscanlearntoswimthisway,
ofcourse,butafewdrown,andmanylearntofearthewater.
In contrast to the method of total immersion, you are about to embark on a
step-by-step,active-participationapproachtoreadingthemedicalevidence.With
thetoolsthatyouwilllearn,youwillsoonbeabletoreadajournalarticlecriti-
callyandefficientl .Considerableemphasisisplacedontheerrorsthatcanoccur
in the various kinds of studies, but try to remember that not every fl w is fa-
tal. The goal of literature reading is to recognize the limitations of a study and
then put them into perspective. This is essential before putting the results into
practice.
Tomakeyourjobeasierweuseacommonframeworktoorganizeourreview
ofeachofthetypesofinvestigations.Beforedevelopingandillustratingthecom-
ponentsoftheframework,however,letusbeginwithafl w-catchingexercise.A
fl w-catchingexerciseisasimulatedjournalarticlecontaininganarrayoferrors
in each of the components of the framework. Read the following fl w-catching
exerciseandthentrytoanswertheaccompanyingquestions.
Cries Syndrome: Caused by Television or Just Bad Taste?
A medical condition known as Cries syndrome has been described as occurring
among children 7 to 9 years old. The condition is characterized by episodes of
uninterruptedcryinglastingatleastanhourperdayfor3consecutivedays.The
diagnosis also includes symptoms of sore throat, runny nose, and fever which
precedes the onset of the crying and are severe enough to keep the child out of
school.
Investigatorsidentifie 100childrenwithCriessyndrome.ForeachCriessyn-
dromechildaclassmatewaschosenforcomparisonfromamongthosewhodid
notmissschool.Thestudywasconductedmorethanonemonthaftertheonsetof
symptoms. The investigators examined 20 variables, which included all the fac-
torstheycouldthinkofasbeingpotentiallyassociatedwithCriessyndrome.They
collected data on all medication use, number of spankings, hours of television
viewing,andnumberofhoursathome,aswellas16othervariables.
Using pictures, they asked the children to identify the medications they had
takenwhiletheyhadCriessyndrome.TheirclassmateswithoutCriessyndrome
werealsoaskedtousethepicturestoidentifymedicationstakenduringthesame
time period. The investigators then asked each child to classify each medica-
tion taken as a good-tasting or bad-tasting medication. The data on spankings
1
P1:IML/FFX P2:IML/FFX QC:IML/FFX T1:IML
GB072-01 Riegelman-1490G Riegelman-v9.cls August30,2004 9:14
2 Ch.1. IntroductionandFlaw-CatchingExercise
wereobtainedfromtheprimarycaregiver.Theinvestigatorsfoundthefollowing
data:
Percentageofchildrenwhoreportedtakingbad-tastingmedication
Criessyndrome: 90%
Controls: 10%
Averagenumberofspankingsperday
Criessyndrome: 1
Controls: 2
Averagenumberoftelevisionviewinghoursperday
Criessyndrome: 8(range5to12)
Controls: 2(range0to4)
Among the 20 variables, analyzed one at a time, the above were the only ones
thatwerestatisticallysignifcantusingtheusualstatisticalmethods.Thepvalues
were 0.05 except for the hours of television, which had aP-value of 0.001. The
investigatorsdrewthefollowingconclusions:
1. Bad-tasting medication is a contributory cause of Cries syndrome because it
wasstronglyassociatedwithCriessyndrome.
2. SpankingprotectschildrenfromCriessyndromebecausethecontrolshadan
increasedfrequencyofbeingspanked.
3. Television viewing at least 4 hours per day is required for the development
of Cries syndrome because all children with Cries syndrome and none of the
controlswatchedtelevisionmorethan4hoursperdayduringtheperiodunder
investigation.
4. Because Cries syndrome patients were 9 times as likely to take bad-tasting
medication,theinvestigatorsconcludedthatremovingbad-tastingmedication
fromthemarketwouldeliminatealmost90%ofCriessyndromecasesamong
childrenlikethoseinthisinvestigation.
5. Inaddition,regularspankingofallchildren7to9yearsoldshouldbewidely
usedasamethodofpreventingCriessyndrome.
Nowtogetanideaofwhatyouwillbelearninginthe“StudyingaStudy”section,
seeifyoucananswerthefollowingquestions:
1. Whattypeofinvestigationisthis?
2. Whatisthestudyhypothesis?
3. Isthecontrolgroupcorrectlyassigned?
4. Arereportingand/orrecallbiaseslikelytobepresentinthisstudy?
5. Doesthemethodofdatacollectionraiseissuesofprecisionandaccuracy?
6. Istheestimateofthestrengthoftherelationshipperformedcorrectly?
7. Isstatisticalsignifcancetestingperformedcorrectly?
8. Isanadjustmentprocedureneeded?
9. Isanassociationestablishedbetweentheuseofbad-tastingmedicineandCries
syndrome?
10. IsitestablishedthatthespankingsoccurredpriortothedevelopmentofCries
syndrome?
11. Isitestablishedthatalteringthefrequencyofspankingswillalterthefrequency
ofCriessyndrome?
