Table Of ContentCambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
Robustness Tests for Quantitative Research
Theuncertaintyresearchersfaceinspecifyingtheirestimationmodelsthrea-
tensthevalidityoftheirinferences.Inregressionanalysesofobservational
data the “true model” remains unknown and researchers face a choice
between plausible alternative specifications. Robustness testing allows
researchers to explore the stability of their main estimates to plausible
variationsinmodelspecifications.Thishighlyaccessiblebookpresentsthe
logicofrobustnesstesting,providesanoperationaldefinitionofrobustness
that can be applied in all quantitative research and introduces readers to
diverse types of robustness tests. Focusing on each dimension of model
uncertaintyinseparatechapters,theauthorsprovideasystematicoverview
ofexistingtestsanddevelopmanynewones.Whetheritbeuncertaintyabout
thepopulationorsample,measurement,thesetofexplanatoryvariablesand
their functional form, causal or temporal heterogeneity, effect dynamics or
spatial dependence, this book provides guidance and offers tests that
researchersfromacrossthesocialsciencescanemployintheirownresearch.
eric neumayer isProfessorofEnvironmentandDevelopmentandPro-
Director Faculty Development at the London School of Economics and
PoliticalScience(LSE).
thomas plu¨mper is Professor of Quantitative Social Research at the
ViennaUniversityofEconomicsandBusiness.
Both authors have published in highly ranked journals including the
American Journal of Public Health, Annals of the American Association
ofGeographers,InternationalOrganization,PoliticalAnalysisandWorld
Development.
© in this web service Cambridge University Press www.cambridge.org
Cambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
MethodologicalToolsintheSocialSciences
editors
PaulM.Kellstedt,TexasA&MUniversity
GuyD.Whitten,TexasA&MUniversity
The Methodological Tools in the Social Sciences series is comprised of
accessible, stand-alone treatments of methodological topics encountered
by social science researchers. The focus is on practical instruction for
applying methods, for getting the methods right. The authors are leading
researchers able to provide extensive examples of applications of the
methods covered in each book. The books in the series strike a balance
between the underlying theory and the implementation of the methods.
They are accessible and discursive, and make technical code and data
available to aid in replication and extension of the results, as well as
enabling scholars to apply these methods to their own substantive
problems. They also provide accessible advice on how to present results
obtainedfromusingtherelevantmethods.
© in this web service Cambridge University Press www.cambridge.org
Cambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
Robustness Tests
for Quantitative
Research
EricNeumayer
LondonSchoolofEconomicsandPoliticalScience
ThomasPlümper
ViennaUniversityofEconomicsandBusiness
© in this web service Cambridge University Press www.cambridge.org
Cambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
4843/24,2ndFloor,AnsariRoad,Daryaganj,Delhi–110002,India
79AnsonRoad,#06–04/06,Singapore079906
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781108415392
DOI:10.1017/9781108233590
©EricNeumayerandThomasPlümper2017
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2017
PrintedintheUnitedKingdombyClays,StIvesplc
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloging-in-PublicationData
Names:Neumayer,Eric,1970–author.|Plümper,Thomas,author.
Title:Robustnesstests:causalinferencewithobservationaldata/EricNeumayer,
LondonSchoolofEconomicsandPoliticalScience,ThomasPlümper.
Description:Cambridge,UnitedKingdom;NewYork,NY,USA:University
PrintingHouse,[2017]|Includesbibliographicalreferencesandindex.
Identifiers:LCCN2017012398|ISBN9781108415392(hardback:alk.paper)|
ISBN9781108401388(paperback:alk.paper)
Subjects:LCSH:Robuststatistics.|Socialsciences–Methodology.
Classification:LCCHA31.5.N482017|DDC519.5/4–dc23
LCrecordavailableathttps://lccn.loc.gov/2017012398
ISBN978-1-108-41539-2Hardback
ISBN978-1-108-40138-8Paperback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracyof
URLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
© in this web service Cambridge University Press www.cambridge.org
Cambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
Contents
ListofFigures pagevi
ListofTables vii
ListofRobustnessTests viii
Acknowledgments xiii
1. Introduction 1
PART1: Robustness–AConceptualFramework 9
2. CausalComplexityandtheLimitstoInferential
Validity 11
3. TheLogicofRobustnessTesting 23
4. TheConceptofRobustness 34
5. ATypologyofRobustnessTests 52
6. AlternativestoRobustnessTesting? 67
PART2: RobustnessTestsandtheDimensionsofModel
Uncertainty 83
7. PopulationandSample 85
8. ConceptValidityandMeasurement 110
9. ExplanatoryandOmittedVariables 130
10. FunctionalFormsBeyondDefault 144
11. CausalHeterogeneityandContextConditionality 157
12. StructuralChangeasTemporalHeterogeneity 176
13. EffectDynamics 190
14. SpatialCorrelationandDependence 211
15. Conclusion 229
References 232
Index 252
© in this web service Cambridge University Press www.cambridge.org
Cambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
Figures
4.1: Example1ofDegreeofRobustnessρ page39
4.2: Example2ofDegreeofRobustnessρ 39
4.3: Example3ofDegreeofRobustnessρ 40
4.4: ρasaFunctionoftheDifferenceinPointEstimatesand
StandardErrors 42
4.5: ρasaFunctionoftheDifferenceinPointEstimatesand
StandardErrors(HeatPlot) 43
4.6: AnExampleofPartialRobustness 50
7.1: JackknifeTest:CorruptCountries 96
7.2: JackknifeTest:Non-corruptCountries 97
8.1: ErrorInjectionTest:CorruptCountries 126
8.2: ErrorInjectionTest:Non-corruptCountries 127
9.1: VisualizationoftheBetween-variationTest 138
10.1: Higher-degreePolynomialTest1 151
10.2: Higher-degreePolynomialTest2 152
11.1: ConditionalityTest 171
11.2: Non-linearConditionalityTest 172
12.1: TrendedEffectTest 185
© in this web service Cambridge University Press www.cambridge.org
Cambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
Tables
2.1: ConceptsofCausalityandtheSocialWorld page15
4.1: DegreeofRobustnessforVariousRobustness
TestEstimates 41
7.1: PopulationBoundaryTest 95
7.2: OutlierEliminationTest 98
7.3: StratificationTests 104
7.4: MultipleImputationTest 108
8.1: MeasurementErrorInjectionTest 125
9.1: Between-variationTest 137
9.2: GroupwiseFixed-effectsTest(replacementtype) 139
9.3: GroupwiseFixed-effectsTest(robustnesslimittype) 141
9.4: Spatial-errorTest 142
10.1: Higher-degreePolynomialsTests 151
11.1: Group-specificEffectTest 164
11.2: ChowTestofGroupHeterogeneity 166
11A.1: FunctionalFormTest:QuadraticRelationship 174
11A.2: FunctionalFormTest:Bi-linearConditional
Relationship 174
12.1: TrendedEffectTest 184
12.2: ChowTestofTemporalHeterogeneity 187
13.1: EffectOnsetTest 200
13.2: EffectOnsetTestwithAlternativeDynamicSpecification
Test 201
13.3: EffectOnsetandRandomCoefficientsDynamicsTests 204
14.1: SpatialSpecificationTests 218
14.2: Spatial-errorTests 219
14.3: AlternativeConnectivityTest 222
14.4: HeterogeneousExposureTest 224
14.5: HeterogeneousResponsivenessTest 225
© in this web service Cambridge University Press www.cambridge.org
Cambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
Robustness Tests
PopulationDefinitionandSampleTests
Name Action Pages
Population a)Includesobservationsthatmaynot 94
boundarytest belongtopopulation
b)Excludesobservationsthatmaybelong
topopulation
c)Structuredpermutationvariants
Bootstraptest Resampleswithreplacement 95
Jackknifetest Dropsoneormoreobservationsatatime 95
Coregrouptest Includesonlycasesknowntobeinthe 97
population
Outlier Drops“outliers” 97
eliminationtest
Cross-validation Drawsnewsample 100
test
Selectiontest Assumescasesareselected 101
Stratificationtest a)Over-samplesunder-representedcases 103
robustnesslimitvariant
b)Under-samplesover-representedcases
robustnesslimitvariant
Interpolationtest Replacesmissingsbyinterpolatedvalues 105
Out-of-sample Replacesmissingsbytheory-basedoutof 106
predictiontest samplepredictions
Multiple Replacesmissingsbymultiplyimputed 106
imputationtest values
© in this web service Cambridge University Press www.cambridge.org
Cambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
ListofRobustnessTests ix
ConceptValidityandMeasurementTests
Name Action Pages
Alternativeproxy Replacesoneproxyvariablewithanother 118
test
Principal Combinesmultipleproxiesintosingle 118
componenttest principalcomponent
Randomized Randomizesweightsofcomponentsof 119
componentstest compositevariable
Principal Adds“jitter”toweightsofprincipal 120
componentjitter component
test
Rescalingtest Changesthescaleofavariable 124
Measurement Adds“artificial”measurementerror 124
errorinjectiontest
Robustnesslimitvariant
Re-categorization Changestheassignedcategoriesof 127
test categoricalvariables
ExplanatoryandOmittedVariablesTests
Name Action Pages
Explanatory Addsand/orremovesright-hand-side 133
variablestest variables
Between- Stepwisereducesthebetween-variation 136
variationtest inthedata
Groupwise Reducesthebetween-variationby 138
fixed-effectstest includinggroupdummiesforsimilarcases
Robustnesslimitvariant
Correlated Addsartificialvariablewithdefined 140
artificialvariable properties
test
Spatial-errortest Addsspatial-errorvariable 141
© in this web service Cambridge University Press www.cambridge.org
Cambridge University Press
978-1-108-41539-2 — Robustness Tests for Quantitative Research
Eric Neumayer , Thomas Plümper
Frontmatter
More Information
x ListofRobustnessTests
FunctionalFormTests
Name Action Pages
Higherdegree Relaxesthefunctionalformbyestimating 149
polynomialtest ahigher-degreepolynomialmodel
Semi-parametric Relaxesthefunctionalformbyestimating 154
test asemi-parametricmodel
Functionalform Allowsfordifferenteffectstrengthat 155
breaktest breakpoint
CausalHeterogeneityandContextConditionalityTests
Name Action Pages
Bootstraptest Teststhehomogeneityassumptionby 162
resamplingwithreplacement
Samplesplittest Teststhehomogeneityassumptionby 162
splittingthesample
Groupwise Teststhehomogeneityassumptionby 163
jackknifetest groupwisedroppingcases
Unit-specific Relaxesthehomogeneityassumptionby 163
effecttest estimatingcase-specificeffects
Group-specific Relaxesthehomogeneityassumptionby 163
effecttest estimatinggroup-specificeffects
Random Allowsrandomvariationineffects 165
coefficientstest
Multileveltest Allowsclusteredvariationineffects 165
Chowtestof Relaxesthehomogeneityassumptionby 165
group interactingvariableswithgroupdummies
heterogeneity
Randomizedpermutationvariant
Conditionality Relaxesthehomogeneityassumptionby 168
test estimatinganinteractioneffectsmodel
Non-linear Relaxesthelinear-symmetryassumption 170
conditionalitytest ofinteractioneffectsmodel
© in this web service Cambridge University Press www.cambridge.org
