Applied Panel Data Analysis for Economic and Social Surveys

336 Pages·2013·2.886 MB·English

by  Hans-Jürgen Andreß, Katrin Golsch, Alexander W. Schmidt (auth.)

Applied Panel Data Analysis for Economic
and Social Surveys
Hans-Jürgen Andreß (cid:2) Katrin Golsch (cid:2)
Alexander W. Schmidt
Applied Panel
Data Analysis
for Economic
and Social Surveys
Hans-JürgenAndreß AlexanderW.Schmidt
LehrstuhlfürEmpirischeSozial- LehrstuhlfürEmpirischeSozial-
undWirtschaftsforschung undWirtschaftsforschung
UniversitätKöln UniversitätKöln
Cologne,Germany Cologne,Germany
KatrinGolsch
FakultätfürSoziologie
UniversitätBielefeld
Bielefeld,Germany
AllnecessarydatasetsandStatasyntaxfilesonhttp://eswf.uni-koeln.de/panel
ISBN978-3-642-32913-5 ISBN978-3-642-32914-2(eBook)
DOI10.1007/978-3-642-32914-2
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2012953902
©Springer-VerlagBerlinHeidelberg2013
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpub-
lication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforany
errorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespect
tothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 BenefitsandChallengesofthePanelDesign . . . . . . . . . . . . 2
1.1.1 Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.2 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 OutlineoftheBook . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3 AudienceandPrerequisites . . . . . . . . . . . . . . . . . . . . . 12
2 ManagingPanelData . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.1 TheNatureofPanelData . . . . . . . . . . . . . . . . . . . . . . 16
2.2 TheBasicsofPanelDataManagement . . . . . . . . . . . . . . . 19
2.2.1 MergingandAppendingData . . . . . . . . . . . . . . . . 20
2.2.2 BasicAppendandMergeCommands . . . . . . . . . . . . 23
2.2.3 BuildingaWorkingDataSetwithAppendandMerge
Commands . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.2.4 WideandLongFormat . . . . . . . . . . . . . . . . . . . 27
2.2.5 SomeGeneralRemarks . . . . . . . . . . . . . . . . . . . 30
2.3 ThreeCaseStudiesonPovertyinGermany . . . . . . . . . . . . . 31
2.3.1 CaseStudy1:HowManyGermanCitizensWerePoor
in2004?ACross-SectionalAnalysis . . . . . . . . . . . . 33
2.3.2 CaseStudy2:DidPovertyIncreaseinGermanyAfter
2004?AnAnalysisofPooledCross-Sections . . . . . . . . 40
2.3.3 CaseStudy3:HowLargeIstheRiskofBecomingPoor
inGermany?APanelAnalysis . . . . . . . . . . . . . . . 42
2.4 HowtoRepresentaPopulationwithPanelData? . . . . . . . . . . 49
2.4.1 WeightedandUnweightedAnalysisofCross-Sections . . . 50
2.4.2 WeightinginBalancedandUnbalancedPanels . . . . . . . 52
2.4.3 WhentoUseWeights?. . . . . . . . . . . . . . . . . . . . 55
2.5 ConclusionandFurtherReading . . . . . . . . . . . . . . . . . . 58
3 DescribingandModelingPanelData . . . . . . . . . . . . . . . . . . 61
3.1 SomeBasicTerminology . . . . . . . . . . . . . . . . . . . . . . 62
3.2 MeasurementsoverTimeAreNotIndependent . . . . . . . . . . . 65
3.3 DescribingtheDependentVariable . . . . . . . . . . . . . . . . . 70
3.4 Explaining the Dependent Variable over Time: Typical
ExplanatoryVariables . . . . . . . . . . . . . . . . . . . . . . . . 79
v
vi Contents
3.4.1 Time-ConstantandTime-VaryingVariables . . . . . . . . 79
3.4.2 SeriallyCorrelatedObservations . . . . . . . . . . . . . . 83
3.5 ModelingPanelData. . . . . . . . . . . . . . . . . . . . . . . . . 86
3.5.1 ModelingtheLeveloftheDependentVariable . . . . . . . 87
3.5.2 ModelingChangeoftheDependentVariable . . . . . . . . 89
3.5.3 AdditionalModels . . . . . . . . . . . . . . . . . . . . . . 92
3.6 EstimatingModelsforPanelData . . . . . . . . . . . . . . . . . . 98
3.6.1 OmittedVariableBias(UnobservedHeterogeneity) . . . . 99
3.6.2 SeriallyCorrelatedandHeteroscedasticErrors . . . . . . . 107
3.6.3 MeasurementErrorBias . . . . . . . . . . . . . . . . . . . 109
3.6.4 AFormalSummaryoftheMainEstimationAssumptions . 114
3.7 OverviewofSubsequentChapters . . . . . . . . . . . . . . . . . . 116
4 PanelAnalysisofContinuousVariables . . . . . . . . . . . . . . . . 119
4.1 ModelingtheLevelofY . . . . . . . . . . . . . . . . . . . . . . . 120
4.1.1 IgnoringthePanelStructure . . . . . . . . . . . . . . . . . 121
4.1.2 ModelingthePanelStructure . . . . . . . . . . . . . . . . 126
4.1.3 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . 174
4.2 ModelingtheChangeofY . . . . . . . . . . . . . . . . . . . . . . 180
4.2.1 AnalysisofChangeUsingChangeScores . . . . . . . . . 180
4.2.2 AnalysisofChangeUsingImpactFunctions . . . . . . . . 185
4.2.3 AnalysisofTrends . . . . . . . . . . . . . . . . . . . . . . 192
4.3 ConclusionandFurtherReading . . . . . . . . . . . . . . . . . . 201
5 PanelAnalysisofCategoricalVariables . . . . . . . . . . . . . . . . 203
5.1 ModelingtheLevelofY:DiscreteResponseModels
forPanelData . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
5.1.1 IgnoringthePanelStructure . . . . . . . . . . . . . . . . . 206
5.1.2 ModelingthePanelStructure . . . . . . . . . . . . . . . . 227
5.1.3 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . 247
5.2 ModelingtheChangeofY:Discrete-TimeEventHistoryModels
forPanelData . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
5.2.1 BasicTerminology . . . . . . . . . . . . . . . . . . . . . . 251
5.2.2 HowtoEstimateaDiscrete-TimeHazardModel . . . . . . 254
5.2.3 ApplyingtheDiscrete-TimeEventHistoryModel . . . . . 262
5.2.4 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . 281
5.3 ConclusionandFurtherReading . . . . . . . . . . . . . . . . . . 284
6 HowtoDoYourOwnPanelAnalysis . . . . . . . . . . . . . . . . . . 287
7 UsefulBackgroundInformation . . . . . . . . . . . . . . . . . . . . . 293
7.1 FunctionsofRandomVariables . . . . . . . . . . . . . . . . . . . 293
7.2 EstimationandTesting. . . . . . . . . . . . . . . . . . . . . . . . 295
7.2.1 OrdinaryLeastSquares . . . . . . . . . . . . . . . . . . . 295
7.2.2 MaximumLikelihood . . . . . . . . . . . . . . . . . . . . 307
7.3 WebSiteoftheTextbook . . . . . . . . . . . . . . . . . . . . . . 312
Contents vii
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
AuthorIndex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
List of Figures
Fig.1.1 Successful interviews with persons and households (SOEP:
SamplesAandB) . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Fig.2.1 Appendingdata . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Fig.2.2 Mergingdatawithonekeyvariable . . . . . . . . . . . . . . . . 21
Fig.2.3 Mergingdatawithtwokeyvariables . . . . . . . . . . . . . . . . 22
Fig.2.4 Many-to-onemerge . . . . . . . . . . . . . . . . . . . . . . . . . 22
Fig.2.5 UnweightedGermanpovertyrates(2004–2006) . . . . . . . . . . 42
Fig.2.6 WeightedandunweightedGermanpovertyrates(2004–2006) . . 52
Fig.3.1 Survivalprobabilitybyincomelevel . . . . . . . . . . . . . . . . 73
Fig.3.2 Incometrajectoriesof19individualswithveryhighandverylow
wages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Fig.3.3 Explainingpaneldata . . . . . . . . . . . . . . . . . . . . . . . . 80
Fig.3.4 Impactfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Fig.3.5 Loghourlywagesbylaborforceexperienceandeducation . . . . 101
Fig.3.6 Pathdiagramofhourlywages . . . . . . . . . . . . . . . . . . . 112
Fig.4.1 Residuals(connectedbylines)foreachoftenindividuals . . . . . 125
Fig.4.2 GovernmentspendinginIrelandandSweden(1961–1993) . . . . 137
Fig.4.3 Explainingbetweenandwithinvariance(uncorrelatedu ). . . . . 161
i
Fig.4.4 Explainingbetweenandwithinvariance(correlatedu ) . . . . . . 161
i
Fig.4.5 Demeaningparameterθ withrespecttonumber
ofmeasurementsT . . . . . . . . . . . . . . . . . . . . . . . . . 172
Fig.4.6 PathdiagramofaRE(left)andaFEmodel(rightpanel) . . . . . 179
Fig.4.7 Meanlifesatisfactionbeforeandafterseparation . . . . . . . . . 186
Fig.5.1 ScatterplotsofacontinuousanddichotomousY . . . . . . . . . 208
Fig.5.2 Observedresponsesandpredictedprobabilityofsecondaryjob
holding(pooledLPM) . . . . . . . . . . . . . . . . . . . . . . . 211
Fig.5.3 Thelogisticregressionfunction . . . . . . . . . . . . . . . . . . 215
Fig.5.4 Conditionaleffectsplotusingtheestimatesofthepooledlogistic
model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
Fig.5.5 Linearprobability,logisticandprobitregressionmodel . . . . . . 226
Fig.5.6 Differenttypesofcensoring . . . . . . . . . . . . . . . . . . . . 252
Fig.5.7 Predicted hazard rate of female retirement (using estimates
ofmodel1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
Fig.5.8 True and observed hazard rate of dying for the control
andtreatmentgroup . . . . . . . . . . . . . . . . . . . . . . . . . 268
ix
x ListofFigures
Fig.5.9 Observedhazardrateof dyingforthetreatmentandcontrol
group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
Fig.5.10 Predictedhazardofretiring . . . . . . . . . . . . . . . . . . . . . 273
Fig.7.1 ExamplewithtwocontinuousvariablesXandY . . . . . . . . . 295
Fig.7.2 Empiricalsamplingdistributionofβ . . . . . . . . . . . . . . . 299
1
Fig.7.3 Howtochoosebetweendifferentestimators . . . . . . . . . . . . 300
Fig.7.4 Drawing inferences from a sample about an unknown
population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
Fig.7.5 Loglikelihoodfunction(β varied,β fixedatβ =−1.4606) . . 309
1 0 0
List of Tables
Table1.1 Familysize-adjustedincometransitiontablesforUSAmerican
familieswithchildren(using40,50and60percentofmedian
income) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Table2.1 Organizingpaneldatainadataset . . . . . . . . . . . . . . . . 17
Table2.2 Pooledcross-sections . . . . . . . . . . . . . . . . . . . . . . . 41
Table2.3 Responsepatternsinunbalancedpaneldata . . . . . . . . . . . 44
Table2.4 PovertydynamicsinGermany(2004–2006,unweighteddata) . . 48
Table2.5 PovertydynamicsinGermany(2004–2006,weighteddata) . . . 55
Table3.1 Loghourlywages1980–1987 . . . . . . . . . . . . . . . . . . . 66
Table3.2 Unionmembership1980–1987(frequenciesandpercentages) . . 68
Table3.3 Sequencesofunionmembershipstatus1980–1983 . . . . . . . 71
Table3.4 Survivalprobabilityofunionmembersfrom1980 . . . . . . . . 72
Table3.5 Loghourlywagesforselectedindividuals1980–1987 . . . . . . 75
Table3.6 Descriptivestatisticsofselectedvariables . . . . . . . . . . . . 81
Table3.7 Loghourlywages1980–1987(demeaneddata) . . . . . . . . . 84
Table3.8 Threedifferentprototypicalcorrelationstructures . . . . . . . . 85
Table3.9 Unionmembershipstatus1985bymembershipstatus1984 . . . 105
Table3.10 Unionmembershipstatus1981bymembershipstatus1980 . . . 111
Table4.1 Determinantsofloghourlywages:PooledOLS(1980–1987) . . 122
Table4.2 SerialcorrelationsinpooledOLSmodelandinrawdata . . . . 124
Table4.3 Determinantsofpublicspending:PooledOLS(1961–1993,
15OECDcountries) . . . . . . . . . . . . . . . . . . . . . . . . 130
Table4.4 Determinants of public spending: LSDV (1961–1993,
15OECDcountries) . . . . . . . . . . . . . . . . . . . . . . . . 132
Table4.5 Determinantsofpublicspending:FE(1961–1993,15OECD
countries) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Table4.6 Determinants of log hourly wages: Pooled OLS and FE
(1980–1987) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Table4.7 CorrelationoftheresidualsfromtheFEmodel . . . . . . . . . . 145
Table4.8 Determinants of psychological distress: Pooled OLS, FE,
andRE(1980–1992) . . . . . . . . . . . . . . . . . . . . . . . 150
Table4.9 CorrelationoftheresidualsfromthepooledOLSmodel . . . . . 151
Table4.10 CorrelationoftheresidualsfromtheREmodel . . . . . . . . . 156
Table4.11 Determinantsofpsychologicaldistress:FE,RE,pooledOLS,
andBE(1980–1992) . . . . . . . . . . . . . . . . . . . . . . . 159
xi
xii ListofTables
Table4.12 Determinants of psychological distress: Hybrid models
(1980–1992) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
Table4.13 Change in overall life satisfaction due to separation
(FDestimation) . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Table4.14 Lifesatisfactionbeforeandafterseparation(RE,FE,andFD
estimation) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
Table4.15 Psychologicaldistressbeforeandafterdivorce . . . . . . . . . . 192
Table4.16 Determinantsofpost-materialism(1984–1996)(FGLSandML
estimates) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
Table5.1 Measuresusedintheanalysisoftheheineck-schwarze
data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Table5.2 Transitionsbetweenpanelwaves . . . . . . . . . . . . . . . . . 209
Table5.3 Determinantsofsecondaryjobholding(pooledLPM) . . . . . . 210
Table5.4 CorrelationofresidualsfromthepooledLPM . . . . . . . . . . 212
Table5.5 Determinants of secondary job holding (pooled LPM
withrobuststandarderrors) . . . . . . . . . . . . . . . . . . . . 213
Table5.6 Determinantsofsecondaryjobholding(pooledlogisticmodel) . 221
Table5.7 Determinantsofsecondaryjobholding(oddsratiosforpooled
logisticmodel). . . . . . . . . . . . . . . . . . . . . . . . . . . 222
Table5.8 Determinants of secondary job holding (pooled linear
probability, logistic, and probit model with cluster robust
standarderrors) . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Table5.9 Determinantsofsecondaryjobholding(pooledandFElogistic
model) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Table5.10 Determinantsofsecondaryjobholding(RElogisticandRE
probitmodel) . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Table5.11 Determinantsofsecondaryjobholding(RElogistichybrid
model) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
Table5.12 Measuresusedintheanalysisofthehankdata . . . . . . . . . 250
Table5.13 Hypothetical data set for estimating discrete-time event
historydata . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
Table5.14 Determinants of female retirement (discrete-time logistic
hazardmodel) . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Table5.15 Determinantsoffemaleretirement(resultsofcompetingmodel
specifications) . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
Table5.16 Truehazardrates . . . . . . . . . . . . . . . . . . . . . . . . . 268
Table5.17 Determinantsoffemaleretirement(logisticandRElogistic
hazardmodel) . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
Table5.18 Women’sretirementbyhomeownership(inpercent) . . . . . . 276
Table5.19 Homeownershipbywomen’sretirement(inpercent) . . . . . . 277
Table5.20 Determinantsoffemaleretirement(complementarylog–log
hazardmodel) . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
Table6.1 Characteristicsofselectedpanelstudies . . . . . . . . . . . . . 288
Table7.1 Expectedvaluesandvariancesforfunctionsofrandomvariables 294
Table7.2 ExamplewithtwocontinuousvariablesXandY . . . . . . . . . 295