Much Ado About Nothing: A Comparison of Missing Data Methods and

12 Pages·2006·0.43 MB·English

Much Ado About Nothing: A Comparison of Missing Data Methods
and Software to Fit Incomplete Data Regression Models
NicholasJ.HortonandKenP.Kleinman
cellent textbooks by Little and Rubin (2002), Schafer (1997),
andAllison(2002)provideacomprehensiveoverviewofmeth-
Missing data are a recurring problem that can cause bias or ods in this setting, focused primarily on multiple imputation.
leadtoinefficientanalyses.Statisticalmethodstoaddressmiss- Although somewhat dated, Little (1992) describes a hierarchy
ingnesshavebeenactivelypursuedinrecentyears,includingim- of approaches to account for missing predictors, including the
putation,likelihood,andweightingapproaches.Eachapproach maximumlikelihoodapproachofIbrahim(1990).Publications
is more complicated when there are many patterns of missing byMeng(2000)andRaghunathan(2004)provideageneralintro-
values, or when both categorical and continuous random vari- duction,andthepaperbyIbrahimetal.(2005)reviewsrecentde-
ables are involved. Implementations of routines to incorporate velopmentsinacomprehensivefashion,thoughtheirapplication
observationswithincompletevariablesinregressionmodelsare (cancerdataset)featuresincompletenessononlyonevariable.A
now widely available. We review these routines in the context usefulonlineannotatedbibliographyprovidesacomprehensive
of a motivating example from a large health services research readinglist(Carpenter2006a).
dataset. While there are still limitations to the current imple- Inthisarticle,weupdatethepriorreviewofHortonandLipsitz
mentations, and additional efforts are required of the analyst, (2001)andapplymethodsdescribedbyIbrahimetal.(2005)to
itisfeasibletoincorporatepartiallyobservedvalues,andthese thelogisticregressionanalysisofadatasetwithincompleteness
methodsshouldbeusedinpractice. onfourvariables(bothcategoricalandcontinuous)usingavari-
etyofsoftwarepackages.Wediscussmodelingassumptions,ap-
KEYWORDS: ConditionalGaussian;Healthservicesresearch; proaches,andcompromisesrequiredforestimationwithincur-
Maximumlikelihood;Multipleimputation;Psychiatricepidemi- rentimplementations.InSection2,webrieflyreviewmethods
ology. forincorporatingincompleteobservationsinregressionmodels
then summarize findings of two surveys of how missing data
methodsareusedinpracticeinSection3.Wedescribeourmoti-
vatingexample(whichfeaturesalargedatasetwithhighpropor-
1. INTRODUCTION tionofmissingvalueswithanonmonotonepattern)inSection4,
detailsupportformissingdatasoftwareinSection5,thenapply
Missing data are a frequent complication of any real-world thesemethodstothemotivatingdatasetinSection6.Wecontrast
study.Thecausesofmissingnessareoftennumerous,somedue thestrengthsandlimitationsofthesepackagesinpractice,and
todesign,andsometochance.Somevariablesmaynotbecol- suggestimprovementsforthefuture.
lected from all subjects, some subjects may decline to provide We focus on methods to incorporate partially observed pre-
values,andsomeinformationmaybepurposelyexcised,forex- dictors;differentissuesarisewhensomeoutcomesarenotfully
ampletoprotectconfidentiality.Whiletheuseofcompletecase observed.Wealsodonotconsiderlongitudinalorclusteredout-
methods that drop subjects missing any observations are com- comes, for which other complications arise (e.g. Laird 1988;
monly seen in practice, this approach has the disadvantage of Robins,Rotnitzky,andZhao1995;andJansenetal.2006).
beinginefficientaswellaspotentiallybiased.
Thedevelopmentofmethodsforanalysisofdatawithincom- 2. INCOMPLETEDATAREGRESSIONMETHODS
plete values has been an active area of research. Models that
2.1 NotationandNomenclature
incorporatepartiallyobservedpredictorsareofparticularinter-
estinmanyreal-worldsettings,sincemissingnessofjustafew Webeginbyintroducingnotationthatwillbeusedthrough-
percent on each of a number of covariates may lead to a large out,assumingthatdataarecollectedonasampleofnsubjects
numberofobservationswithsomemissinginformation.Theex- andthatprimaryinterestrelatestotheparametersgoverningthe
conditionaldistributionf(Yi|Xi,βββ).Tosimplifyexposition,we
suppressthesubjectindicator.Foragivensubjectwecanparti-
NicholasJ.HortonisAssistantProfessor,DepartmentofMathematicsandStatis-
tics,ClarkScienceCenter,SmithCollege,Northampton,MA01063-0001(E- tionXintocomponentsdenotingobservedvariables(Xobs)and
mail:[email protected]).KenP.KleinmanisAssociateProfessor,De- those that are missing for that subject (Xmis). We denote by R
partment of Ambulatory Care and Prevention, Harvard Medical School and a set of response indicators (i.e., Rj = 1 if the jth element of
HarvardPilgrimHealthCare,Boston,MA.Helpfulcommentsandassistance
regardingdetailsofthesoftwareimplementationswereprovidedbyFrankHar- X is observed, and equals 0 otherwise), governed by parame-
rell,Jr.,KristaKilmer,GaryKing,PatrickRoyston,PralaySenchaudhuri,Stef ters φφφ. Little and Rubin (2002) introduced a nomenclature for
vanBuuren,andYangYuan.WethankSuzanneSwitzerforassistancewiththe missingnessintermsofprobabilitymodelsforR.Themissing
reviewofmissingdatamethodsintheNewEnglandJournalofMedicineand
completelyatrandom(MCAR)assumptionisdefinedas
JamesCarpenter,AmyHerring,aswellasOwenThomasforusefulcomments.
WearegratefulforthesupportprovidedbyNIMHgrantR01-MH54693andthe
SmithCollegePickerProgram. P(R|Y,X)=P(R|Y,Xobs,Xmis)=P(R|φφφ),
(cid:3)c2007 AmericanStatisticalAssociation DOI:10.1198/000313007X172556 TheAmericanStatistician,February2007,Vol.61,No.1 79
where in addition φφφ and βββ are presumed distinct. Note that Hypothetical Hypothetical
thisdependsonanassumedrelationshipbetweenRandtheun- Monotone Non-monotone
observed and unknowable Xmis. Heuristically, this assumption Pattern Y X1 X2 X3 Y X1 X2 X3
— — — — — — — — —
states that missingness is not related to any factor, known or
1 Obs Obs Obs Obs Obs Obs Obs Obs
unknown,inthestudy. 2 Obs Obs Obs M Obs Obs Obs M
Itmaybemoreplausibletopositthatmissingnessismissing 3 Obs Obs M M Obs Obs M M
4 Obs M M M Obs Obs M Obs
atrandom(MAR),whichassumesthat
5 Obs M Obs Obs
P(R|Y,X)=P(R|Y,Xobs,φφφ). 6 Obs M M Obs
This assumption states that the missingness depends only on Figure 1. Monotoneandnonmonotonepatternsofmissingness(Obs
=observed,M=missing)
observedquantities,whichmayincludeoutcomesandpredictors
(in which case the missingness is sometimes labeled covariate
dependentmissingness(CDM)).Theassumptionregardingthe theCCestimatoristhatiftherearemanydifferentvariableswith
lack of association with unobserved quantities (Xmis) remains. missingvalues,thenalargefractionoftheobservationsmaybe
We note that at first glance, the meaning of the term “missing dropped. For the example datasets in Figure 1, only data from
atrandom”canbeconfusing,sincemissingnesscanactuallybe pattern1isincluded,thoughpartialinformationisavailablefrom
predicted(butisrandomaftercontrollingformissingnessdueto theotherpatterns(e.g.,thejointdistributionoff(Y,X ,X )can
1 2
observedquantities). beestimatedfrompattern2).Theefficiencylossesofthecom-
Researchershavenotedthatbyincludingarelativelyrichset pletecaseestimatorcanbesubstantial(LittleandRubin2002,
ofpredictorsinthemodel,theMARassumptionmaybemade p.42).
moreplausible(Collins,Schafer,andKam2001).Othershave
noted that incorporation of information regarding the outcome
2.3 “Ad-hoc”Methods
improvesestimationofmissingpredictors(Moonsetal.2006).
ItispossibletoformallytesttheMCARassumptionagainst Aseriesof“ad-hoc”methodshavebeensuggestedtoaddress
thealternatehypothesisofMAR(Little1988;Diggleetal.2002). missing data. One approach for continuous variables involves
Finally,ifthemissingnesslawP(R|Y,X)cannotbesimplified recoding missing values to some common value, creation of
(i.e.,itdependsonunobservedquantities),theprocessistermed an indicator of missingness as a new variable, and including
nonignorable (abbreviated NINR [nonignorable nonresponse] boththesevariablesalongwiththeirinteractionintheregression
orMNAR[missingnotatrandom]).InaNINRsetting,thecor- model.Asimilarapproachforcategoricalvariablesinvolvesthe
rectspecificationofthemissingnesslawmustbegiventoyield creationofanadditionalcategoryformissingvalues.Thesead-
consistent estimates of the regression parameters. Researchers hoc approaches have the potential to induce bias and are not
have addressed issues in the NINR setting (Diggle and Ken- recommended(Jones1996;GreenlandandFinkle1995).
ward1994;Little1994).NINRmodelsareparticularlyusefulin Asecondapproachinvolvesdroppingvariablesfromtheanal-
assessingthesensitivityofresultstodeviationsfromMARmiss- ysisthathavealargeproportionofmissingvalues.Thisisnot
ingness(Carpenter,Kenward,andVansteelandt2006).Without attractive,sinceitmayleadtotheexclusionofimportantfactors
additionalinformation,itisinherentlyimpossibletotestwhether intheregressionmodel,withconsequentbiasorunnecessarily
MARholds(LittleandRubin1987). largestandarderrors.
Another important concept regarding missing data, particu- Two other approaches involve imputation of missing values
larly where there are multiple variables with missing values, using mean imputation (average of the observed values) or for
relatestothepatternofmissingdata.Ifthedatamatrixcanbe longitudinalstudies,thelastobservedvalue(alsoknownaslast
rearrangedinsuchawaythatthereisahierarchyofmissingness, valuecarriedforward,LVCForlastobservationcarriedforward,
sothatobservingaparticularvariableXb forasubjectimplies LOCF).Bothmethodshavethepotentialofinducingbiasaswell
thatXaisobserved,fora <b,thenthemissingnessissaidtobe asunderstatingvariabilityandarenotrecommended(Carpenter
monotone.Figure1displaystwohypotheticaldatasets.Theleft etal.2004;Cook,Zeng,andYi2004;Jansenetal.2006).
handdatasethasbeenrearrangedtocreateamonotone(“stair-
step”)pattern,thoughthisisnotpossibleintherighthandsideof 2.4 MultipleImputation
the figure. When missingness is nonmonotone, models for the
missingness of one variable may include covariates which are Multipleimputationisathree-stepapproachtoestimationof
alsomissingvalues.Simplermethodscanbeusedifthepattern incomplete data regression models due to Rubin (1976). First,
ismonotone,thoughamonotonepatternisuncommoninmost plausiblevaluesformissingobservationsarecreatedthatreflect
realisticsettings(includingourmotivatingexample). uncertaintyaboutthenonresponsemodel.Thesevaluesareused
to “fill-in” or impute the missing values (using an assumption
ofMAR).Thisprocessisrepeated,resultinginthecreationofa
2.2 CompleteCaseMethod
numberof“completed”datasets.Second,eachofthesedatasets
The simplest method for the analysis of incomplete data re- isanalyzedusingcomplete-datamethods.Finally,theresultsare
gressionmodelsinvolvestheanalysisofthesetofobservations combined,whichallowstheuncertaintyregardingtheimputation
withnomissingvalues.WhenmissingnessisMCAR,thenthe tobetakenintoaccount.Typically5–10imputationsarecreated,
completecase(CC)estimatorisunbiased.Themaindrawbackof thoughmorearecomputationallyfeasibleandbettercharacterize
80 StatisticalSoftwareReviews
thevariabilityintroducedintotheresultsduetotheimputation algorithm,animputationisgeneratedforthemissingvariable,
process. thenthisimputedvalueisusedintheimputationofthenextvari-
The method of multiple imputation was first proposed in a able.Thisprocessrepeats,imputingmissingvaluesusingaGibbs
publicusesurveydatasetting.Multipleimputationremainside- samplingprocedureuntiltheprocessreachesconvergence.Sep-
ally suited to this setting, since the creators of the dataset can aratechainsareusedtogeneratethemultipleimputations.
use auxiliary confidential and detailed information that would Forcontinuousvariables,themodelmayinvolvealinearre-
beinappropriatetoincludeinthepublicdataset(Rubin1996). gressionmodelorusepredictivemeanmatching(wheretheim-
Given the completed datasets, users may utilize existing soft- putedvariablestakesonthevalueofoneofasetofnearestob-
waretoanalyseeachofthedatasets.Giventheresultsforeach servedvalueinthedataset).Fordichotomousvariables,logistic
analysis,anoverallsummaryisstraightforwardtocalculate. regression can be fit, while polytomous models are needed for
Multiple imputation is, however, more commonly used in a categoricalvariables.Implementationsofthechainedequation
settingwheretheimputerandtheanalystarethesameperson. approachareavailableintheMICElibrary(forRandS-Plus),
Anextensiveliteratureonthetopicexists(Rubin1987;Glynn ICE(forStata),IVEware(forSASorstandalone)oraregImpute
etal.1993;Rubin1996;Schafer1999;BarnardandMeng1999; (forRandS-Plus).
vanBuurenetal.1999;Allison2000;KenwardandCarpenterin One problem with the chained equation approach is that it
press)asdoimplementationsingeneralpurposestatisticalsoft- maynotconvergetoasensiblestationarydistributionifthesep-
ware.Ausefulguidetosoftwareimplementationscanbefound aratemodelsarenotcompatiblewithamultivariatedistribution
atthemultiple-imputation.comWebsite(vanBuuren2006). (Raghunathanetal.2001),thoughvanBuurenetal.(2006)show
Thekeyissuefortheanalystistheappropriatespecification inaseriesofsimulationstudiesthatreasonableimputationswere
oftheimputationmodel,sinceifthisismisspecified,thereisthe obtainedevenwhentheseparatemodelswereincompatible.Ad-
potential for bias. Often a multivariate normal model has been ditional work is needed to further establish the validity of the
used,asitiscomputationallytractable(sinceonlythemeanvec- approach.
torandvariance-covariancematrixneedstobeestimated).This 2.4.3 MethodsforMonotoneDatasets
model has been used even when some of the variables are not
AnumberofapproachesareimplementedinSASPROCMI
Gaussian,thoughthiscomplicatesanalysesandifimputedval-
for datasets with monotone missing structure. The predictive
ues are rounded, can lead to bias (Horton, Lipsitz, and Parzen
meanmatchingmethodcanbeusedtoimputeavaluerandomly
2003; Allison 2005; Bernaards, Belin, and Schafer in press).
fromasetofobservedvalueswhosepredictedvaluesareclosest
These issues are particularly salient when multiple categorical
to the predicted value from a specified regression model. This
andcontinuousvariableshavemissingvalues,asthejointdistri-
processisstraightforwardwhenimputingacontinuousrandom
butionmaybecomplicated.Finally,notethattheanalysisshould
variable,butmorecomplicatedwhenimputingacategoricalvari-
notusearichermodelthanthatusedforimputation(Littleand
able with more than two levels. The process begins with the
Rubin2002).
observations with only one missing value, and then uses those
Wewillnowreviewanumberofmethodsthathavebeenpro-
values in the imputation of the observations with two missing
posedforimputationmodelsforcategoricalandcontinuousvari-
values,andsoon.AnalystsarewarnedthatAllison(2000)found
ables.
thatpredictivemeanmatchingapproachesledtobiasedresults
whenappliedtomissingpredictormodels.
2.4.1 ConditionalGaussian
Similarly,regressionorpropensityscoremodelscanbeused
One approach to imputation when there are both continu-
toimputemissingvalues.
ous and discrete missing values is the Conditional Gaussian
One disadvantage of these approaches is that for datasets
approach, popularized by Schafer (1997). A log-linear model
with nonmonotone missingness, some observations need to be
(Bishop, Fienberg, and Holland 1975) is specified for the dis-
droppedfromtheanalysis,orsome“ad-hoc”procedureused.As
crete random variables, and conditional on this distribution, a
anexample,considerthehypotheticalnonmonotonepatternsin
multivariate normal distribution is assumed for the continuous
Figure1(p.80).Itispossibletocreateamonotonedatasetby
variables.Thisgenerallocationmodel(OlkinandTate1961)can
onlyincludingpatterns(1,2,3),(1,3,6),(1,3,4),or(1,5,6).In
befitasasaturatedmultinomialwithseparatemeansandshared
practice,however,inadditiontobeinghighlyarbitrary,creation
covariance,butthismayleadtoaproliferationofparametersin
of a dataset with monotone missingness may exclude a large
real-worldapplicationswithmultiplecategoricalvariables.Asa
numberofobservationsandbeinefficientaswellaspotentially
result,simplificationofthelog-linearmodelisrequiredinprac-
biased.
tice.ThisapproachhasbeenimplementedintheMIXprograms
Theanalystmaybeabletocreativelyexploitpatternsofmiss-
ofSchafer(assumingaformofmonotonicity)aswellasS-Plus
ingvaluesinaparticulardataset.Consider,forexample,using
missingdatalibrary. patterns (1, 2, 3) to estimate f(X ,X |Y,X ), and using pat-
2 3 1
terns(1,4)toestimatef(X |Y,X ,X ).Wepursuethistypeof
2.4.2 ChainedEquations 2 1 3
strategytoavoiddroppingobservationsinourmotivatingexam-
An alternative approach involves a variable-by-variable ap-
ple.
proachusingchainedequations(vanBuurenetal.1999;Raghu-
nathanetal.2001;vanBuurenetal.2006;vanBuureninpress). 2.4.4 OtherIssuesWithImputation
Theimputationmodelisspecifiedseparatelyforeachvariable, Therearemanyadditionalimputationissuesthatarebeyond
involvingtheothervariablesaspredictors.Ateachstageofthe thescopeofthisarticle.Forexample,theremaybeboundson
TheAmericanStatistician,February2007,Vol.61,No.1 81
onlyafewcategoricalvariablesasaturatedmultinomialdistri-
butioncanbefit.Whentherearemorevariables,somesimpli-
fication of the joint distribution is often necessary. Lipsitz and
Ibrahim(1996)suggestedaconditionalapproachwhere
f(X ,X ,X ,...,X )=f(X )f(X |X )f(X |X ,X )
1 2 3 p 1 2 1 3 1 2
...f(X |X ,X ,...,X ),
p 1 2 p−1
whereeachofthemarginalmodelstypicallyinvolveonlymain
effects.Furthercomplicationsarisewithcontinuouscovariates,
since some form of MCEM (Monte Carlo EM) is required
(Ibrahim,Chen,andLipsitz1999).
Anothercomplicationformaximumlikelihoodrelatestothe
calculationofthestandarderrorsofestimates.Implementations
of maximum likelihood that address these complications are
availableinLogXactversion7,theS-Plusmissingdatalibrary
andinSPSS(vonHippel2004).
2.6 WeightingMethods
Figure2. Useoflikelihood-basedapproachwithEMalgorithmtoincor- Anotherapproachtoaccountingformissingpredictordatais
poratepartiallyobserveddata.
theuseofweightingmethods(Robinsetal.1995;XieandPaik
1997; Horton and Lipsitz 1999; Horton et al. 2001; Carpenter
etal.2006).Inthisapproach,amodelfortheprobabilityofmiss-
imputedvalueswhichneedtoaccountedfor(e.g.,asonlyzero
ingnessisfit,andtheinverseoftheseprobabilitiesareusedas
valuesforyearssmokingareplausibleforanonsmoker)(Raghu-
weights for the complete cases. Weighting approaches can be
nathanetal.2001).Similarissuesarisewhenmissingvaluesare
fitinsoftwarethatallowsforweights(e.g.,Stata,SUDAAN,or
knowntobewithinacertainrange(e.g.,betweenthreeandfour
SAS).Thisapproachisdetailedforasinglemissingpredictorin
on a five-point Likert scale), or when variables require trans-
Ibrahim et al. (2005) and Carpenter et al. (2006), but becomes
formations.Whileimportantissues,wedonotfurtherconsider
considerablylesstractablewithmultiplemissingvariables,par-
them.
ticularlywhentheyarenonmonotone.Duetothislimitation,and
thefactthatourmotivatingexampleisdecidedlynonmonotone,
2.5 Likelihood-BasedApproaches wedonotfurtherpursueestimationusingweightingmethods.
Maximum likelihood is an alternative approach which also
2.7 BayesianApproaches
assumes that missingness is MAR. Typically, primary interest
relates to the regression parameters governing the conditional While multiple imputation was derived from within a
distribution:f(Y|X,βββ).Whensomeofthepredictorsaremiss- Bayesian framework (sampling from the posterior distribution
ing, however, Ibrahim (1990) suggested that information can ofinterest),Bayesianapproacheshavebeenappliedmoregen-
be reclaimed by estimating the distribution of the covariates: erally. Ibrahim et al. (2005) described estimation with a prior
f(X|γγγ).Thejointdistributionf(X,Y|βββ,γγγ)ismaximized,typ- distributiononthecovariates,andthecloserelationshipbetween
icallythroughuseoftheEM(expectation-maximization)algo- theBayesianapproachandMLandMImethods.Inpartbecause
rithm(Dempster,Laird,andRubin1977).Foreachobservation thesemethodsaresoflexible,specificcodingofpriordistribu-
withmissingdata,multipleentriesarecreatedinanaugmented tionsandmodelrelationshipswithapackagesuchasWinBugs
dataset for each possible value of the missing covariates, and isrequiredforestimation.Suchcodingisrelativelystraightfor-
aprobabilityofobservingthatvalueisestimatedgiventheob- ward,andexamplesofmissingdatamodelsareavailable(Car-
serveddataandcurrentparameterestimates(E-step).Theaug- penter,Pocock,andLamm2002;Carpenter2006b),howeverwe
mentedcomplete-datadatasetcanthenbeusedtofittheregres- donotfurtherdiscusstheseapproaches.
sion model, accounting for these weights. Figure 2 displays a
hypothetical dataset with a completely observed outcome (Y) 3. SURVEYSOFMISSINGDATAMETHODSUSEIN
and three dichotomous covariates that are sometimes missing PRACTICE
(X ,X ,andX ).Observation3ismissingX ,sotwoentries
1 2 3 3
are created in the augmented dataset, with w31 + w32 = 1, Despite the existence of principled methods for the analysis
w31 =P(X3 =0|Y =1,X1 =1,X2 =0,βββ,γγγ).Forobserva- of incomplete data regression models, there is some evidence
tion7,thereare8entriesintheaugmenteddataset.Hortonand that their use in applied settings remains limited. We base this
Laird(1999)reviewedthismethodologyindetail,whileIbrahim statement on two recent studies of statistical methods used in
etal.(2005)comparedandcontrasteditwithotherapproaches. medicalresearch.
Oneofthecomplicationsofthismethodistheneedtomodel Burton and Altman (2004) reviewed the reporting of miss-
thenuisancedistributionofthecovariates.Insomesettingswith ingcovariatedatain100cancerprognosticstudiespublishedin
82 StatisticalSoftwareReviews
1. quantificationofcompletenessofcovariatedata
(a) ifavailabilityofdataisanexclusioncriterion,specifythenumberofcasesexcludedforthisreason,
(b) providethetotalnumberofeligiblecasesandthenumberwithcompletedata,
(c) reportthefrequencyofmissingdataforeveryvariableconsidered.Ifthereisonlyasmallamountofoverallmissingness(e.g.>90%
ofcaseswithcompletedata),thenthenumberofincompletevariablesandthemaximumamountofmissingnessinanyvariableare
sufficient
2. approachesforhandlingmissingcovariatedata
(a) providesufficientdetailsofthemethodsadoptedtohandlemissingcovariatedataforallincompletecovariates
(b) giveappropriatereferencesforanyimputationmethodused
(c) foreachanalysis,specifythenumberofcasesincludedandtheassociatednumberofevents
3. explorationofthemissingdata
(a) discussanyknownreasonsformissingcovariatedata
(b) presenttheresultsofanycomparisonsofcharacteristicsbetweenthecaseswithorwithoutmissingdata
Figure3. Proposedguidelinesforreportingmissingcovariatedata(BurtonandAltman2004).ReprintedwithpermissionofMacmillanPublishers
Ltd:BritishJournalofCancer,2004July5;91(1),4–8.
2002.Extensivedetectiveworkbytheauthorsdeterminedthat thatwouldbesufficienttoreplicatethisanalyticapproach.
81%ofthearticleshadmissingdata(thoughthestatuswasun- Bothreviewsindicatethatthereisaconsiderablegapbetween
clearfor4articles,and13hadavailabilityofdataasaninclusion statisticalmethodologiesandmethodsthatarecommonlyused
criteria).Ofthe81articleswithmissingdata,32statedmethods in practice. Flexible comprehensive implementations of these
fortheanalysisofincompleteobservations(severalarticlesused methodsmayspurtheiruse.
more than one method). A total of 12 papers used a complete
caseapproach,12availablecase,6omittedbetweenoneandfour 4. MOTIVATINGEXAMPLE:KIDS’INPATIENT
variables,4usedamissingindicatorapproach,3usedanad-hoc DATABASE
singleimputationprocedure,andonly1paperusedmultipleim-
putation. In responses to these limitations, they proposed a set These methods are demonstrated using the Healthcare Cost
ofguidelinesforthereportingofstudieswithmissingcovariate andUtilizationProject(HCUP)KIDS’InpatientDatabase(KID)
data(Figure3).Inclosing,BurtonandAltman(2004)notedthat: fortheyear2000(HCUPKids’InpatientDatabase(KID)2000).
This dataset, which is publicly available for a fee from the
Weareconcernedthatveryfewauthorshaveconsid-
AgencyforHealthcareQualityandResearch,collectsdatafrom
eredtheimpactofmissingcovariatedata;itseemsthat
statesonchildhospitalizationstoimprovethequalityofhealth
missing data is generally either not recognised as an
care.Weinvestigatedwhatfactorspredictedwhetherapediatric
issueorconsideredanuisancethatisbesthidden(p.
subject with a psychiatric or substance abuse diagnosis had a
6).
routinedischargefromthehospital.
Horton and Switzer (2005) reviewed the use of statistical More specifically, we included all 10–20 year-old subjects
methods in original research articles published during an 18 withaClinicalClassificationsSoftware(CCS)categoryforpri-
month period of 2004–2005 in The New England Journal of mary, secondary or tertiary diagnosis equal to (66) alcohol-
Medicine, a widely read and highly cited medical journal. Of related mental disorders, (67) substance-related mental disor-
then=331papersthatwerereviewed,26(8%)reportedsome ders, (68) senility and organic mental disorders, (69) affective
form of missing data methods. For the purposes of this article disorders, (70) schizophrenia and related disorders, (71) other
wefurtherreviewedthose26manuscripts,findingthat12useda psychoses,(72)anxiety;somatoform;dissociative;andperson-
variantoflastvaluecarriedforward,13usedanad-hocimputa- alitydisorders;(73)pre-adultdisorders,(74)othermentalcon-
tionstrategy(e.g.,meanimputation),and2undertooksensitivity ditions,or(75)personalhistoryofmentaldisorder;mentaland
analyseswheremissingvalueswerereplacedbyworstcaseval- behavioralproblems;observationandscreeningformentalcon-
ues. Two papers used multiple imputation (Smith et al. 2004; dition.
vandeBeeketal.2004).Theentiredescriptionofthisstrategy Theoutcomeinourmodelwasroutinedischargeversusnon-
inSmithetal.(2004)wasthat“missingvalueswereestimated routine discharge (including transfer to a short-term hospital,
bymultiplemultivariateimputation”andacitationwasgivento other facility, release to home health care, dying in hospital or
theMICEapproachofvanBuurenetal.(1999).Thearticleby leavingagainstmedicaladvice).
vandeBeeketal.(2004)notedthatonly320ofthe696patients Predictors in the logistic regression included an indicator of
hadcompletedata,andallpredictorswereusedtoimputemiss- gender(FEMALE,1=female,0=male),AGE(inyears),length
ingvalues(usingamultivariatenormalmodel,thoughsomeof ofstay(LOS,indays),admissiontype(ATYPE,1=emergency,
thevariableswerecategorical).Neitherofthesetwopaperspro- 2=urgent, 3=elective), admission month (AMONTH, used to
videdtheinformationsuggestedbyBurtonandAltman(2004) deriveseasonofadmission,NSEASON),admissiononweekend
TheAmericanStatistician,February2007,Vol.61,No.1 83
AMONTHwasrecodedintoavariableASEASONwherewinter
Table1. DescriptiveStatisticsforKIDDataset
wasdefinedasmonthsDecember,January,orFebruary,spring
asmonthsMarch,April,orMay,andsoon.
Variable Proportion
Figure 4 displays the pattern of missing data using routines
routinedischarge 85.8% withinStata;asimilarpresentationcanbecreatedwithSAS,R
orS-Plus.
weekend 19.7%
female 53.7%
4.3 Results
RACE/ETHNICITY
white 68.4%
black 16.0% Table2displaystheresultsfromthecompletecaseestimator
hispanic 10.3% (n=79,017).However,theuseofthecompletecaseestimator
other 5.3%
meansthatincompleteobservationsareexcludedfromtheanal-
SEASON ysis,eventhoughforalmostallsubjects,completedataonthe
winter 23.7% outcome as well as all but one or two predictors are available.
spring 27.1% We now review software implementations to incorporate these
summer 22.9%
incompleteobservations.
fall 26.3%
ADMISSIONTYPE
emergency 50.6%
5. SOFTWAREPACKAGES
urgent 33.1%
elective 16.3%
Table3liststhemissingdataimplementationsconsideredin
VARIABLE MEAN(SD)
thisreview.Foreachpackage,weprovideageneralintroduction
age(inyears) 16.3(2.7)
lengthofstay(LOS,indays) 6.4(12.7) anddiscussanyparticularissuesrelatedtotheimplementation,
totalcharges(TOTCHG) $9,230($17,371) strengthsorlimitations.Foreachpackage,wehaveincludedthe
numberofdiagnoses(NDX) 3.5(2.0)
code to analyze the motivating dataset along with the relevant
output in an Appendix posted at http://www.math.smith.edu/
muchado-appendix.pdf.
(WEEKEND, 1=Saturday or Sunday, 0=otherwise), number
of diagnoses on original record (NDX), race/ethnicity (RACE,
1=white, 2=black, 3=hispanic, 4=other), and total charges 5.1 AmeliaII
(TOTCHG,indollars).
Amelia II (Honaker, King, and Blackwell 2006) uses a
bootstrapping-basedEMalgorithm(e.g.,EMis,Kingetal.2001)
4.1 DescriptiveStatistics
that is both fast and robust. It includes features for imputing
Table 1 provides descriptive statistics for the observed data cross-sectionalsurveys,timeseriesdata,andtime-series/cross-
fromtheKIDdataset.Morethanfour-fifthsofthesamplewere sectionaldata.Thepackageallowsuserstoputpriorsonindivid-
discharged in a routine fashion, one-fifth during the weekend, ualmissingcellvaluesinthedatamatrix,whenthatknowledge
withmorethanhalffemaleandtwo-thirdswhite/caucasian.The is available. The article by Honaker and King (2006) provides
average age was 16 years, and the length of stay, total charges additionaldescriptionofthepackage.
andnumberofdiagnoseswereallskewedtotheright. Amelia II performs the imputation step. Separate analyses
andcombinationofresultscaneitherbeundertakeninRusing
the Zelig (Imai, King, and Lau 2006) software, or in a sepa-
4.2 MissingData
rate statistics package (e.g., SAS or Stata). The code to install
A total of 133,774 observations were recorded. Data were AmeliaandZeligwithinRorS-PluscanbefoundinFigure
complete for the ROUTINE, FEMALE, AGE, LOS, WEEK- A.1 (Appendix), along with the code to combine the multiple
END,andNDXvariables. imputations.AscreenshotofAmeliaView()canbefoundin
There were missing values for TOTCHG (4% of dataset), Figure A.3 (Appendix) while the output is displayed in Figure
ATYPE(11%ofdataset),RACE(16%ofdataset),andNSEA- A.4(Appendix).
SON (12% of dataset). AMONTH and ATYPE were missing In addition, a self-install package is available which allows
bydesignsincesomestatesrestricttheavailabilityofinforma- a user to install Amelia II without any knowledge of, or even
tion to minimize the possibility of inadvertent reidentification directlyrunning,R.Ifthisrouteitpreferred,Ameliacanoutput
ofsubjectsinsmallerhospitals,whilesomestatesprohibitedre- datasetsforanalysisandcombinationinanotherpackage.Figure
portingdataonRACE.Atotalof79,574(59%)ofobservations A.2 (Appendix) displays the SAS code to read (lines 1–19),
hadcompletedata. analyze(lines21–34),andcombine(lines30–40)outputdatasets
Because LogXact requires variables with missing values to fromAmelia;moreinformationonSAScanbefoundinSection
have no more than 5 levels (coded 0, 1, ..., 4), the variable 5.7.
84 StatisticalSoftwareReviews
. net from http://www.indiana.edu/˜jslsoc/stata
. net install spost9_ado
. misschk
Variables examined for missing values
# Variable # Missing % Missing
--------------------------------------------
1 age 0 0.0
2 atype 15093 11.2
3 aweekend 0 0.0
4 female 0 0.0
5 los 0 0.0
6 ndx 0 0.0
7 race 21880 16.2
8 totchg 5018 3.7
9 routine 0 0.0
10 nseason 15614 11.6
Missing for |
which |
variables? | Freq. Percent Cum.
-------------+-----------------------------------
_2___ _78__ | 33 0.02 0.02
_2___ _7___ | 234 0.17 0.20
_2___ __8__ | 1,213 0.90 1.10
_2___ ____0 | 12 0.01 1.11
_2___ _____ | 13,601 10.09 11.20
_____ _78__ | 73 0.05 11.25
_____ _7__0 | 213 0.16 11.41
_____ _7___ | 21,327 15.82 27.24
_____ __8_0 | 37 0.03 27.26
_____ __8__ | 3,662 2.72 29.98
_____ ____0 | 15,352 11.39 41.37
_____ _____ | 79,017 58.63 100.00
-------------+-----------------------------------
Total | 134,774 100.00
Missing for |
how many |
variables? | Freq. Percent Cum.
------------+-----------------------------------
0 | 79,017 58.63 58.63
1 | 53,942 40.02 98.65
2 | 1,782 1.32 99.98
3 | 33 0.02 100.00
------------+-----------------------------------
Total | 134,774 100.00
Figure4. Descriptionofmissingdata(usingStatamisschkfunction).
5.2 Hmisc 28). Figure A.6 displays the output from aregImpute and
FigureA.7(Appendix)aseriesofgraphicaldisplaysofmissing
The aregImpute function with the HMisc package (Har- datapatterns(fromHmisc).
rell2006)forRandS-Plussupportspredictivemeanmatching
withoptionalweightedprobabilitysamplingfromsimilarcases.
5.3 ICE/Stata
Predictivemeanmatchingworksforbinary,categorical,andcon-
tinuous variables, without the need for computing residuals or PatrickRoyston’sICE(imputationusingchainedequations),
constrainingimputedvaluestobeintherangeofobservedval- implementedwithinStata,providessupportforcategoricalmiss-
ues.Thisapproachtakesallaspectsofuncertaintyintoaccount ingvalues(Royston2005).Binaryvariablesarepredictedfrom
by using the bootstrap to approximate the full Bayesian pre- othervariablesusinglogisticregression,whilecategoricalvari-
dictive distribution for imputations. There is also support for ables with more levels using either a multinomial or ordered
regressionimputationwithinHMisc. logistic regression. Some housekeeping is needed when using
The packages is available for download from CRAN (the indicatorvariablestorepresentthemultiplelevelsofacategori-
Comprehensive R Archive Network). Figure A.5 (Appendix) calvariable.FigureA.8(Appendix)displaysthecodetoanalyze
displaysthecodetoreadinthedataset(lines1–12),creategraph- theKIDdatasetusingICE.
icaldisplays(lines13–15),imputeusingaregImpute(lines The software can be installed over the Internet from within
17–20),thenassessconvergenceandcombineresults(lines21– Stata (lines 1–2). Much of the syntax involves the creation of
TheAmericanStatistician,February2007,Vol.61,No.1 85
Table2. ResultsfromCompleteCaseEstimator(Stata)
. logistic routine age nseas1 nseas2 nseas3 aweekend r2 r3 r4 female
a2 a3 los totchg3 ndx
Logistic regression Number of obs = 79017
LR chi2(14) = 1100.68 Prob > chi2 = 0.0000
Log likelihood = -30802.402 Pseudo R2 = 0.0176
------------------------------------------------------------------------------
routine | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
age | .9603541 .0038511 -10.09 0.000 .9528357 .9679318
nseas1 | .935468 .0278596 -2.24 0.025 .8824273 .9916969
nseas2 | .9818723 .0302025 -0.59 0.552 .9244255 1.042889
nseas3 | 1.038108 .0308933 1.26 0.209 .9792904 1.100459
aweekend | .9437081 .0246635 -2.22 0.027 .8965856 .9933072
r2 | .9825766 .0286533 -0.60 0.547 .9279919 1.040372
r3 | .8133424 .0320256 -5.25 0.000 .7529342 .878597
r4 | .8782054 .039837 -2.86 0.004 .8034966 .9598607
female | 1.093442 .0230012 4.25 0.000 1.049277 1.139465
a2 | 1.401183 .0343642 13.75 0.000 1.335424 1.470181
a3 | 1.474421 .0478478 11.96 0.000 1.383561 1.571248
los | .9960628 .0008627 -4.55 0.000 .9943734 .9977551
totchg3 | .9751805 .0067648 -3.62 0.000 .9620116 .9885298
ndx | .8979635 .0044983 -21.48 0.000 .8891901 .9068234
------------------------------------------------------------------------------
. logit
Logistic regression Number of obs = 79017
LR chi2(14) = 1100.68 Prob > chi2 = 0.0000
Log likelihood = -30802.402 Pseudo R2 = 0.0176
------------------------------------------------------------------------------
routine | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
age | -.0404532 .0040101 -10.09 0.000 -.0483128 -.0325936
nseas1 | -.0667083 .0297815 -2.24 0.025 -.1250789 -.0083377
nseas2 | -.018294 .0307601 -0.59 0.552 -.0785828 .0419947
nseas3 | .0374 .0297592 1.26 0.209 -.020927 .0957271
aweekend | -.0579384 .0261347 -2.22 0.027 -.1091615 -.0067153
r2 | -.017577 .0291614 -0.60 0.547 -.0747323 .0395783
r3 | -.2066032 .0393753 -5.25 0.000 -.2837774 -.1294289
r4 | -.1298747 .0453619 -2.86 0.004 -.2187824 -.0409671
female | .0893302 .0210356 4.25 0.000 .0481013 .1305592
a2 | .3373171 .0245251 13.75 0.000 .2892488 .3853855
a3 | .3882654 .0324519 11.96 0.000 .3246608 .45187
los | -.003945 .0008661 -4.55 0.000 -.0056425 -.0022475
totchg3 | -.0251326 .0069369 -3.62 0.000 -.0387288 -.0115365
ndx | -.1076259 .0050095 -21.48 0.000 -.1174442 -.0978075
_cons | 2.803206 .0730586 38.37 0.000 2.660013 2.946398
------------------------------------------------------------------------------
86 StatisticalSoftwareReviews
Table3. GeneralPurposeSoftwareImplementationsofMissingDataRoutines
Software Vendor/ Approaches
Routine package author implemented
AmeliaII R Honaker, King and hybridEMwithbootstrap
Blackwell
Hmisc RandS-Plus FrankHarrell chained-equationusingpredictedmeanmatch-
ingorregressionimputation
ICE Stata PatrickRoyston chained-equation
IVEware SAS or standalone exe- UniversityofMichigan chained-equation(supportscomplexsurveyde-
cutable signs+constraints)
LogXact LogXact7 Cytel Maximumlikelihood
MICE RandS-Plus vanBuurenetal chained-equation(andpotentialNINRmodels)
PROCMI SASv9.1 SASInstitute MCMCforGaussian,PMM,regression,logistic,
polytomousanddiscriminantmodels
missingdatalibrary S-Plus7 Insightful MaximumlikelihoodorconditionalGaussianim-
putationmodel
indicatorvariableswithappropriatemissingnessstructure(lines ortotalcharges)thatwascontinuous,wehadtodropobserva-
6–24).ThismustbedoneinadvancesothatICEcancreatethem tionswhereTOTCHGwasnotobserved(approximately4%of
from imputed values using the passive and substitute thedataset).Oneofourmissingpredictorswasmonthofadmis-
statements (lines 32–35). To allow commands to be split onto sion;becauseonlyfivelevelswereallowedwecodedthisinto
separatelines,thedelimitstatementisused(lines28and36). a SEASON variable with four levels. Figure A.13 (Appendix)
MultinomiallogitmodelsareusedfortheSEASON,RACE,and displaysascreenshotofLogXact’sresultsfromthemissingdata
ATYPEvariables,whilealinearregressionmodelisusedforto- model.
talcharges(line31).Theimputeddatasetsaresavedintoadataset We ran into some precision or collinearity issues related to
calledimputedandthisisusedtofittheregressionmodelof therangeofTOTCHG(whenthisvariablewasdividedby1,000
interest(lines40–41).FiguresA.9andA.10(Appendix)display the models worked, as well as when we fit the untransformed
theoutput. variable using only complete cases). There was also a minor
bug in the display routines (Cytel reports that these bugs have
been fixed and will ship in the next release). The requirement
5.4 IVEWARE
that categorical variables take on values starting from zero in
sequencetoamaxoffourrequiredsometediousrecoding.
IVEware(http://www.isr.umich.edu/src/smp/ive)byRaghu-
Although there was an option to display the variance-
nathanetal.isaSASversion9callableroutinebuiltusingthe
covariancematrixoftheregressionparameters,therewerefatal
SASmacrolanguageorastandaloneexecutable.Inadditionto
memoryerrorsonthetestingmachinewhenthesewerechecked.
supportingchainedequations,itextendsmultipleimputationto
Accesstothevariancecovariancematrixisneededtocalculate
supportcomplexsurveysampledesigns.
functionsoftheregressionparameters,ortoconductmultiple-df
InstallationoftheSAS-callableversionisstraightforward,as
tests.
itconsistsof19SAScommandfiles.Thedistributionincludesan
exampledatasettohelpverifytheinstallation.FigureA.11(Ap-
pendix)displaysthecodetofitthelogisticregressionimputation 5.6 MICE
modelwhileFigureA.12(Appendix)providestheoutput.
MICE (Multiple Imputation by Chained Equations) is a li-
braryforS-PlusandR.ThepackageisavailablefromtheCom-
prehensiveRArchiveNetwork.Avarietyofimputationmodels
5.5 LogXact
aresupported,includingformsofpredictivemeanmatchingand
LogXact is a standalone package for the analysis of gener- regressionmethods,logisticandpolytomousregression,anddis-
alized linear models. LogXact version 7 incorporates support criminantanalysis.Inaddition,MICEallowsuserstoprogram
forthelikelihoodmethodologyofIbrahim(1990)forupto10 their own imputation functions. In theory, this could facilitate
categoricalcovariateswithmissingvalues(eachwithuptofive sensitivity analyses of different (possibly nonignorable) miss-
levels). There is no practical limit on the number of fully ob- ingness models. The imputation step is carried out using the
servedpredictors.Thesoftwarefunctionsviaagraphicalinter- mice()function.
face.Sinceourmotivatingexamplehadapredictor(TOTCHG, Wenotethatabuginitiallyyieldedincorrectinferencesfrom
TheAmericanStatistician,February2007,Vol.61,No.1 87
thelogisticregressionmodel.Apatchisnowavailableforrelease were missing for TOTCHG (approximately 4% missing), 20
1.14.FigureA.14(Appendix)displaysthecodefrommicefor imputed datasets were created as a function of the fully ob-
readinginthedataset(lines1–9),imputation(lines10–11),and served variables (lines 1–5). Imputation was then carried out
combination of results (lines 12–14). Figure A.15 (Appendix) for the three remaining variables (T=ATYPE, S=SEASON,
displaystheoutputandresults. R=RACE). Lines 15–17 create a variable that describes the
missingness pattern (111=fully observed, 112=race and type
observed,seasonmissing,etc.).Separatemonotoneimputations
5.7 SASPROCMI
arecarriedoutforeachpattern,basedonwhatisobserved(lines
Analysis using multiple imputation in SAS/STAT is carried 26–46)andtheseimputedvaluesaremergedintoasingledata
outinthreesteps.First,theimputationiscarriedoutbyPROC (lines 48–52). The logistic regression model is fit for each im-
MI.Then,completedatamethodsareemployedusinganyofthe puteddataset(lines56–61)andtheresultsareconsolidatedusing
SASproceduresforcompletedataanalysis(e.g.,PROCGLM, MIANALYZE(lines72–76).
GENMOD, PHREG, or LOGISTIC); the “BY” statement re- Thethree-stageapproachusedinSAShighlightsthenotionof
peatstheanalysisforeachcompleteddataset.Finally,theresults usingdifferentsoftwaretoimpute,analyze,andcombinemeth-
arecombinedusingPROCMIANALYZE.Noadditionalinstal- ods.Forexample,ifadesirableimputationpackagedoesnotex-
lationwasneededforPROCMI/PROCMIANALYZE,sinceit istinone’spreferredanalysispackage,onecouldimputeusing
ispartoftheSAS/STATproduct. astand-aloneimputationpackage,savetheresultingcompleted
PROCMIincorporatesanumberofdifferentimputationmeth- datasets,andimportthemfordataanalysis.Theanalysescould
ods.Fornonmonotonemissingdatapatterns,theMCMCstate- then be exported to a third package for combining the results.
mentcanbeusedeitherforallmissingdataortoimputeenough ThisapproachisdemonstratedusingAmeliaforimputationand
datasothattheremainingmissingdataismonotone.Atpresent SASforanalysisinFigureA.2(Appendix).
only continuous variables (those not specified in the CLASS
statement) can be included for MCMC imputation. Extensive
5.8 MissingDataLibraryforS-Plus
control and graphical diagnostics of the MCMC methods are
provided. S-Plus version 7.0 includes a missing data library that ex-
For monotone data, the MONOTONE statement is used to tendsS-Plustosupportmodel-basedmissingdatamodelsusing
describe the imputation method. Continuous variables can be themethodologyofSchafer(1997),byuseoftheEMalgorithm
imputedusingmultivariateregression,regressionusingpredic- (Dempsteretal.1977),anddataaugmentation(DA)algorithms
tivemeanmatching(eachoftheseassumingnormality),orvia (TannerandWong1987).DAalgorithmscanbeusedtogener-
propensityscores.Categoricalvariables,specifiedintheCLASS atemultipleimputations.Themissingdatalibraryprovidessup-
statement,canbeimputedvialogistic(orordinallogistic)regres- portformultivariatenormaldata(impGauss),categoricaldata
sion,discriminantanalysis(onlycontinuouspredictorsallowed) (impLoglin)andconditionalGaussianmodels(impCgm)for
or propensity scores. The approaches can be combined in the imputationsinvolvingbothdiscreteandcontinuousvariables.
same PROC MI, allowing, for example, regression, propensity FigureA.18(Appendix)displaysthecodetoaccessthelibrary
score,anddiscriminantanalysistobeusedincompletingasingle andreadinthedata(lines1–14),createimputeddatasets(lines
dataset. 18–22), and combine results (lines 23–29). Figure A.19 (Ap-
Thepropensityscoreapproachfitsalogisticregressionmodel pendix)displaystheoutputfromtheS-Plusmissingdatalibrary.
predicting the missingness indicator, then orders the observa- Infinitevalueswerecreatedforfourobservationswithmissing
tionsbypredictedprobabilityofmissingness.Next,theordered TOTCHG, which led to nonconvergence of the EM algorithm.
observations are split into G equal-sized groups. Loosely, the These values were dropped in the imputations. This anomaly
imputed value is then chosen at random from among the ob- wasreportedtoInsightfulTechnicalSupport,thoughresolution
served values in the same one of the G groups as the missing isunknownasofpresstime.
observationLavorietal.(1995);Gcanbechosenbytheuser.
The logistic (or ordinal logistic) regression approach is the
5.9 OtherPackagesandRoutines
standard one proposed by Rubin (1987). Analogous to the lo-
gistic regression approach, the discriminant analysis approach Otherpackageswhichprovidesupportforimputationinclude
createsaprobabilityofeachlevelofacategoricalvariable,then JosephSchafer’sfreesoftware(macrosforS-Plusandstandalone
draws a random uniform variate and assigns an imputed level windowspackageNORM),SOLAS,andSPSS.Schafer’ssoft-
basedonthis.Thedistinctionisthattheprobabilitiesarebased wareroutinesareanexcellentcompaniontohisbook,butthey
ondiscriminantanalysis,ratherthanlogisticregression.Theuser do not support general-purpose regression modeling and more
hassomecontroloverdetailsoftheprocessused.Theadvantage modernimplementationsareavailableinS-Plus.SOLASisde-
of this approach is that nominally valued variables can be im- signedspecificallyfortheanalysisofdatasetswithmissingob-
puted,ratherthantheordinalvaluesrequiredforordinallogistic servations,andversion3.0wasreviewedpreviously(Hortonand
regression. Lipsitz1999).BecausethecurrentversionofSOLAS(3.2)does
ThecodetofitPROC/MIusingatwo-stageimputationisdis- notsupportestimationoflogisticregressionmodels,wedidnot
played in Figure A.16 (Appendix), with output in Figure A.17 fit models using the package. Support for missing data is in-
(Appendix). This approach exploits the particular patterns ob- cludedintheSPSSversion12.0missingvaluelibrary,reviewed
servedinthemotivatingexample.Becausesofewobservations byvonHippel(2004).
88 StatisticalSoftwareReviews