Testing for Multigroup Invariance Using AMOS Graphics: A Road Less Traveled

31 Pages·2011·9.18 MB·English

by  Barbara M. Byrne

This article was downloaded by: [University of Groningen]
On: 19 December 2011, At: 02:37
Publisher: Psychology Press
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,
UK
Structural Equation Modeling: A
Multidisciplinary Journal
Publication details, including instructions for
authors and subscription information:
http://www.tandfonline.com/loi/hsem20
Testing for Multigroup
Invariance Using AMOS
Graphics: A Road Less Traveled
Barbara M. Byrne
Available online: 19 Nov 2009
To cite this article: Barbara M. Byrne (2004): Testing for Multigroup Invariance
Using AMOS Graphics: A Road Less Traveled, Structural Equation Modeling: A
Multidisciplinary Journal, 11:2, 272-300
To link to this article: http://dx.doi.org/10.1207/s15328007sem1102_8
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.tandfonline.com/page/terms-
and-conditions
This article may be used for research, teaching, and private study purposes.
Any substantial or systematic reproduction, redistribution, reselling, loan,
sub-licensing, systematic supply, or distribution in any form to anyone is
expressly forbidden.
The publisher does not give any warranty express or implied or make any
representation that the contents will be complete or accurate or up to
date. The accuracy of any instructions, formulae, and drug doses should be
independently verified with primary sources. The publisher shall not be liable
for any loss, actions, claims, proceedings, demand, or costs or damages
whatsoever or howsoever caused arising directly or indirectly in connection
with or arising out of the use of this material.
1
1
0
2
r
e
b
m
e
c
e
D
9
1
7
3
2:
0
at
]
n
e
g
n
ni
o
r
G
f
o
y
sit
r
e
v
ni
U
[
y
b
d
e
d
a
o
nl
w
o
D
STRUCTURAL EQUATION MODELING,11(2),272–300
Copyright ©2004, Lawrence Erlbaum Associates, Inc.
TEACHER’S CORNER
Testing for Multigroup Invariance Using
11 AMOS Graphics: A Road Less Traveled
0
2
r
e
b Barbara M. Byrne
m
e University of Ottawa
c
e
D
9
1
37 Thepurposeofthisarticleistoillustratethestepsinvolvedintestingformultigroup
2: invarianceusingAmosGraphics.Basedonanalysisofcovariance(ANCOV)struc-
0
at tures,2applicationsaredemonstrated,eachofwhichrepresentsadifferentsetofcir-
n] cumstances.Application1focusesontheequivalenceofameasuringinstrumentand
e
g testsforitsinvarianceacross3teacherpanels,givenbaselinemodelsthatareidenti-
n
ni calacrossgroups.Application2centersontheequivalenceofapostulatedtheoreti-
o
r calstructureacrossadolescentboysandgirlsinlightofbaselinemodelsthataredif-
G
f ferentiallyspecifiedacrossgroups.Takentogether,theseillustratedexamplesshould
o
y be of substantial assistance to researchers interested in testing for multigroup
sit invariance using the Amos program.
r
e
v
Uni Insubstantiveresearchthatfocusesonmultigroupcomparisons,itistypicallyas-
y [ sumedthattheinstrumentofmeasurementisoperatinginexactlythesameway,and
b
d thattheunderlyingconstructbeingmeasuredhasthesametheoreticalstructurefor
e
d eachgroupunderstudy.Asevidencedfromreviewsoftheliterature,however,these
a
o twocriticallyimportantassumptionsarerarely,ifever,testedstatistically.Oneap-
nl
w proachtoaddressingthisissueofequivalenceistotestfortheinvarianceofboththe
o
D items and the factorial structure across groups using the analysis of covariance
(ANCOV)structures.Assuch,onewouldtestfortheequivalenceofallitemsde-
signedtomeasuretheconstructunderlyingeachsubscale(i.e.,factor),aswellasre-
lationsamongtheseunderlyingconstructs.Operatingwithinaparallelframework,
thismethodologycanalsobeusedtotestforthemultigroupinvarianceofatheoreti-
calstructurewherethedimensionalityofapsychologicalconstructisofprimaryin-
terest.Forexample,basedondualtheoriesandtheirrelatedempiricalresearch,are-
searcher may wish to test for the invariance of a postulated four-factor structure
versusapostulatedtwo-factorstructureofsocialself-concept(SC)foradolescents
representingdiverseculturalgroups.
TESTING FOR INVARIANCE VIA AMOS 273
To date, almost all studies reported in the literature that address multigroup
invariancebasedontheANCOVstructureshaveusedeithertheLISREL(Jöreskog
&Sörbom,1996)orEQS(Bentler&Wu,2002)programs.Asaconsequence,it
hasbeencustomarytousetheinputfile-basedstrategyassociatedwiththesepro-
grams.(Foradetailedreviewandillustratedapplicationsofthisprocessforboth
LISRELandEQS,readersarereferredtoByrne,1998,1994,respectively.)More
recently,however,thelessconventionalgraphicalapproachtoANCOVstructures
usedbytheAmosprogram(Arbuckle,1999)hasgainedmuchpopularitywithre-
searchers new to the application of this methodology. Although Amos incorpo-
1
1 ratesastandardtext-basedinterface(termedAmosBasic),itsgraphicalinterface
0
2 (termed Amos Graphics) is the one more commonly used.
r
be DespitethisincreaseduseofAmos,however,areviewoftheliteraturesuggests
m
e its negligible use in testing for multigroup invariance. For example, a search of
c
e threejournalsinwhichmultiplegroupcomparisonsareoftenpresentedyieldedno
D
9 evidence of tests for invariance using Amos; these were as follows: Structural
1
7 EquationModeling(Vol.1,1994–Vol.10,2003),JournalofCross-CulturalPsy-
3
2: chology (Vol. 27, 1996–Vol. 34, 2003), Psychological Methods (Vol. 1–Vol. 7,
0
at 2002).Givenitsratherunconventionalapproachtotheseanalyses,thisvoidlikely
] derivesfromalackoffamiliaritywiththeAmosstrategy.Thepurposeofthisarti-
n
ge cle,then,istoprovidesomeassistanceinalleviatingthisdifficulty.Morespecifi-
n
ni cally,basedonANCOVstructures,andusingthegraphicalinterfaceoftheAmos
o
r 4.0program,theprocessoftestingforequivalenceinlightoftwodifferentscenar-
G
f iosisillustrated:(a)Thehypothesizedmultigroupmodelisidenticallyspecified
o
y acrossgroups,and(b)thehypothesizedmultigroupmodelisdifferentiallyspeci-
sit fiedacrossgroups.Although,conceptually,theanalyticapproachissimilaracross
r
e
v these two perspectives, technically, it must necessarily differ when using Amos
ni
U Graphics.Bothsituationsareaddressedinthisarticle.InApplication1,testingfor
[
y theinvarianceofameasuringinstrumentforwhichthespecifiedfactorialstructure
b
d ofthemeasureisidenticalacrossgroupsisdemonstrated.InApplication2,testing
e
d fortheinvarianceofatheoreticalconstructforwhichthepatternoffactorloadings
a
o
nl differsacross groups is demonstrated.
w
o
D
TESTING FORMULTIGROUPINVARIANCE:THE
GENERAL PROCEDURE
Intestingforequivalenciesacrossgroups,setsofparametersareputtothetestina
logicallyorderedandincreasinglyrestrictivefashion,dependingonthemodeland
hypothesestobetested.Inthecaseoftestsfortheinvarianceofameasuringinstru-
mentand/ortheinvarianceofatheoreticalconstruct,thetwoapplicationstobeil-
lustrated in this article, only the factor-loading regression paths and the factor
covariances are of interest. Except in particular instances when, for example, it
274 BYRNE
mightbeofinteresttotestfortheequivalentreliabilityofanassessmentmeasure
across groups (see, e.g., Byrne, 1988), the equality of error variances and
covariances is probably of least importance (Bentler, 2004). Indeed, it is now
widelyacceptedthattestingfortheinvarianceoftheseerrorparametersrepresents
an overly restrictive test of the data.
IntheJöreskogtradition,testsofhypothesesrelatedtogroupinvariancetypi-
callybeginwithscrutinyofthemeasurementmodel.1Inparticular,thepatternof
factorloadingsforeachobservedmeasureistestedforitsequivalenceacrossthe
groups.Onceitisknownwhichobservedmeasuresaregroupinvariant,thesepa-
1
1 rametersareconstrainedequalwhilesubsequenttestsofthestructuralparameters
0
2 areconducted.Aseachnewsetofparametersistested,thoseknowntobegroupin-
r
be variantareconstrainedequalacrossgroups.Giventheunivariateapproachtothe
m
e testing of these hypotheses, as implemented in the Amos program—compared
c
e with,forexample,themultivariateapproachusedintheEQSprogram(Bentler&
D
9 Wu,2002)—thisorderlysequenceofanalyticstepsisbothnecessaryandstrongly
1
7 recommended.
3
2: Asaprerequisitetotestingforfactorialinvariance,itiscustomarytoconsidera
0
at baselinemodel,whichisestimatedforeachgroupseparately.Thisbaselinemodel
] representsonethatbestfitsthedatafromtheperspectivesofbothparsimonyand
n
ge substantivemeaningfulness.However,measuringinstrumentsareoftengroupspe-
n
ni cificinthewaytheyoperate,andthus,baselinemodelsarenotexpectedtobecom-
o
r pletelyidenticalacrossgroups.Forexample,whereasthebaselinemodelforone
G
f groupmightincludecross-loadings2and/orerrorcovariances,thismaynotbeso
o
y forothergroupsunderstudy.Aprioriknowledgeofsuchgroupdifferencesiscriti-
sit caltotheapplicationofinvariance-testingprocedures.Althoughthebulkofthelit-
r
e
v eraturesuggeststhatthenumberoffactorsmustbeequivalentacrossgroupsbefore
ni
U furthertestsofinvariancecanbeconducted,thisstrategyrepresentsalogicalstart-
[
y ingpointonly,andisnotanecessarycondition.Indeed,onlythesimilarlyspeci-
b
d fied parameters within the same factor need be equated (see, e.g., Byrne,
e
d Shavelson, &Muthén, 1989;Werts, Rock, Linn, &Jöreskog, 1976).
a
o
nl Because the estimation of baseline models involves no between-group con-
w
straints,thedatacanbeanalyzedseparatelyforeachgroup.However,intestingfor
o
D invariance,equalityconstraintsareimposedonparticularparameters,andthus,the
dataforallgroupsmustbeanalyzedsimultaneouslytoobtainefficientestimates
(Bentler,2004;Jöreskog&Sörbom,1996);thepatternoffixedandfreeparame-
1AlthoughJöreskog(1971)initiallyrecommendedthatalltestsforinvariancebeginwithaglobal
testoftheequalityofcovariancematricesacrossgroups,researchhasshownthatthistestoftenleadsto
contradictoryfindings.Asaconsequence,itisnolongerregardedasanecessaryprerequisitetothetest-
ing of specific hypotheses related to invariance. (For an elaboration of this issue, see Byrne, 2001.)
2A variable’s measurement loading on more than one factor.
TESTING FOR INVARIANCE VIA AMOS 275
tersnonethelessremainsconsistentwiththebaselinemodelspecificationforeach
group. We turn now to the example applications of interest in this article.
APPLICATION 1
Inthisfirstapplication,hypothesesrelatedtotheinvarianceofasinglemeasuring
instrument are tested across three groups of teachers. Specifically, we test for
1 equivalency of a 20-item adaptation of the Maslach Burnout Inventory (MBI;
1
0 Maslach&Jackson,1986)acrosselementary(n=1,159),intermediate(n=388),
2
r and secondary (n= 1,384) teachers.
e
b
m
e
c
e
D The Hypothesized Model
9
7 1 ThemodelusedinthisfirstapplicationistakenfromastudybyByrne(1993)in
3 whichtheinitialtaskwastotestforthevalidityoftheMBI,a22-iteminstrument
2:
0 designedtomeasurethreedimensionsofburnout—emotionalexhaustion,deper-
] at sonalization,andpersonalaccomplishment.BasedonconsistentfindingsthatItem
n
e 12 (designed to measure Personal Accomplishment) and Item 16 (designed to
g
n measureEmotionalExhaustion)wereproblematicinfittingthemodeltothedata
ni
o forelementary,intermediate,andsecondaryschoolteachers,amodifiedversionof
r
G
f theinstrumentthatexcludedtheseitemswasproposed.Thehypothesizedmodelto
o
y be tested here is based on this 20-item adaptation of theMBI.3
sit
r
e
v
ni The Baseline Models
U
[
y Intestingforthevalidityofthe20-itemMBImodelforeachteachergroup,find-
b
d ings were consistent in revealing exceptionally large correlated errors between
e
ad Items 1 and 2 and between Items 10 and 11. Scrutiny of the content for each of
o
nl these items revealed evidence of substantial overlap between each of these item
w
o pairs,asituationthatcantriggererrorcovariances.Inlightofthissubstantivejusti-
D
fication, these error terms were subsequently specified as free parameters in the
modelforeachteachergroup.Afinalmodelthatreflectedthesemodificationswas
fully cross-validated for independent samples of elementary, intermediate, and
secondary teachers.
Thistestingforabaselinemodel,then,yieldedonethatwasidenticallyspecified
foreachofthethreeteachingpanels.However,itisimportanttoemphasizethatjust
becausetherevisedmodelwassimilarlyspecifiedforeachteachergroup,itinno
3Foramoredetailedaccountofanalysesleadinguptothe20-itemmodel,readersarereferredtothe
original article (Byrne, 1993), or to Byrne (2001).
276 BYRNE
wayguaranteestheequivalenceofitemmeasurementsandunderlyingtheoretical
structureacrossteachergroups;thesehypothesesmustbetestedstatistically.
Thehypothesizedmodelundertestinthisexamplerepresentstherevised20-item
MBIstructure,togetherwiththeadditionoftwoerrorcovariances,asschematically
depictedinFigure1.AtissueistheextenttowhichthismodifiedversionoftheMBI
isequivalentacrosselementary,intermediate,andsecondaryschoolteachers.
1
1
0
2
r
e
b
m
e
c
e
D
9
1
7
3
2:
0
at
]
n
e
g
n
ni
o
r
G
f
o
y
sit
r
e
v
ni
U
[
y
b
d
e
d
a
o
nl
w
o
D
FIGURE1 Baselinemodelofrevised20-itemMBIstructureforelementary,intermediate,
and secondary teachers. From Byrne, 2001. Reprinted with permission.
TESTING FOR INVARIANCE VIA AMOS 277
TESTING FORMULTIGROUPINVARIANCE
USING AMOS GRAPHICS
WhenworkingwithANCOVstructuresthatinvolvemultiplegroups,thedatare-
latedtoeachgroupmust,ofcourse,bemadeknowntotheprogram.Typically,for
moststructuralequationmodeling(SEM)programs,thedataresideinsomeexter-
nalfile,thelocationofwhichisspecifiedinaninputfile.Althoughnoinputfileis
usedwithAmosGraphics,boththenameofeachgroupandthelocationofitsdata
filemustbeconveyedtotheprogrampriortotheanalyses.Thisprocedureisac-
1
1 complishedviatheManageGroupsdialogbox,whichismadeavailableeitherby
0
2 pullingdowntheModel-Fitmenuandselectingthe“ManageGroups”option,or
r
be byusingtheManageGroupsicon .Tobegin,weclickon“New”intheManage
m
e Groupsdialogbox,whichisshowninFigure2.4Eachclickwillyieldthename
c
e “Group,”alongwithanassignednumber.InthecaseofFigure2,thegroupnumber
D
9 (3)pertainstosecondaryteachers;thisnamechangeisinvokedsimplybytyping
1
7 over the former name (see Figure 3).
3
2: Once the group names have been established, the next task is to identify a
0
at datafileforeach,whichisaccomplishedthroughactivationoftheDataFiledia-
] log box. This information is made available either by clicking on the Data File
n
ge icon ,orbypullingdowntheFilemenuandselectingthe“DataFiles”option.
n
ni The Data File dialog box for this application is shown in Figure 4.
o
r Specificationofamultigroupmodel,usingAmosGraphics,isguidedbythede-
G
f faultrulethat,unlessexplicitlydeclaredotherwise,allgroupsintheanalysishave
o
y anidenticalpathdiagramstructure.Assuch,amodelstructureneedsonlytobe
sit drawnforthefirstgroup;allothergroupswillhavethesamestructurebydefault.
r
e
v Giventhatallthreegroupsinthisfirstapplicationhavethesamebaselinemodel,
ni
U this default rule poses no problem.
[
y Testingforinvariancenecessarilyentailsamultistepprocess.However,whenthe
b
d analyses involve more than two groups, and findings reveal evidence of nonin-
e
d variance, the number of steps required in identifying the source of such nonin-
a
o
nl variancecanincreasesubstantially.Intheinterestofclarity,giventhatthisexample
w
involvesthreegroups,eachstepoftheprocesshasbeenidentifiedaccordingly.
o
D
Step 1:Testing for the Validity of the Hypothesized Model
Across Elementary, Intermediate, and Secondary Teachers
Asapreliminarystepintestingforinvarianceacrossgroups,wetestforthevalidity
of MBI structure as best represented by the hypothesized three-factor structure
4Inthebackground,totheleftofFigure2,youseetheAmostoolpalette,whichcontainsthevarious
dialog box icons; to the right, is the model under study (Figure 1), which must remain activated
throughout the analytic process.
1
1
0
2
r
e
b
m
e
c
e
D
9
1
7
3
2: FIGURE2 TheManageGroupsdialogbox:Additionofanewgroup.FromByrne,2001.Re-
at 0 printed with permission.
]
n
e
g
n
ni
o
r
G
f
o
y
sit
r
e
v
ni
U
[ FIGURE3 TheManageGroupsdialogbox:Labelingofanewgroup.FromByrne,2001.Re-
y
b printed with permission.
d
e
d
a
o
nl
w
o
D
FIGURE4 TheDataFilesdialogbox:Identificationofdatafiles.FromByrne,2001.Re-
printed with permission.
TESTING FOR INVARIANCE VIA AMOS 279
showninFigure1.Giventhatthistestofmodelfitwaspreviouslyconductedinthe
processofdeterminingthebaselinemodels,readersmaywonderwhyitisnecessary
torepeattheprocess.Therearetwoimportantreasonsfordoingso.First,whereasthe
formertestswereconductedforeachgroupseparately,testsforthevalidityoffacto-
rialstructureinthisinstanceareconductedacrossthethreegroupssimultaneously.
Inotherwords,parametersareestimatedforallthreegroupsatthesametime.Sec-
ond,intestingforinvarianceusingtheAmosprogram,aswithotherSEMprograms,
thefitofthissimultaneouslyestimatedmodelcanprovidethebaselinevalueagainst
whichallsubsequentlyspecifiedmodelsarecompared.Incontrasttosingle-group
1
1 analyses,however,thismultigroupanalysisyieldsonlyonesetoffitstatisticsfor
0
2 overallmodelfit.Giventhatchi-squarestatistics,togetherwiththeirdegreesoffree-
r
be dom,aresummative,theoverallchi-squarevalueforthemultigroupmodelshould
m
e equalthesumofthechi-squarevaluesobtainedwhenthebaselinemodelistested
c
e separatelyforeachgroupofteachers.Thismultigroupmodelreflectstheextentto
D
9 whichtheMBIstructurefitsthedatawhennocross-groupconstraintsareimposed.
1
7
3
2: Model assessment. Goodness-of-fit statistics related to this three-group
0
at unconstrainedmodel(Model1)arereportedinTable1.Thechi-squarevalueof
] 2243.21,with495df,providesthebaselinevalueagainstwhichsubsequenttests
n
ge for invariance may be compared. Comparative fit index (CFI) and root mean
n
ni squarederrorofapproximation(RMSEA)valuesof.93and.04,respectively,indi-
o
r catedthatthehypothesizedthree-factormodelofMBIstructure,althoughsome-
G
f whatlessthantherecommendedcutoffcriterionof.95recommendedbyHuand
o
y Bentler (1999), still represented a relatively good fit across the three panels of
sit teachers.Accordingly,wenowproceedintestingfortheinvarianceoftherevised
r
e
v 20-itemMBIacrossgroups.(Readersinterestedindetailedinformationprovided
ni
U in the Amos output files are referred to Byrne, 2001.)
[
y
b
d
e Step 2:Testing for Invariance of the Fully Constrained
d
oa Model Across Elementary, Intermediate, and Secondary
nl Teachers
w
o
D Priortotestingfortheequalityofsetsofparameters,asoutlinedearlier,itisalways
worthwhile to test for the possibility that a fully constrained model is invariant
across groups. Regarding this application, this would mean specification of a
modelinwhichallfactorloadings,allfactorvariances,allfactorcovariances,and
thetwoerrorcovariancesareconstrainedequalacrosselementary,intermediate,
andsecondaryteachers.Although,ingeneral,testingfortheequalityoferrorvari-
ancesacrossgroupsisconsideredtobeexcessivelystringent,testingrelatedtothe
error covariances specified in this context is well justified both statistically and
substantively.Thisdecisionwasbasedonthefactthat,foreachteachergroup,the
twoerrorcovarianceswerefoundtobeexcessivelylarge.Scrutinyoftheitemsas-