Table Of ContentSpringer International Handbooks
of Education
Trude Nilsen · Agnes Stancel-Piątak
Jan-Eric Gustafsson
Editors
International
Handbook of
Comparative
Large-Scale Studies
in Education
Perspectives, Methods and Findings
Springer International Handbooks of
Education
The Springer International Handbooks of Education series aims to provide easily
accessible, practical, yet scholarly, sources of information about a broad range of
topics and issues in education. Each Handbook follows the same pattern of exam-
iningindepthafieldofeducationaltheory,practiceandappliedscholarship,itsscale
andscopeforitssubstantivecontributiontoourunderstandingofeducationand,in
sodoing,indicatingthedirectionoffuturedevelopments.Thevolumesinthisseries
form a coherent whole due to an insistence on the synthesis of theory and good
practice. The accessible style and the consistent illumination of theory by practice
make the series very valuable to a broad spectrum of users. The volume editors
representtheworld'sleadingeducationalists.Theirtaskhasbeentoidentifythekey
areasintheirfieldthatareinternationallygeneralizableand,intimesofrapidchange,
ofpermanentinteresttothescholarandpractitioner.
(cid:129) ą (cid:129)
Trude Nilsen Agnes Stancel-Pi tak
Jan-Eric Gustafsson
Editors
International Handbook of
Comparative Large-Scale
Studies in Education
Perspectives, Methods and Findings
With194Figuresand114Tables
Editors
TrudeNilsen AgnesStancel-Piątak
DepartmentofTeacherEducation IEAHamburg
andSchoolResearch,Facultyof Hamburg,Hamburg,Germany
EducationalSciences
UniversityofOslo
Oslo,Norway
Jan-EricGustafsson
UniversityofGothenburg
Gothenburg,Sweden
ISSN2197-1951 ISSN2197-196X(electronic)
SpringerInternationalHandbooksofEducation
ISBN978-3-030-88177-1 ISBN978-3-030-88178-8(eBook)
https://doi.org/10.1007/978-3-030-88178-8
©SpringerNatureSwitzerlandAG2022
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartofthe
materialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsorthe
editorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforanyerrors
oromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictionalclaims
inpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG.
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Sometimeago,wewereextremelypleasedtoseeourmailboxesbeingfilledupwith
positive responses to our invitations to contribute chapters to the handbook. We
knewwehadaimedhighandwerehencethrilledtoseethatthegeneralmessageof
thefeedbackwasthatthisisanexcellentinitiativeandthatsuchahandbookissorely
missed and highly needed. Almost all of the large number of distinguished
researchersweinvitedacceptedourinvitation.
Indeed, there was an agreement that a handbook on international large-scale
assessment(ILSA)isneeded.ILSAshaveaffectededucationalresearch,contributed
to the development of methodology and theory, and have had a large impact on
educationalpolicy.Afterabout60yearsofinternationallarge-scalestudiesandafter
more than 25 years since the first of the current ILSAs (TIMSS 1995) was
established, huge developments are evident. Enormous amounts of data have been
produced,whichhavebeenmadefreelyavailableonline.Thisaccumulationofhigh-
quality data and research calls for reviews to collect and summarize all the extant
evidence.Suchreviewshavebeenoneofthemaingoalsofthehandbook.
Atthesametime,mythsandlegendstendtoproliferatemediaandotheroutlets,
andILSAshavebeenmadetargetsofbothinaccurateandaccuratecritique.ILSAs
heavilydependoncriticismtoimproveastheyimpactbothresearchandpolicy.Itis
thereforeimportantthatthecriticismisvalidandthatitreachesabroadaudienceof
educationalpolicymakersandstakeholders,studentsandbeginningresearchers,and
researchers who representmany different fields. Furthermore,researchers involved
inILSAsneedknowledgeonhowtousealargesetoftools,forexampletoanalyze
data, make causal inferences, create robust conclusions, and present findings. The
handbook hence aims to gather and present information from several perspectives,
includingtheory,history,policy,methodology,andILSAfindings.
Suchabroadandall-encompassinghandbookneededcarefulplanning.Weaimed
for an inclusive handbook that would present different perspectives and mirror
various research traditions. It was important for us to include both Western and
non-Westernperspectives,aswellasincludingvoicesfromtheILSAenvironments,
but also critical voices. And we wanted to review findings from studies that have
usedILSAdatafromallmajorfields.Thiswasalargeendeavorthatdemandedalot
ofplanning.Thus,wewouldliketoexpressourgratitudetothesectioneditorProf.
SigridBlömekeforhercritiqueandfruitfulsuggestionsthathelpedusreachouraim
v
vi Preface
ofahandbookembracingculturaldiversityandmultipleperspectives.Moreover,we
wouldliketothankthesectioneditorProf.RonnySchererforhiscontributionstothe
development and the extensive help with the review process and to the section
editorsProf.DavidKaplanandSarahHowiefortheirpatience,insights,andsupport.
We are very grateful to all authors who contributed to the book with their
excellent research, their insightful thoughts and reflections, and their sophisticated
constructivecriticismonILSA.
Finally, we would like to extend our special thanks to the reviewers who have
worked on the handbook conducting thorough reviews and providing extensive
comments to the submitted chapters. Numerous reviewers have worked on the
handbook, despite the pandemic situation and the difficulties everyone had to face
duringthistime.Theircontributiontothepublicationisenormous.
The different strengths and profiles of the editors have contributed to excellent
collaborations and enhanced the quality of the process and product. We hope the
readers will enjoy the insights and knowledge from the many authors of this
handbook.
Oslo,Norway TrudeNilsen
Hamburg,Germany AgnesStancel-Piątak
Gothenburg,Sweden Jan-EricGustafsson
August2022
Contents
Volume 1
PartI AboutThisHandbook ............................... 1
1 IntroductiontotheComparativeLarge-ScaleStudiesin
Education:StructureandOverviewoftheHandbook .......... 3
SigridBlömeke,SarahJ.Howie,DavidKaplan,andRonnyScherer
2 Background,Aims,andTheoriesoftheComparative
Large-ScaleStudiesinEducation .......................... 13
TrudeNilsen,Jan-EricGustafsson,andAgnesStancel-Piątak
PartII Meta-perspectivesonILSAs:TheoreticalMeta-perspectives
onILSAs ................................................. 25
3 ThePoliticalEconomyofILSAsinEducation:TheRole
ofKnowledgeCapitalinEconomicGrowth .................. 27
EricA.HanushekandLudgerWoessmann
4 ReasonsforParticipationinInternationalLarge-Scale
Assessments ........................................... 55
JiLiuandGitaSteiner-Khamsi
5 EducationalAccountabilityandtheRoleofInternational
Large-ScaleAssessments ................................. 75
SusannaLoebandErikaByun
6 InternationalLarge-ScaleAssessmentsandEducationSystem
Reform .............................................. 97
M.Ehren
7 TheRoleofInternationalLarge-ScaleAssessments(ILSAs)in
EconomicallyDevelopingCountries ........................ 119
SyedaKashfeeAhmed,MichelleBelisle,ElizabethCassity,
TimFriedman,PetraLietz,andJeanieneSpink
vii
viii Contents
PartIII Meta-perspectivesonILSAs:TheRoleofTheory
inILSAs ................................................. 143
8 ComprehensiveFrameworksofSchoolLearninginILSAs ...... 145
AgnesStancel-PiątakandKnutSchwippert
9 AssessingCognitiveOutcomesofSchooling .................. 175
FrederickKoonShingLeungandLeisiPei
10 SocioeconomicInequalityinAchievement ................... 201
RolfStrietholtandAndrésStrello
11 MeasuresofOpportunitytoLearnMathematicsinPISA
andTIMSS:CanWeBeSurethatTheyMeasureWhatThey
AreSupposedtoMeasure? ............................... 221
HansLuytenandJaapScheerens
12 UsingILSAstoPromoteQualityandEquityinEducation:
TheContributionoftheDynamicModelofEducational
Effectiveness .......................................... 253
LeonidasKyriakides,CharalambosY.Charalambous,and
EviCharalambous
PartIV Meta-perspectivesonILSAs:CharacteristicsofILSAs .... 277
13 OverviewofILSAsandAspectsofDataReuse ............... 279
NathalieMertes
14 IEA’sTIMSSandPIRLS:MeasuringLong-TermTrendsin
StudentAchievement ................................... 305
InaV.S.MullisandMichaelO.Martin
15 IEA’sTeacherEducationandDevelopmentStudyinMathematics
(TEDS-M) ............................................ 325
SigridBlömeke
16 OECDStudiesandtheCaseofPISA,PIAAC,andTALIS ...... 379
AndreasSchleicher,MiyakoIkeda,WilliamThorn,and
KarineTremblay
17 RegionalStudiesinNon-WesternCountries,andtheCaseof
SACMEQ ............................................ 419
SarahJ.Howie
18 TheUseofVideoCapturinginInternationalLarge-Scale
AssessmentStudies:MethodologicalandTheoretical
Considerations ......................................... 469
KirstiKlette
Contents ix
19 ComparisonofStudies:ComparingDesignandConstructs,
AligningMeasures,IntegratingData,Cross-validating
Findings .............................................. 511
EckhardKlieme
PartV Meta-perspectivesonILSAs:Accomplishments,
Limitations,andRecommendations .......................... 545
20 ILSAinArtsEducation:TheEffectofDramaon
Competences .......................................... 547
RikkeGürgensGjærum,AdamCziboly,andStigA.Eriksson
21 FutureDirections,Recommendations,andPotential
DevelopmentsofILSA .................................. 579
DirkHastedtandHeikoSibberns
22 ConceptualandMethodologicalAccomplishmentsofILSAs,
RemainingCriticismandLimitations ....................... 603
SigridBlömeke,TrudeNilsen,RolfV.Olsen,andJan-EricGustafsson
Volume 2
PartVI Methodology:DesigningandImplementingILSAs ...... 657
23 SamplingDesigninILSA ................................ 659
SabineMeinckandCarolineVandenplas
24 DesigningMeasurementforAllStudentsinILSAs ............ 685
DavidRutkowskiandLeslieRutkowski
25 ImplementingILSAs .................................... 701
JulietteLyons-Thomas,KadriyeErcikan,EugeneGonzalez,and
IrwinKirsch
26 DilemmasinDevelopingContextQuestionnairesforInternational
Large-ScaleAssessments ................................. 721
MartinHooper
27 MultistageTestDesignConsiderationsinInternational
Large-ScaleAssessmentsofEducationalAchievement ......... 749
LeslieRutkowski,DavidRutkowski,andDubravkaSvetinaValdivia
PartVII Methodology:MethodsofAnalysis .................. 769
28 SecondaryAnalysisofLarge-scaleAssessmentDatabases ....... 771
EugenioJ.Gonzalez
29 MethodsofCausalAnalysiswithILSAData ................. 803
Jan-EricGustafssonandTrudeNilsen
