Challenges in Language Testing Around the World: Insights for Language Test Users

553 Pages·2021·9.251 MB·

by  Betty Lanteigne, Christine Coombe, James Dean Brown

Betty Lanteigne
Christine Coombe
James Dean Brown   Editors
Challenges
in Language
Testing Around
the World
Insights for language test users
Challenges in Language Testing Around the World
· ·
Betty Lanteigne Christine Coombe
James Dean Brown
Editors
Challenges in Language
Testing Around the World
Insights for language test users
Editors
BettyLanteigne ChristineCoombe
LCCInternationalUniversity HigherCollegesofTechnologyDubai
Klaipeda,Lithuania DubaiMen’sCollege
Dubai,UnitedArabEmirates
JamesDeanBrown
UniversityofHawai‘iatMa¯noa
Honolulu,HI,USA
ISBN978-981-33-4231-6 ISBN978-981-33-4232-3 (eBook)
https://doi.org/10.1007/978-981-33-4232-3
©SpringerNatureSingaporePteLtd.2021
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Foreword
The themes of most edited collections in language assessment address traditional
areasoftestdevelopment,scoringmodels,statisticalanalyses,scoreinterpretation,
research, and policy. In addition, they typically provide guidelines for idealized
situationswithnearperfectsolutions.Thereisalmostnevermentionofthingsthat
couldgetoverlooked,misapplied,misinterpreted,misused,andareerroneousintest
development,practice,research,andpolicy.Thisiswherethiscollectioncomesin—
itrevealshumaninfallibilitiesandmisinterpretationsinthedevelopment,practice,
and research in a variety of contexts from many countries. It then offers reflective
chaptersonhowtoresolveorreconcilethesematters.Thisisanuntoldstoryinthe
fieldoflanguageassessment.
Thethirty-sevenchaptersinsixpartsinthiscollectionaredevotedtoexperience-
basedaswellasdata-basedissuesbutarecenteredonreflectingonmistakes,issuesof
washback,fairness,andconstruct-irrelevance.Chaptersreflectonfailingtoconsider
theeffectsofdescriptivestatisticsininterpretingotherimportanttestingstatisticslike
reliability estimates, correlations, criterion-related validity, and failing to account
for the assumptions that underlie higher-order statistics. Other chapters reflect on
importantchallengesrelatedtofairnessofscoreuse,washback,andconsequences
aswellastheuseoflocaltestdevelopment,high-stakesassessments,andplacement
andclassroomassessments.
Oneofthetravestiesinthefieldhasbeenthatinsightsintotestdevelopment,prac-
tice,research,andpolicyhavedependedlargelyoninsightsfromWesternEnglish-
speakingworldcountries.Suchaperspectiveislimitinginvisionandlackingmean-
ingfulnessandapplicabilitytoothercontexts.Thus,itisrefreshingforacollection
toincludechaptersonassessinglanguagevarieties(British,Australasian,andNorth
American),indifferentlanguages(Arabic,Hebrew,andSlovenian)fromavarietyof
countries(Japan,Ukraine,France,Iran,MexicoaswellastheUSA).Furthermore,it
isencouragingtohavechaptersthatdiscussassessingtraditionalskillareas(listening,
speaking,reading,andwriting)fromtheperspectiveoflocalhigh-stakesplacement
testsandclassroomtests.Ofcourse,thebackboneofassessmentinschools,colleges
anduniversitiesislanguageteachers.Itiscriticalthereforethattheyhavetherequi-
site understanding and methods to deal with routine as well as challenging high-
stakescontexts.Thisburgeoningarea,nowreferredtoasassessmentliteracy,isalso
v
vi Foreword
includedinthiscollectionwithperspectivesfromteachersfromTurkey,SriLanka,
Bangladesh,Malta,Iran,theUnitedArabEmirates,andtheUSA.
Another delightful aspect of this collection is the inclusion of authors who are
probablyfirst-timeauthorsorlesspublishedresearchersalongwithwell-knownand
famousauthors.Thisinclusionisanintelligentchoiceastheformergroupofauthors
couldofferaviewthatisnewandeye-openingwhilethelattergroupmightseekto
consolidatetheirwell-knownpositionsonemoretime.Thisideaofmixingauthors
from different regions can also be seen in the choice of editors of this collection:
Betty Lanteigne (from Europe), Christine Coombe (from the Asia/Middle-East),
and James Dean Brown (from the USA). Such collaboration can surely bring to
theforemultipleperspectivesinchoosingauthors,themes,andchapters.Weseean
outstandingexampleofthisinthisextraordinaryvolume.
SanGabriel,CA,USA AntonyJohnKunnan
October2020
Acknowledgments
First and foremost we would like to thank all of the contributors for sharing their
observations about, experiences with and investigations of challenges in language
testing in many parts of the world. We also appreciate their patience throughout
theentirepublicationprocess.GratitudeisalsoextendedtotheSpringerreviewers,
whoseinsightfulcommentshelpedshapeChallengesinLanguageTestingAroundthe
World.SpecialthanksgototheSpringerteam:LawrenceLiu,SeniorEditor,whose
initiative was the spark that began this project; Lay Peng Ang, Senior Editorial
Assistant; Sriram Srinivas, Project Coordinator; and Grace Liyan Ma, Editor, for
theirorganizedandpracticalassistancewithproducingthiseditedvolume.
Wededicatethisvolumetoourstudentsandcolleagueswhohavehelpeduswork
throughassessmentchallengesandturnedthemintolearningopportunities.
vii
Contents
1 IntroducingChallengesinLanguageTestingAroundtheWorld ..... 1
BettyLanteigne,ChristineCoombe,andJamesDeanBrown
PartI LearningfromLanguageTestInterpretationProblems,
NegativeEffects,orMisuse
2 Problems Caused by Ignoring Descriptive Statistics
inLanguageTesting ........................................... 15
JamesDeanBrown
3 Disregarding Data Due Diligence Versus Checking
andCommunicatingParametricStatisticalTestingProcedure
Assumptions .................................................. 25
PhillipB.Rowles
4 Washback of the Reformed College English Test Band
4 (CET-4) in English Learning and Teaching in China,
andPossibleSolutions ......................................... 35
FeifeiHan
5 FairnessinCollegeEntranceExamsinJapanandthePlanned
UseofExternalTestsinEnglish ................................ 47
YukoGotoButlerandMasakazuIino
6 (Mis)UseofHigh-Stakes Standardized TestsforMultiple
PurposesinCanada?ACallforanEvidence-BasedApproach
toLanguageTestingandRealignmentofInstruction ............. 57
Li-ShihHuang
7 Testing in ESP: Approaches and Challenges in Aviation
andMaritimeEnglish ......................................... 67
NeilBullockandCarolynWestbrook
8 AConceptualFrameworkonthePowerofLanguageTests
asSocialPractice .............................................. 79
TuçeÖztürkKaratas¸andZuhalOkan
ix
x Contents
9 TheWashbackEffectoftheVietnamSix-LevelsofForeign
LanguageProficiencyFramework(KNLNNVN):TheCase
oftheEnglishProficiencyGraduationBenchmarkinVietnam ..... 99
,,
PhuongHoaÐinhThi.andHandoyoPujiWidodo
10 AvoidingScoringMalpractice:SupportingReliableScoring
ofConstructed-ResponseItemsinHigh-StakesExams ............ 127
KristinaLeitnerandBenjaminKremmel
11 Score Changes with Repetition of Paper Version(s)
oftheTOEFLinanArabGulfState:ANaturalExperiment ...... 147
BettyLanteigneandHanaSulieman
PartII LearningfromTestsofWorldLanguages
12 WhoseEnglish(es)AreWeAssessingandbyWhom? ............. 169
LiyingCheng,Gwan-HyeokIm,andRubaiyatJabeen
13 Challenges in Developing Standardized Tests for Arabic
Reading Comprehension for Secondary Education
intheNetherlands ............................................. 181
AnnekedeGraaf
14 TheConflictandConsequencesofTwoAssessmentMeasures
inIsrael:GlobalPISAvs.theNationalMEITZAV ................ 191
OfraInbar-LourieandElanaShohamy
15 How to Challenge Prejudice in Assessing the Productive
SkillsofSpeakersofCloselyRelatedLanguages(theCase
ofSlovenia) ................................................... 201
InaFerbežarandPetraLikarStanovnik
PartIII LearningfromProgram-LevelLanguageTests
16 EFLPlacementTestinginJapan ................................ 223
JamesCarpenterandSawakoMatsugu
17 TEFLTestPracticesataUkrainianUniversity:Summative
TestDesignThroughTeacherCollaboration ..................... 233
OlgaKvasova
18 Designing a Multilingual Large-Scale Placement Test
withaFormativePerspective:ACaseStudyattheUniversity
ofGrenobleAlpes ............................................. 243
CristianaCerviniandMonicaMasperi
19 TheRelationshipBetweenEnglishPlacementAssessments
and an Institution: From Challenge to Innovation
foranIntensiveEnglishProgramintheUSA .................... 255
NicholasSantavicca
Contents xi
20 PlacementDecisionsinPrivateLanguageSchoolsinIran ......... 267
KioumarsRazavipourandTaherehFiroozi
21 Perceptions of (Un)Successful PET Results at a Private
UniversityinMexico ........................................... 287
LuisAlejandroFigueroaandKrisztinaZimányi
PartIV LearningfromTestsofLanguageSkills
22 CompletingtheTriangleofReadingFluencyAssessment:
Accuracy,Speed,andProsody .................................. 307
JihyeShin
23 (Re)Creating Listening Source Texts for a High-Stakes
Standardized English Test at a Vietnamese University:
AbandoningtheSearchinVain ................................. 317
XuanMinhNgo
24 The Oral Standardized English Proficiency Test:
Opportunities Provided and Challenges Overcome
inanEgyptianContext ........................................ 327
DeenaBoraieandRamyShabara
25 Opening the Black Box: Exploring Automated Speaking
Evaluation .................................................... 333
NahalKhabbazbashi,JingXu,andEvelinaD.Galaczi
26 Developing a Meaningful Measure of L2 Reading
ComprehensionforGraduateProgramsataUSAResearch
University:TheRoleofPrimaryStakeholders’Understanding
oftheConstruct ............................................... 345
AhmetDursun, NicholasSwinehart, JamesMcCormick,
andCatherineBaumann
27 ChallengingtheRoleofRubrics:PerspectivesfromaPrivate
UniversityinLebanon ......................................... 357
ChristineSabieh
28 AMixed-MethodsApproachtoStudytheEffectsofRater
Training on the Scoring Validity of Local University
High-StakesWritingTestsinSpain ............................. 383
JuliaZabala-Delgado
PartV LearningfromTests,Teachers,andLanguageAssessment
Literacy
29 ACriticalEvaluationoftheLanguageAssessmentLiteracy
ofTurkishEFLTeachers:SuggestionsforPolicyDirections ....... 411
ElçinÖlmezer-Öztürk,GökhanÖztürk,andBelginAydın