ERIC ED435192: Beyond Classroom Achievement: Standardized Turkish Tests.

8 Pages·1999·0.21 MB·English

by  ERIC

DOCUMENT RESUME
FL 026 045
ED 435 192
Basci, Pelin
AUTHOR
Beyond Classroom Achievement: Standardized Turkish Tests.
TITLE
ISSN-1095-8096
ISSN
1999-00-00
PUB DATE
NOTE
6p.
AATT Bulletin, Portland State University, Dept. of Foreign
AVAILABLE FROM
Languages and Literatures, P.O. Box 751, Portland, OR 97207.
Descriptive (141)
Reports
Journal Articles (080)
PUB TYPE
AATT Bulletin; n23-24 p14-17 Spr-Fall 1999
JOURNAL CIT
MF01/PC01 Plus Postage.
EDRS PRICE
Higher Education; *Language Tests; Professional
DESCRIPTORS
Associations; Program Descriptions; Rating Scales; Scoring;
*Standardized Tests; *Test Construction; Test Use; *Turkish;
Uncommonly Taught Languages
ABSTRACT
The goals, concerns, and issues addressed by a committee of
the American Association of Teachers of Turkic Languages (AATT) in the
process of developing a standardized Turkish language proficiency test at the
college level are examined. The testing committee aimed at incorporating
recent scholarship on second language acquisition, teaching methods, and
assessment into development of a test for intermediate and advanced
proficiency levels. Scoring was designed to indicate both a competitive
(rating) score and a proficiency level, to meet both diagnostic and placement
needs. The test incorporates visual and authentic materials. Considerations
in test administration and in test reliability and validity are discussed
briefly.
(MSE)
Reproductions supplied by EDRS are the best that can be made
from the original document.
BEYOND CLASSROOM ACHIEVEMENT:
STANDARDIZED TURKISH TESTS
Pelin Basci
Portland State University
U.S. DEPARTMENT OF EDUCATION
Office of Educational Research and Improvement
PERMISSION TO REPRODUCE AND
EDUCATIONAL RESOURCES INFORMATION
DISSEMINATE THIS MATERIAL HAS
CENTER (ERIC)
BEE GRANTED BY
This document has been reproduced as
received from the person or organization
Patin
0_1
originating it.
Minor changes have been made to
improve reproduction quality.
TO THE EDUCATIONAL RESOURCES
° Points of view or opinions stated in this
INFORMATION CENTER (ERIC)
document do not necessarily represent
1
official OERI position or policy.
Spring 1999 Fall 1999
AATT Bulletin
14
BEYOND CLASSROOM
for example, Spanish
or other Romance
languages in the US. Both the
ACHIEVEMENT:
demand for
standardized tests and the
corpus of any data
STANDARDIZED
that result from such tests and would
inform
TURKISH TESTS
assessment research are smaller. Moreover,
in many US academic institutions,
Turkish
instructors are asked to strike
a difficult
Pe lin Besot
balance between their increasingly
diver-
Portland State University
sified academic and teaching
responsibilities
and limited resources.
U
One of the main goals of the testing
committee for Turkish, which
was founded
An increasing separation of tasks has
taken
in the late 1990s, has been to broaden
the
place in many "commonly taught"
second
scope of interest and research in the testing
or foreign languages such as English and
and assessment of Turkish, and thereby
Spanish, where the test builder is
no longer
promote professional awareness of the topic.
necessarily the instructor. This is, of
course,
It is with these broader goals in mind
that I
especially true for standardized
tests,
would like to expand further
on the ques-
although the same principle also applies
to
tions that the testing committee had
to
oral proficiency assessment.' As
a matter
address in preparing model intermediate
of fact, for some of these language
areas,
and advance tests for the 1998
program
assessment is both an independent field of
of the American Research Institute
in
research and a full-fledged industry.
There
Turkey
(ARIT)
for language study in
are advantages and disadvantages to such
Bogazici University.
expansions in any field: the advantage
Second language acquisition research,
stems from the professional work committed
particularly the debate that centered
around
to assessment research and implementation,
the distinction between learning and
acqui-
while the disadvantage most remarkably
sition,2 regardless of its
shortcomings par-
arises from the commercial interests
built
ticularly for non-cognate languages,;
around highly standardized tests such
as the
provided useful analytic categories for
re-
Test of English as a Foreign Language
thinking classroom instruction. Debates
(TOEFL). In some of these
cases, special
about the place of comprehensible input
crash courses, study books, and exercise
produced in meaningful authentic
contexts,
tapes all come to function as parts of
a vast
the presence or absence of the
"monitor" as
enterprise.
a "grammar police" in learners' minds, and
One doubts seriously that standardized
the delicate balance between
accuracy and
tests in Turkish will ever become
commer-
communicative competence have been
cialized to the extent of, for example,
informing foreign language methodology
TOEFL. While this may indeed be the
as well as assessment.4 The preparation
good news, the bad news is that testing
of multi-dimensional tests, rather
than one-
and assessment of Turkish within the US
dimensional paper-and-pen examinations,
context has hardly reached the widespread
incorporating authentic material that
has
professionalism that underlies the
assess-
some appeal and relevance to the test-taker's
ment research, test preparation, and imple-
needs, and designing assessment
models that
mentation in some of the commonly taught
are user-friendly and tests that help students
language areas. No doubt, this is also
due,
rather than keeping them in their
place, have
at least in part, to the status of Turkish
as
been accompanying debates of
instructional
a non-cognate critical language in a largely
methodology.3 Moreover,
with the national
English-speaking world: there is
no com-
move towards clearly defined proficiency
parison between the number of students
goals as the organizing principle
in teaching,
who take Turkish and those who take,
"knowing" a second
or foreign language
Spring 1999 Fall 1999
AATT Bulletin
15
has been re-defined as the ability to perform,
"to do" things, to carry out diverse tasks
In addition to selection, readiness, and
using all four language skills in the authentic
entrance goals, the tests' ability to diagnose
environment of the target language and
and place students into expected levels of
culture.
proficiency implied that the contexts, func-
The AA'rT testing committee aimed
tions, and levels of accuracy to be tested
at incorporating much of the useful recent
should reflect the expectations of the re-
scholarship on acquisition, methods, and
ceiving program. This brought to the fore
assessment into the process of developing
further issues regarding instructional artic-
model tests for two different proficiency
ulation, the implied continuity of levels, and
levels. In this process, the Turkish pro-
expectations among different programs both
ficiency guidelines set the framework for
within and without the US contextan issue
developing the model tests. However, as
that plays a part in decision-making, but is
the committee was charged with the task
beyond the scope of any testing committee's
of creating models for the ARIT test, the
immediate work.
question of intent had to be addressed: to
Having listed some of the goals, major
what degree were models, like those
concerns, and issues to be addressed, it
prepared for ARIT, proficiency assessment
would be thoroughly unrealistic to claim
tests? For the models to be successful, other
with any level of certainty that all of these
significant variables had to be accounted for.
issues were resolved and that "perfect"
Prepared within the framework of pro-
models for standardized national tests
ficiency guidelines, the models had to
were in fact created by the Turkish testing
become a point of departure for reliable
committee. Yet the following may shed light
standardized tests that best suited the needs
on how the committee handled some of
of ARIT and reflected the educational
these concerns.
context that produced the intermediate and
advanced students of Turkish who would
(1) Addressing the objectives: The
take these tests. For the testing committee,
committee designed the scoring to indicate
the identification and recognition of all
both a competitive score and a proficiency
the variables and different criteria in the
level in order to meet the different set of
construction of models were of great
goals mentioned above. This meant that the
significance, for they helped to define
tests would indicate both a demonstrated
what Bachman calls "the abilities we wish
level of proficiencyas in intermediate-mid
to measure and the means of measure-
or intermediate-highand a numerical score
ment."2
showing a ranking of the student in relation
Testswhile by no means the only
to other competitors for selection purposes.
criterion of evaluationhelp ARIT decide
In designing the questions, the com-
what student to select for an in-country
mittee made a serious effort to address
study program and whom to reward with
different proficiency levels for different
a scholarship. In this sense, the models had
skills. For example, in the section that tested
to address simultaneously the "selection,"
speaking, intermediate students were asked
"readiness," and "entrance" goals. Mean-
to communicate concrete, descriptive infor-
while, there was also a greater need for
mation about the
and their immediate
the tests to inform instructors and program
environment; deliver this information
administrators with increased accuracy
through a range of speech conventions
about the students' readiness to pursue
such as questions and commands, using
intermediate and advanced study in the
grammatical structures such as "var/yok",
program, which in turn, required that the
and the future, present continuous, past, and
tests "diagnose" student levels and reduce
aorist tenses; and display a certain amount
the burden of further "placement" pro-
of control and appropriateness in their use
cedures.
4
16
MIT Bulletin
Spring 1999 Fall 1999
large extent on realia: photos and cartoons
of these basic structures. Their functional
from newspapers rather than drawings,
skills were put to test in well-defined tasks
composed
newspaper clippings rather than
in
that related to the expected benchmarks
and unscripted
or altered written texts,
intermediate proficiency levels, such as
conversations depicting a certain negotiation
inviting friends over for dinner and pro-
of meaning such as a rendezvous, to name a
viding them with directions and ordering
few.4
students were
at a restaurant. Advanced
about
asked to display the ability to talk
(4) Reliability and validity: In order to
specialized
the self in a more detailed and
increase the reliability of the model tests,
options,
fashion, incorporating their career
in
attempts were made, even if limited
future plans, and a discussion of cultural
"pre-testing" on those
scope, to administer
differences. The situational contexts in
themselves participate
groups who could not
which their functional skills were tested
constraints
in the "competition." Often, time
include, but are not limited to, retrieving
hampered further
more than anything else
stolen valuables such as a passport, making
despite
work in this direction. Moreover,
grievances at a hotel while employing
all that is said and done, any assessment
appropriate structures and increased
assessed when
process itself can only be
rules
awareness of the socio-linguistic
there is additional input on implementation,
of Turkish.
which is to say the actual administration
process.
(2) Constructing a multi-dimensional test
Finally, it is the input coming from the
while addressing issues of scoring: In
implementation process that will help to
addition to a written text, the tests included
improve the quality and reliability of such
in the form of pictures,
a variety of visuals
models for future use. Were the instructions
photographs, newspaper clippings, and an
clear
to the students and to the proctor
audio section which contained questions
enough? How did the students perform with
spoken
read on a tape and required answers
the time restrictions, since the tests were not
into a tape. The model tests were con-
did
constructed as "speed-tests"? What role
structed to measure student skills at ad-
the physical conditions and limits of the
vanced and intermediate levels, separately
identity
testing environment play? Even the
examining all four areasspeaking,
role in
of the proctor could play a significant
listening, reading, and writingeven though
the implementation of any test. In this case,
of
the committee felt strongly that each
instructor who would
a Turkish-speaking
these seemingly separate receptive and
have an idea of the test material, realize
productive skills contributed to one another.
what is being tested, and could handle
listening
At least in the case of speaking and
unexpected situations regarding the test, is
comprehension the committee therefore
certainly preferable to a substitute with no
designed the scoring guides accordingly.
functional language abilities in Turkish
received
A certain percentage of the scores
and no training in any aspect of teaching
the
in listening contributed, for example, to
and testing. Similarly, the quality and
versa.3
student's scores in speaking and vice
availability of tape recorders at a given
institution, the conditions of the room in
which the test was administeredall of
(3) The use of authentic material: In the
the
these and other variables could impact
testing of different skills, including reading
reliability of any standardized test. It is
and writing, the committee tried to provide
particularly, but not exclusively, on this note
students with communicative contexts such
about implementation that the testing com-
form (intermedi-
as filling in a subscription
the
mittee would like to solicit responses to
ate) and processing information from a film
model tests and invite a larger participation
review (advanced). In all of the questions in
each sub-section, the committee relied to a
5
Spring 1999 Fall 1999
AATT Bulletin
17
in the debate on the assessment of Turkish
as a foreign and/or second language.
NOTES
' There are those who believe that oral-
proficiency assessment, particularly in the form
of an interview, works best if the interviewer is
not the teacher.
2 For a sample of Krashen's prolific work,
see Stephen Krashen, Second Language
Acquisition and Second Language Learning
(New York: Pergamon Press, 1981).
3 For a critique of Krashen and the monitor
model, see Ronald M. Barasch and C. Vaughan
James, ed., Beyond the Monitor Model:
Comments on Current Theory and Practice in
Second Language Acquisition (Boston,
Massachusetts: Heinle & Heinle Publishers,
1994).
4 Theodore V. Higgs, ed., Teaching for
Proficiency, the Organizing Principle
(Lincolnwood: National Textbook Company,
1989).
5 Andrew D. Cohen, Assessing Language
Ability in the Classroom (Boston,
Massachusetts: Heinle & Heinle Publishers:
1994).
5 Lyle Bachman, Fundamental
Considerations in Language Testing (New
York: Oxford University Press, 1990), 81.
6For the scoring guide please,
see Giiliz
Kuruoglu, "Turkish Proficiency Tests: A
National Model" in this issue of the Bulletin.
7 This would be one example in which
certain justifiable compromises were made in the
use of the material. To maintain high-quality
audibility, recordings were produced at a later
time in a studio environment, although every
effort was made during this process to maintain
the natural flow of the original unscripted
conversation.
6
U.S. Department of Education
Office of Educational Research and Improvement (OERI)
I _J
National Library of Education (NLE)
Educational Resources Information Center (ERIC)
REPRODUCTION RELEASE
(Specific Document)
I. DOCUMENT IDENTIFICATION:
Title:
fc.,Gi e
C I
a.3 Croo
Ci
art_c;I
res 4-s "
g
C sCf
Author(s):
POW()
Corporate Source:
- Publication Date:
Spoebi--Fctil lelql
PrATT
Catechli
II. REPRODUCTION RELEASE:
In order to disseminate as widely as possible timely and significant materials of interest to the educational
community, documents announced in the
monthly abstract journal of the ERIC system, Resources in Education (RIE),
are usually made available to users in microfiche, reproduced paper copy,
and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit
if
is given to the source of each document. and-
reproduction release is granted, one of the following notices is affixed to the document.
If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottom
of the page.
The sample sticker shown below will be
The sample sticker shown below will be
The sample sticker shown below will be
affixed to all Level 1 documents
affixed to all Level 2A documents
affixed to all Level 2B documents
PERMISSION TO REPRODUCE AND
DISSEMINATE THIS MATERIAL IN
PERMISSION TO REPRODUCE AND
PERMISSION TO REPRODUCE AND
MICROFICHE, AND IN ELECTRONIC MEDIA
DISSEMINATE THIS MATERIAL IN
DISSEMINATE THIS MATERIAL HAS
FOR ERIC COLLECTION SUBSCRIBERS ONLY,
MICROFICHE ONLY HAS BEEN GRANTED BY
BEEN GRANTED BY
HAS BEEN GRANTED BY
\e
\O
Sa
Sad
TO THE EDUCATIONAL RESOURCES
TO THE EDUCATIONAL RESOURCES
TO THE EDUCATIONAL RESOURCES
INFORMATION CENTER (ERIC)
INFORMATION CENTER (ERIC)
INFORMATION CENTER (ERIC)
2A
2B
Level 1
Level 2A
Level 2B
n
Check here for Level 1 release, permitting reproduction
Check here for Level 2A release, permitting reproduction
Check here for Level 2B release, permitting
and dissemination in microfiche or other ERIC archival
and dissemination in microfiche and in electronic media
reproduction and dissemination in microfiche only
media (e.g., electronic) and paper copy.
for ERIC archival collection subscribers only
Documents will be processed as indicated provided reproduction quality permits.
If permission to reproduce is granted, but no box is checked, documents will be processed at Level 1.
I hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate this document
as indicated above. Reproductidii from the ERIC microfiche or electronic media by persons other than ERIC employees and its system
contractors requires permission from the copyright holder. Exception is made for non-profit reproduction by libraries and other service agencies
to satisfy information needs of educators in response to discrete inquiries.
Sign
Pe
Signature:
Printed Name/Position/Title:
r
("'n
cAz
here,-)
fel
(livers- p
Orgartizatioeddress: tioritc,44,4 st-ctfe
FAX:
please
Mbhi3 .3 2 c- c9 g 9
of
fore r
n k_e>vv.
E-Mail Adpresa,
PO &ox 15' I
7-216-7-
(Cry
170(4-10v.c),
p
Divov,
aocitIn
Inn
'ect.t
(over)
III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE):
If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from another source, please
unless it is publicly
provide the following information regarding the availability of the document. (ERIC will not announce a document
available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantly more
stringent for documents that cannot be made available through EDRS.)
Publisher/Distributor:
Pr f, P
8c*cl)
AA TT
e
ulle 1,^r)
ci Rot-
11.'0
Fo refs
fa(c,
De pact--44
Address:
/Dor
t.1.-ev`vers
e/7 1-
PO gox *Fs
01,19u_ai3es ovtcl A fl-e7atures,
0 g q
2©
Price:
+0 AirTY
b
Free of C4tcy-9e,
b wt.*. 01
IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER:
If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name and
wiriress:
Name:
Address:
V. WHERE TO SEND THIS FORM:
Send this form to the following ERIC Clearinghouse:
OUR NEW ADDRESS AS OF SEPTEMBER 1, 1998
Center for Applied Linguistics
4646 40th Street NW
Washington DC 20016-1859
However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document being
contributed) to:
C Processing and Reference F
1100 West Street, 2" Flo
I, Maryland 207'
L-
598
Telepho
1-497-4080
Toll F
99-3742
8
.3
301-953 I
.v
-mail: [email protected].
: http://ericfac.piccard.csc.
EFF-088 (Rev. 9/97)
PREVIOUS VERSIONS OF THIS FORM ARE OBSOLETE.