Language Proficiency Testing for Chinese as a Foreign Language: An Argument-Based Approach for Validating the Hanyu Shuiping Kaoshi (HSK)

432 Pages·2014·22.65 MB·English

by  Florian Kaǧan Meyer

Language Testing
and Evaluation
Series editors: Rüdiger Grotjahn
and Günther Sigott
Volume 32
Zur Qualitätssicherung und Peer Notes on the quality assurance
Review der vorliegenden Publikation and peer review of this publication

Die Qualität der in dieser Reihe Prior to publication, the quality
erscheinenden Arbeiten wird of the work published
vor der Publikation durch die
Herausgeber der Reihe geprüft.
in this series is reviewed by
the editors of the series.
Florian Kağan Meyer
Language Proficiency
Testing for Chinese
as a Foreign Language
An Argument-Based Approach for Validating
the Hanyu Shuiping Kaoshi (HSK)
Bibliographic Information published by the Deutsche
Nationalbibliothek
The Deutsche Nationalbibliothek lists this publication in the Deutsche
Nationalbibliografie; detailed bibliographic data is available in the
internet at http://dnb.d-nb.de.
Zugl.: Bochum, Univ., Diss., 2012
Library of Congress Cataloging-in-Publication Data
Meyer, Florian Kagan, 1976-
Language proficiency testing for Chinese as a foreign language : an
argument-based approach for validating the Hanyu Shuiping Kaoshi
(HSK) / Florian Kagan Meyer. – Peter Lang Edition.
pages cm. – (Language testing and evaluation ; Volume 32)
ISBN 978-3-631-64891-9
1. Chinese language–Ability testing. 2. Chinese language–Acquisition.
3. Chinese language–Study and teaching–Foreign speakers. 4. Chinese
language–Spoken Chinese–Examinations. 5. Chinese language–
Examinations, questions, etc. I. Title.
PL1119.M49 2014
495.180076–dc23
2014005520
D 294
ISSN 1612-815X
ISBN 978-3-631-64891-9 (Print)
E-ISBN 978-3-653-03934-4 (E-Book)
DOI 10.3726/978-3-653-03934-4
© Peter Lang GmbH
Internationaler Verlag der Wissenschaften
Frankfurt am Main 2014
All rights reserved.
Peter Lang Edition is an Imprint of Peter Lang GmbH.
Peter Lang – Frankfurt am Main · Bern · Bruxelles · New York ·
Peter Lang – Frankfurt am Main · Bern · Bruxelles · New York ·
Oxford · Warszawa · Wien
All parts of this publication are protected by copyright. Any utilisation
outside the strict limits of the copyright law, without the permission of
the publisher, is forbidden and liable to prosecution. This applies in
particular to reproductions, translations, microfilming, and storage and
processing in electronic retrieval systems.
This book is part of the Peter Lang Edition list and was peer reviewed
prior to publication.
www.peterlang.com
About the author(s)/editor(s)
Florian Meyer studied Sinology, Communication Science and Korean at Free
University Berlin, and Chinese at Peking University. He worked as a lecturer for
Modern Chinese at Ruhr University Bochum (Germany), where he studied
Language Teaching Research and completed his PhD.
About the book
How did the (old) Hanyu Shuiping Kaoshi (HSK) assess the Chinese proficiency
of non-native speakers of Chinese? What inferences can be derived from HSK
test taker scores, especially from Western test takers? How difficult is it to learn
Chinese according to the HSK? Thirty years of research have been synthesized
into an argument-based approach for validating the most widespread test for
Chinese as a foreign language. In addition, the author has analyzed the scores of
a sample of over 250 German test takers in order to investigate how many hours
German natives needed on average to study for reaching a specific HSK level.
This work also extensively discusses validation theory for psychological testing,
and it demonstrates how to put an argument-based approach to validation into
practice.
This eBook can be cited
This edition of the eBook can be cited. To enable this we have marked the start
and end of a page. In cases where a word straddles a page break, the marker is
placed inside the word at exactly the same position as in the physical book. This
means that occasionally a word might be bifurcated by this marker.
| 5 →
Preface and Acknowledgements
In 2003, a German classmate and I listened to some HSK mock tests in my dorm
at Peking University. Unfortunately, we did not comprehend the essential parts
of the dialog we listened to, and not surprisingly, we could not answer the
related items. Suddenly, my Mongolian roommate entered the room, and he
burst out laughing when he listened to the tape. Obviously, we had missed
something funny. My roommate explained what made him laugh: A woman asks
a guy if he also attended the concert last night. In fact, the guy was there as well
and liked it very much. This irritates the woman because she thinks the concert
was horrible, especially the last artist. It turns out that the singer is the wife of
the guy. The woman embarrassedly says that the artist sang quite terrifically
(“with a voice like a bird”), and the problem was only that she had just picked
such an awful song, whereupon the man comments, “So, you didn’t like the
song? I have spent half a year composing it.”1 (cf. Essentials of HSK Listening,
2003 (2nd ed.), p. 53) Afterwards, we listened to the tape again and understood at
least the major parts. My classmate considered the whole dialog “typical HSK
crap” and said that the test would be completely useless2, whereas I thought the
dialog was not absolutely untypical for a daily life situation in China. But why
could we not grasp the main content? I was surprised that I still did not know
what “to applaud”3 meant, although I had been learning Chinese for five years,
and I was astonished that our Mongolian friend, who had studied Chinese as
long as us Germans, was able to understand the dialog.4 Some weeks later, I took
the HSK and reached level 4 out of 8. Many other ← 5 | 6 → Westerners
achieved similar scores, and several complained about the HSK, e.g., that the
test did not directly assess spoken language and that it was simply a multiple-
choice test without any value for assessing the “real” Chinese proficiency. As a
result, many did not take the HSK again. However, I noticed that my Chinese
ability still had shortcomings, and I registered for the HSK again; in the
following years I took the HSK several times, gradually improving my score.
For me, the HSK provided a somehow fair measure of “my” Chinese language
proficiency, and the test appeared useful to me.
Acknowledgements
Acknowledgements
First of all, I want to thank Andreas Guder and Erling von Mende for their
support when I first started my project in October 2007 and when I conducted an
initial survey among 35 HSK test takers in Hannover and Berlin. By May 2009,
surveys had been conducted at all seven HSK test sites, in the end yielding 257
valid questionnaires. Several persons helped to carry out these surveys. Thus, I
want to thank Heidi Brexendorff, Anja Warnecke-Bi, Kathrin Grünke, Liú
Xi oyàn 刘小艳, L Róng, Hú Ch nch n 胡春春, Julia Hauser, Carsten Krause,
ǎ ǐ ū ū
Nadine Reick and Dèng Xi ojìng 邓小婧. I would also like to thank Peter Röhl,
ǎ
who programmed a webpage where test takers participated in the survey via the
Internet.
I must express my deepest thanks to Rüdiger Grotjahn and Henning Klöter,
who adopted my project after I came to Ruhr University Bochum. Rüdiger
Grotjahn gave me advice regarding validity theory and applying validation
approaches. Without his mentoring, which started when I participated in his
seminars about language testing and language assessment, research
methodology, and statistics, this work would not have been possible. Henning
Klöter encouraged me to write this dissertation in English, and he helped me to
submit it to the Faculty of East Asian Studies of the Ruhr University Bochum. I
would like to thank Renée Beine and Mary Ann Poerner for proofreading and
correcting my thesis. Finally, I thank Rüdiger Grotjahn and Günther Sigott for
publishing this work in the series of Language Testing and Evaluation.
Florian Ka an Meyer
ğ
Beijing, January 2014

1 The whole dialog has been transcribed in Appendix A.
2 One can argue that (a) the situation in the dialog is not typical for the life of
foreign university students in the P.R. of China and (b) the text was
unauthentic because it is unclear if the woman knows the man or not. If she
knew him (and the dialog sounded like this), why would she not know that
he was a songwriter from the concert the day before?
3 Whether “to applaud” is an important word or not reflects several issues.
First, the word frequency in the target language domain is important.
Should an HSK test taker who wants to demonstrate that he or she is able to
study in China know this word? Second, if it is part of the target language