Questionnaires in Second Language Research: Construction, Administration, and Processing (Second Language Acquisition Research Series)

165 Pages·2002·6.32 MB·English

by  Zoltan Dornyei

Questionnaires in Second Language
Research
Construction, Administration, and Processing
Second Language Acquisition Research
Theoretical and Methodological Issues
Susan Gass and Jacquelyn Schachter, Editors
Tarone/Gass/Cohen • Research Methodology
in Second Language Acquisition
Schachter/Gass • Second Language Classroom
research: Issues and Opportunities
Birdsong • Second Language Acquisition
and the Critical Period Hypothesis
Ohta • Second Language Acquisition
Processes in the Classroom:
Learning Japanese
Major • Foreign Accent: Ontogeny
and Phylogeny of Second
Language Phonology
Monographs on Research Methodology
Gass/Mackey • Stimulated Recall Methodology
in Second Language Research
Yule • Referential Communication Tasks
Markee • Conversation Analysis
Dornyei • Questionnaires in Second Language
Research: Construction,
Administration, and Processing
Of Related Interest
Gass/Sorace/Selinker • Second Language Learning
Data Analysis, Second Edition
Gass/Selinker • Second Language Acquisition:
An Introductory Course, Second
Edition
Questionnaires in Second Language
Research
Construction, Administration, and Processing
Zoltan Dornyei
University of Nottingham
LAWRENCE ERLBAUM ASSOCIATES, PUBLISHERS
2003 Mahwah, New Jersey London
The camera ready copy for the text of this book was provided by the author.
Copyright © 2003 by Lawrence Erlbaum Associates, Inc.
All rights reserved. No part of this book may be reproduced in any
form, by photostat, microform, retrieval system, or by any other means,
without the prior written permission of the publisher.
Lawrence Erlbaum Associates, Inc., Publishers
10 Industrial Avenue
Mahwah, New Jersey 07430
Cover design by Kathryn Houghtaling Lacey
Library of Congress Cataloging-in-Publication Data
Dornyei, Zoltan.
Questionnaires in second language research: construction, administration,
and processing / Zoltan Dornyei.
p. cm.
Includes bibliographical references and index.
ISBN 0-8058-3908-9 (acid-free paper) - ISBN 0-8058-3909-7 (acid-
free paper)
1. Second language acquisition-Research-Methodology.
2. Questionnaires. I. Title.
P1 18. 2. D67 2002
418'. 007'2-dc21
2002071290
Books published by Lawrence Erlbaum Associates are printed on
acid-free paper, and their bindings are chosen for strength and
durability.
Printed in the United States of America
10 987654321
Contents
INTRODUCTION 1
1 QUESTIONNAIRES IN SECOND LANGUAGE
RESEARCH 3
1. 1 What are 'questionnaires' and what do they measure? 5
1. 1. 1 What a questionnaire is not 6
1. 1. 2 What do questionnaires measure? 8
1. 2 Why use questionnaires and why not? 9
1. 2. 1 Advantages 9
1. 2. 2 Disadvantages 10
1. 3 Questionnaires in quantitative and qualitative
research 14
2 CONSTRUCTING THE QUESTIONNAIRE 16
2. 1 General features 17
2. 1. 1 Length 17
2. 1. 2 Layout 19
2. 1. 3 Sensitive topics and anonymity 21
2. 2 The main parts of a questionnaire 25
2. 2. 1 Title 25
2. 2. 2 Instructions 26
2. 2. 3 Questionnaire items 28
2. 2. 4 Additional information 29
2. 2. 5 Final 'thank you' 30
2. 3 Questionnaire content and multi-item scales 31
2. 3. 1 Appropriate sampling of the content 31
2. 3. 2 Using multi-item scales 32
2. 4 'Closed-ended' questionnaire items 35
2. 4. 1 Rating scales 36
vi CONTENTS
2. 4. 2 Multiple-choice items 43
2. 4. 3 Rank order items 44
2. 4. 4 Numeric items 46
2. 4. 5 Checklists 46
2. 5 Open-ended questions 47
2. 5. 1 Specific open questions 48
2. 5. 2 Clarification questions 48
2. 5. 3 Sentence completion items 49
2. 5. 4 Short-answer questions 49
2. 6 How to write good items 50
2. 6. 1 Drawing up an 'item pool' 51
2. 6. 2 Rules about item wording 52
2. 6. 3 Writing sensitive items 57
2. 7 Grouping and ordering items 59
2. 8 Computer programs for constructing questionnaires 62
2. 9 Piloting the questionnaire and conducting item
analysis 63
2. 9. 1 Initial piloting of the item pool 66
2. 9. 2 Final piloting ('dress rehearsal') 67
2. 9. 3 Item analysis 68
3 ADMINISTERING THE QUESTIONNAIRE 70
3. 1 Selecting the sample 70
3. 1. 1 Sampling procedures 70
3. 1. 2 How large should the sample be? 73
3. 1. 3 The problem of respondent self-selection 75
3. 2 Main types of questionnaire administration 76
3. 2. 1 Administration by mail 77
3. 2. 2 One-to-one administration 81
3. 2. 3 Group administration 82
3. 3 Strategies to increase the quality and quantity of participant
response 83
3. 3. 1 Advance notice 84
3. 3. 2 Attitudes conveyed by teachers, parents, and other
authority figures 85
3. 3. 3 Respectable sponsorship 85
CONTENTS vii
3. 3. 4 The behavior of the survey administrator 86
3. 3. 5 Communicating the purpose and significance of the
survey 86
3. 3. 6 Emphasizing confidentiality 88
3. 3. 7 Questionnaire instructions 89
3. 3. 8 The style and lay out of the questionnaire 89
3. 3. 9 Promising feedback on the results 90
3. 4 Questionnaire administration, confidentiality, and other
ethical issues 91
3. 4. 1 Basic ethical principles of data collection 91
3. 4. 2 Obtaining parental consent for children 93
3. 4. 3 Strategies for getting around anonymity 93
4 PROCESSING QUESTIONNAIRE DATA 96
4. 1 Coding questionnaire data 97
4. 1. 1 First things first: Identification code and 'Research
Logbook' 97
4. 1. 2 The coding frame and the codebook 98
4. 2 Entering the data into a computer file 101
4. 3 Processing closed questions 103
4. 3. 1 Data cleaning 104
4. 3. 2 Data manipulation 105
4. 3. 3 Reducing the number of variables in the
questionnaire 107
4. 3. 4 Main types of questionnaire data 109
4. 3. 5 Examining the reliability and validity of the
data 110
4. 3. 6 Statistical procedures to analyze data 114
4. 4 Content analysis of open-ended questions 116
4. 5 Computer programs for processing questionnaire
data 118
4. 6 Summarizing and reporting questionnaire data 119
4. 6. 1 General guidelines 119
4. 6. 2 Technical information to accompany survey
results 122
4. 6. 3 Reader-friendly data presentation methods 725
viii CONTENTS
4. 7 Complementing questionnaire data with other
information 725
CONCLUSION AND CHECKLIST 132
REFERENCES 137
APPENDIX: SELECTED LIST OF PUBLISHED L2
QUESTIONNAIRES 144
AUTHOR INDEX 150
SUBJECT INDEX 753
Introduction
One of the most common methods of data collection in second lan-
guage (L2) research is to use questionnaires of various kinds. The
popularity of questionnaires is due to the fact that they are easy to
construct, extremely versatile, and uniquely capable of gathering a
large amount of information quickly in a form that is readily process-
able. Indeed, the frequency of use of self-completed questionnaires as
a research tool in the L2 field is surpassed only by that of language
proficiency tests.
In spite of the wide application of questionnaires in the L2 field,
there does not seem to be sufficient awareness in the profession about
the theory of questionnaire design and processing. The usual - and in
most cases false - perception is that anybody with a bit of common
sense can construct a good questionnaire. This situation resembles
somewhat the 'pre-scientific' phase of language testing (i. e., the pe-
riod before the 1950s) when language tests were used without paying
enough attention to their psychometric qualities, and every language
teacher was, by definition, assumed to be capable of devising and
grading tests and exams without any special training. It is my impres-
sion that many questionnaire users are unaware of the fact that there is
considerable relevant knowledge and experience accumulated in vari-
ous branches of the social sciences (e. g., psychometrics, social
psychology, sociology). This is why it is all too common to find
studies which start out with exciting research questions but are flawed
by a badly designed or inadequately processed questionnaire.
In one sentence...
"The essential point is that good research cannot be built on
poorly collected data... "
(Gillham, 2000, p. 1)
1