Appraising research in second language learning : a practical approach to critical analysis of quantitative research

336 Pages·2002·6.26 MB·Language learning and language teaching, v. 3

by  Graeme Keith Porte

Appraising Research in Second Language Learning
Language Learning & Language Teaching (LL&LT)
The LL&LT monograph series publishes monographs, edited volumes and
text books on applied and methodological issues in the field of language
pedagogy. The focus of the series is on subjects such as classroom discourse
and interaction; language diversity in educational settings; bilingual
education; language testing and language assessment; teaching methods and
teaching performance; learning trajectories in second language acquisition;
and written language learning in educational settings.
Editors
Nina Spada Nelleke Van Deusen-Scholl
Ontario Institute for Studies in Education, Center for Language Study,
University of Toronto Yale University

Volume 28
Appraising Research in Second Language Learning. A practical approach
to critical analysis of quantitative research. Second edition
by Graeme Keith Porte
Appraising Research
in Second Language Learning
A practical approach to critical analysis
of quantitative research
Second edition
Graeme Keith Porte
University of Granada

John Benjamins Publishing Company
Amsterdam / Philadelphia
TM The paper used in this publication meets the minimum requirements of
8
American National Standard for Information Sciences – Permanence of
Paper for Printed Library Materials, ansi z39.48-1984.
Library of Congress Cataloging-in-Publication Data
Porte, Graeme Keith.
Appraising research in second language learning : a practical approach to critical analysis
of quantitative research / by Graeme Keith Porte. -- 2nd ed.
p. cm. (Language Learning & Language Teaching, issn 1569-9471 ; v. 28)

Includes bibliographical references and index.
1. Second language acquisition--Research--Methodology. I. Title.
P118.2.P66 2010
418.007’2--dc22 2010022707
isbn 978 90 272 1991 6 (Hb ; alk. paper)
isbn 978 90 272 1995 4 (Pb ; alk. paper)
isbn 978 90 272 8796 0 (Eb)
© 2010 – John Benjamins B.V.
No part of this book may be reproduced in any form, by print, photoprint, microfilm, or any
other means, without written permission from the publisher.
John Benjamins Publishing Co. · P.O. Box 36224 · 1020 me Amsterdam · The Netherlands
John Benjamins North America · P.O. Box 27519 · Philadelphia pa 19118-0519 · usa
To Cristina and Amy, expert appraisers

Table of contents
Preface xi
Notes on the second edition xxiii
Textbook and worked sample appraisals 1
chapter 1
Introduction: The abstract 3
1.1 Worked sample appraisal 4
chapter 2
Introduction: The problem statement 9
2.1 The background to the problem 11
2.2 The problem statement 12
2.3 Initial indication of variables in the problem statement 13
2.4 Stated contribution to theory and practice 15
Worked sample appraisal 16
chapter 3
Introduction: The review of the literature 20
3.1 Relevance and importance of work cited 22
3.2 Critical address of the literature 24
3.3 Relating the review to the background statement 26
3.4 Time period covered in review 27
3.5 Empirical studies c overed in review 28
Worked sample appraisal 29
chapter 4
Introduction: Research questions and variables 36
4.1 Research questions and hypotheses 36
4.2 Nature of the research questions 38
4.3 Nature of the research hypotheses 39
4.4 General appraisal of the research questions/hypotheses 42
4.5 Identification of variables 45
4.6 Variables potentially affecting findings 50
4.7 Operational definitions 52
Worked sample appraisal 56
viii Appraising research in second language learning
chapter 5
Method: Participants and materials 62
5.1 Descriptive data: Participants 64
5.2 Descriptive data: Groups 68
5.3 Threats to internal validity 72
5.4 Threats to external validity 74
5.5 Descriptive data: Materials 76
5.6 Instruments of measurement: Reliability 78
5.7 Instruments of measurement: Validity 80
5.8 Realisation of main variables 84
Worked sample appraisal 85
chapter 6
Method: Procedures and research design 94
6.1 Procedures: Timing of events 96
6.2 Procedures: Test or practice effects 97
6.3 Procedures: Ethical issues and instructions to participants 99
6.4 Procedures: Reactivity 103
6.5 Procedures: Environmental conditions 107
6.6 Research design: Design boxes 110
6.7 Research design: Classifications 116
Worked sample appraisal 128
chapter 7
Method: Data analysis 136
7.1 Data analysis: Description of procedures 136
7.2 Data analysis: Appropriateness of procedures 138
7.3 Data analysis: Statistical assumptions 140

Worked sample appraisal 146
chapter 8
Results: The nature and presentation of findings 148
8.1 Adequacy of data provided 153
8.2 Graphical displays of results 154
8.3 Descriptive statistics 157
8.4 Statistical procedures 162
8.4.1 Correlation 164
8.4.2 Regression 169
8.4.3 t-tests 173
8.5 Explanations of data 176
8.6 Effect size 179
Table of contents ix
8.7 Other statistical procedures 183
8.7.1 Analysis of variance 183
8.7.2 Chi-square 194
8.8 Practical significance and meaningfulness 199
Worked sample appraisal 200
chapter 9
Discussion and conclusions 206
9.1 Outcomes in terms of the original research questions/hypotheses 208
9.2 Inferences drawn from outcomes 210
9.3 Outcomes related to literature review 212
9.4 Limitations and future research 213
9.5 Practical implications 215
9.6 Assessment of overall confidence 219
Worked sample appraisal 222
Workbook 231
Workbook (guided and unguided appraisals) 233
Abstract 3 (Guided appraisal) 233
Abstract 4 (Unguided appraisal) 235
The background to the problem and the problem
statement 3 (Guid ed appraisal) 236
The background to the problem and the problem
statement 4 (Unguid ed appraisal) 238
The review of the literature 3 (Guided appraisal) 240
The review of the literature 4 (Unguided appraisal) 243
Research questions and variables 3 (Guided appraisal) 245
Research questions and variables 4 (Unguided appraisal) 247
Participants and materials 3 (Guided appraisal) 250

Participants and materials 4 (Unguided appraisal) 254
Procedures and research design 3 (Guided apprais al) 256
Procedures and research design 4 (Unguided apprais al) 259
Results: The nature and presentation of findings 3 (Guided apprais al) 261
Results: The nature and presentation of findings 4 (Unguided apprais al) 264
Discussion and conclusions 3 (Guided appraisal) 267
Discussion and conclusions 4 (Unguided appraisal) 270
Glossary of key terms in quantitative research 273
Appendices 291
Further reading 303
Index of main subject entries in textbook 305