Table Of ContentMeta-Regression Analysis in
Economics and Business
Meta-Regression Analysis in Economics and Business is the first guide through
the rapidly expanding field of meta-analysis in economics and business. Have you
ever wondered, for example, whether a raise in the minimum wage really l owers
employment or if taxes will cause people to conserve water? Meta-analysis is the way
that science takes stock of our vast research output. Meta-analysis is a s tatistical and
systematic review of all relevant research. It produces the a uthoritative a ssessments
required for evidence-based practice in medicine, social sciences, economics, and
business.
The purpose of this book is to introduce novice researchers to the tools of
meta-analysis and meta-regression analysis and to summarize the state of the art
for existing practitioners. Meta-regression analysis addresses the rising “Tower of
Babel” that current economics and business research has become. Meta-analysis
is the statistical analysis of previously published, or reported, research findings
on a given hypothesis, empirical effect, phenomenon, or policy intervention. It is
a systematic review of all the relevant scientific knowledge on a specific subject
and is an essential part of the evidence-based practice movement in medicine,
education, and the social sciences. However, research in economics and business
is often fundamentally different from what is found in the sciences and thereby
requires different methods for its synthesis—meta-regression analysis. This book
develops, summarizes, and applies these meta-analytic methods.
Meta-Regression Analysis in Economics and Business offers the first
comprehensive guide to conducting and understanding the type of meta-analysis
(meta-regression analysis) needed for econometric studies. Actual systematic
reviews of research are used throughout the book to illustrate the use of these
meta-analytic methods. Among other things, it contains the first theory of meta-
regression analysis, novel methods for correcting publication bias, and a rigorous
demonstration that study quality will not affect meta-regression analysis.
T.D. Stanley is Bill and Connie Bowen Odyssey Professor of Economics at
Hendrix College, Conway, AR, USA.
Hristos Doucouliagos is Professor in the School of Accounting, Economics and
Finance, Deakin University, Melbourne, Australia.
Routledge Advances in Research Methods
1 E-Research
Transformation in scholarly practice
Edited by Nicholas W. Jankowski
2 The Mutual Construction of Statistics and Society
Edited by Ann Rudinow Sætnan, Heidi Mork Lomell, and Svein Hammer
3 Multi-Sited Ethnography
Problems and possibilities in the translocation of research methods
Edited by Simon Coleman and Pauline von Hellermann
4 Research and Social Change
A relational constructionist approach
Sheila McNamee and Dian Marie Hosking
5 Meta-Regression Analysis in Economics and Business
T.D. Stanley and Hristos Doucouliagos
Meta-Regression Analysis
in Economics and Business
T.D. Stanley and
Hristos Doucouliagos
First published 2012
by Routledge
2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN
Simultaneously published in the USA and Canada
by Routledge
711 Third Avenue, New York, NY 10017
Routledge is an imprint of the Taylor & Francis Group, an informa business
© T.D. Stanley and Hristos Doucouliagos 2012
The right of T.D. Stanley and Hristos Doucouliagos to be identified as authors
of this work has been asserted by them in accordance with sections 77 and 78
of the Copyright, Designs and Patent Act 1988.
All rights reserved. No part of this book may be reprinted or reproduced or
utilised in any form or by any electronic, mechanical, or other means,
now known or hereafter invented, including photocopying and recording,
or in any information storage or retrieval system, without permission in
writing from the publishers.
Trademark notice: Product or corporate names may be trademarks or
registered trademarks, and are used only for identification and explanation
without intent to infringe.
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
Library of Congress Cataloging-in-Publication Data
Stanley, T. D., 1950–
Meta-regression analysis in economics and business/T.D. Stanley and
Hristos Doucouliagos.
p. cm.
1. Economics–Research–Methodology. 2. Economics–Research–
Evaluation–Statistical methods. 3. Economics literature–Evaluation–
Statistical methods. 4. Regression analysis. I. Doucouliagos,
Hristos. II. Title.
HB131.S73 2012
330.01’519536–dc23
2011052596
ISBN: 978-0-415-67078-4 (hbk)
ISBN: 978-0-203-11171-0 (ebk)
Typeset in Times New Roman
by Sunrise Setting Ltd
Contents
List of figures vii
List of tables viii
Acknowledgments ix
1 Introduction 1
1.1 The Tower of Research 1
1.2 A historical sketch of meta-regression analysis 5
1.3 Practical examples 9
1.4 Plan of the book 10
2 Identifying and coding meta-analysis data 12
2.1 Identifying studies 13
2.2 What data to collect 20
2.3 Effect sizes in economics and their standard errors 22
2.4 Coding issues 29
2.5 The quality conundrum: should estimates be combined? 33
2.6 Summary 37
3 Summarizing meta-analysis data 38
3.1 Illustrating data 38
3.2 Summary measures 43
3.3 Statistical significance versus economic significance 48
3.4 Testing for heterogeneity 48
3.5 Recap: summarizing research 49
4 Publication bias and its discontents 51
4.1 Publication selection 51
4.2 Funneling research to identify and correct publication
selection bias 53
vi Contents
4.3 Simple meta-regression models of publication selection 60
4.4 Alternative approaches to publication selection 72
4.5 Recap: The FAT-PET-PEESE approach to publication
selection 78
5 Explaining economics research 80
5.1 Heterogeneity 81
5.2 Multivariate models of research 84
5.3 Illustrations of multiple meta-regression analysis 89
5.4 Robustness and dependence 99
5.5 Will the real meta-regression analysis model please
stand up? 102
5.6 Recap: explaining the heterogeneity of economics
research 104
6 Econometric theory and meta-regression analysis 106
6.1 The theory of meta-regression analysis 106
6.2 Improving meta-regression analysis with unbalanced
panel models 112
6.3 Meta-regression models of publication selection 117
6.4 In defense of simple statistical methods 120
Appendix: assumptions about error structures 123
7 Further topics in meta-regression analysis 125
7.1 Alternative applications of meta-regression analysis 125
7.2 Specification of the meta-regression analysis 130
7.3 Functional form of the meta-regression analysis 131
7.4 Exclusion restrictions 132
7.5 Evaluating predictions from meta-regression analysis 132
7.6 Effects with interaction and non-linear terms 135
7.7 Multiple effect size analysis 136
7.8 Meta-meta-analysis 140
7.9 Summary 145
8 Summary and conclusions 147
Notes 154
References 168
Index 180
Figures
1.1 Meta-analysis in economics over time 8
1.2 The exponential growth of meta-analysis in economics 8
3.1 Funnel plot of union-productivity partial correlations 40
3.2 Chronological ordering of data 43
4.1 Funnel plot of union-productivity partial correlations 53
4.2 Symmetric funnel plots 54
4.3 Funnel graph of price elasticity for water demand 55
4.4 Value of a statistical life 57
4.5 Asymmetrical funnel plots 58
4.6 Funnel graph of estimated minimum-wage effects 59
4.7 Schema for investigating and correcting publication bias 79
5.1 The value of a statistical life 90
5.2 Funnel graph of estimated minimum-wage effects 95
5.3 Schema for investigating research heterogeneity 105
7.1 Funnel plot of meta-estimates of income elasticity of VSL 142
Tables
3.1 Four illustrative meta-analyses 39
3.2 Vote counting 44
3.3 Unweighted and weighted averages 47
4.1 Simple meta-regression analysis of publication selection 62
4.2 PEESE estimates of corrected effect – MRA (4.3) 67
4.3 Panel and cluster MRA of publication selection among minimum
wage employment effects 70
5.1 Q-tests for heterogeneity 82
5.2 WLS and “random-effects” PEESE 83
5.3 Moderator variables for minimum-wage research 87
5.4 Moderator variables for hedonic estimates of the value of a
statistical life 87
5.5 General-to-specific multiple MRA of the value of a statistical life 92
5.6 Multiple MRA of minimum-wage research: WLS of model (5.5) 97
7.1 Structure of effect sizes 136
7.2 OLS versus SUR estimates of FAT-PET models 139
7.3 WLS-M2RA of the income elasticity of VSL 143
7.4 Learning from meta-analyses, the determinants of economic growth 144
Acknowledgments
We are especially grateful for comments and suggestions from Margaret Giles,
Jost Heckemeyer, Julian Higgins, Stian Skår Ludvigsen, Debdullal Mallick, Jon
Nelson, Geoff Pugh, Randy Rosenberger, and Hossam Zeitoun. Our research
collaborators over the years have been instrumental in the development of our
ideas: Janto Haman, Steve Jarrell, Patrice Laroche, Martin Paldam, Andrew
Rose, Randy Rosenberger, and Mehmet Ulubasoglu. Furthermore, we need to
acknowledge the ideas, feedback and support that we received from numerous
scholars during various seminars and MAER Network colloquia. Needless to
say, any errors or omissions are solely our responsibility.
