Table Of ContentThe Annotated Bibliography of
International Programme Evaluation
The Annotated Bibliography of
International Programme Evaluation
Edited by
Craig Russon and Karen Russon
~.
"
Springer Science+Business Media, LLC
Library of Congress Cataloging-in-Publication Data
The annotated bibliography of international programme evaluation / edited by Craig
Russon and Karen Russon.
p.cm.
ISBN 978-1-4613-7072-7 ISBN 978-1-4615-4587-3 (eBook)
DOI 10.1007/978-1-4615-4587-3
1. Evaluation research (Social action programs)--Bibliography. 2. Social
surveys--Bibliography. I. Russon, Craig. II. Russon, Karen.
Z7164.S667 A55 2000
[H62+]
300' .7'23--dc21
00.035657
Copyright 2000 by Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers, New York in 2000
Softcover reprint of the hardcover 1st edition 2000
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system
or transmitted in any form or by any means, mechanical, photo-copying, recording, or otherwise,
without the prior written permission of the publisher, Springer Science+Business Media, LLC.
Printed on acid-free paper.
CONTENTS
PREFACE .................................................... VII
INTRODUCfION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. XI
1
AFRICA
Kale Spring & Mahesh Patel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. I
2
ASIA
Anmaselam Rasappan, Osman Zaiton, Lee Ong Kim,
Selva Ramachandran, Roger Pearson & Karin Kohlweg .................. 41
3
AUSTRALASIA
Patricia 1. Rogers with Peter Sullivan, Jerome A. Winston, Colin Sharp,
Ian Trotman, & Lisa Morice ....................................... 67
4
EUROPE
Arnold J. Love . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 107
5
LATIN AMERICA & THE CARIBBEAN
Antoinette B. Brown & Ada Ocampo ............................... 223
6
MIDDLE EAST
Giray Berberoouli. Mark Waysman, & Aysenur Togrol. ................ 261
7
NORTH AMERICA
Kelly Spence & Holly Miller ...................................... 307
INDEX OF AUTHORS & KEYWORDS .......................... 343
PREFACE
The evaluation profession has experienced rapid growth in the past five years.
Prior to 1995. there were five national evaluation organizations: the American
Evnluntion Association (AEA). the Asociacion Centroamericana de Evaluacion
(ACE). the Australasian Evaluation Society (AES). the Canadian Evaluation
Society (CES). and the European Evaluation Society (EES). In November 1995.
AEA and CES cosponsored an international evaluation conference in Vancouver,
British Columbia. The theme of the conference was "Evaluation for a New
Century-A Global Perspective." Delegates from 50 countries throughout
Europe, Australia. New Zealand, Asia, Africa, and Central and South America
attended the conference. The conference combined workshops and lecture
fonnats to bring participants the most up-to-date and relevant information in a
variety of sectors. Following the Vancouver conference, there was a gestation
period, after which several national evaluation organizations in Europe were born
(AEA/CES, Evaluation '95, On-Site Program).
In 1997, EES held a conference in Stockholm. The theme of the conference was
"What Works and for Whom'?" More than 280 evaluation professionals from 30
countries in Europe and throughout the world attended the conference. The
conference provided a forum for academic professionals and civil servants to
meet and share their experiences. Leaders emerged with goals to increase
membership and to create extensive professional networks within the society
(EES Newsletter, 2/97).
Also in 1997. a discussion took place on EVALTALK, the AEA's listserv, on the
international nature ofthe evaluation profession. One of the principal issues that
wns discussed was the creation of a federation of national evaluation
organizations. Shortly nfter the discussion, the International & Cross-Cultural
Evaluntion Topical Interest Group (I&CCE) invited the presidents of nine
regional and national evaluation organizations to sit on a panel and debate the
merits of a federation during AEA' s 1998 conference. In addition, I&CCE
designated one of its cochairs to sit on the panel to represent the interests of
developing countries and regions of the world that did not have a formal
evaluation organization. The Presidents Panel served as a catalyst for some
informnl evaluation networks in developing countries to take the step to become
fomml associations and societies (Russon & Love, 1999).
In October 199R, the EES, in cooperation with the Associazione Italiana de
Valutazione (AIV), held a European conference in Rome, Italy. The theme of
the conference was "Evaluation: Professional, Business or Politics?" Some 420
VIlI
evaluation professionals from 30 countries presented more than 100 papers
during the regular sessions (EES Newsletter, 1/99).
The following month, during a plenary session of the AEA conference, the
presidents of several regional and national evaluation organizations sat on a
panel and discussed the creation of a worldwide evaluation community. The
organizations that were represented on the panel were the American Evaluation
Association, Associazione Italiana de Valutazione (AIV), Australasian
Evaluation Society, Canadian Evaluation Society, the Kenyan Evaluation
Association (KenEA), and the United Kingdom Evaluation Society (UKES).
Arnold Love. fornler president of the Canadian Evaluation Society, served as
moderator. Position papers were submitted in advance by the organizations
represented on the panel as well as EES and the Malaysian Evaluation Society
(MES). The proceedings of the panel were published in a document entitled,
Creating a Worldwide Evaluation Community, listed in this bibliography
(Russon & Love, 1999).
One of the outcomes of the panel was the decision of the presidents to move
ahead slowly with the initiative. Therefore, I&CCE submitted, on behalf of the
presidents, a three-part proposal to the W. K. Kellogg Foundation (WKKF). The
Foundation funded a residency meeting to be held in Barbados, West Indies,
during which a strategic plan for collaboration among regional and national
evaluation organizations would be prepared. In order to ensure that the
presidents of new national evaluation organizations would be informed
participants at the Barbados meeting, the Foundation also funded preresidency
scholarships. The scholarships enabled recipients to attend the conference of a
mature national evaluation organization so that they could become familiar with
the issues to be discussed in the Barbados meeting. A third, and as yet unfunded
part of the proposal called for an international summit of national evaluation
organizations. International evaluation owes a great debt to the W. K. Kellogg
Foundation for the foresight it showed in funding this important initiative.
In September 1999, the African Evaluation Association (AfrEA), with support
from UNICEF's Eastern and Southern African Region Office, convened its
inaugural conference in Nairobi, Kenya. Three hundred fifty delegates from
throughout Africa and the world attended the conference. Michael Quinn Patton
made four half-day presentations on utilization-focused evaluation. Many
Africans presented papers and demonstrations during the conference. Work
groups were formed to ensure the progress of the evaluation profession on the
continent. The work groups developed plans of action for adapting the Program
Evaluation Standards for Africa, forming national evaluation organizations, and
forming a Pan-African evaluation association.
IX
As a direct result of the AfrEA conference, the number of existing regional and
national evaluation organizations was doubled. The report from the national
evaluation work group recognized that national evaluation organizations exist in
Niger, Malawi, Madagascar, Rwanda, Kenya, and Comoros. During the
meeting, delegates from Nigeria and Ethiopia joined together to form
associations. In addition, representatives from many countries represented at the
inaugural meeting declared their intention to return home and form national
evaluation organizations of their own. KenEA, the oldest and largest national
evaluation organization in Africa, was the model emulated by most other
countries.
On 18-20 February, 2000, the WKKF-supported residency meeting was held in
Barbados, West Indies. The organizations that were represented at the meeting
were the African Evaluation Association, American Evaluation Association,
Asociacion Centroamericana de Evaluacion, Associazione Italiana de
Valutazione, Australasian Evaluation Society, Canadian Evaluation Society,
European Evaluation Society, Israeli Association for Program Evaluation
(JAPE), Kenyan Evaluation Association, La Societe Francaise de l'Evaluation
(SFE), Malaysian Evaluation Society, Programme for Strengthening the
Regional Capacity for Eyaluation of Rural Poverty Alleviation Projects in Latin
America and the Caribbean (PREVAL), Reseau Ruandais de Suivi et Evaluation
(RRSE), Sri Lanka Evaluation Association (SLEvA), and the United Kingdom
Evaluation Society. In addition, representatives from the W. K. Kellogg
Foundation, The University of the West Indies, the Caribbean Development
Bank, and the United Nations Capital Development Fund were present. The
meeting was facilitated by Arnold Love and Craig Russon.
After working through some very difficult trust issues, the group went on to
identify purposes that might orient a partnership of regional and national
evaluation organizations, broad organizational principles that might guide a
partnership, and an extensive list of activities that might be undertaken by a
partnership. A drafting committee that reflected the diverse nature of the group
was selected to prepare a proposal. Each president and representative will take
the subsequent proposal back to his or her organization's policymaking body for
endorsement. Once this has occurred, the group may form an organizing
committee that would work toward establishing a formal partnership. Bilateral
and multilateral activities are encouraged during the interium.
In November 2000, AEA plans to hold its annual conference in Hawaii. While
not formally a joint conference, it is expected that the venue will make this event
accessible to many Pacific Rim countries. Among the events being planned are
an extended presession workshop for international evaluators, travel grants for
evaluators from developing countries, a silent auction to raise money for future
x
travel grants, and many international papers and demonstrations. Undoubtedly,
there will be a report on the Barbados meeting.
It is within this mileau that this annotated bibliography of international
programme evaluation was prepared. Several of the authors have been actively
involved in the effort to internationalize the evaluation profession. For example,
Arnold Love. author of the European chapter, was one of the principal organizers
of the Vancouver conference. Mahesh Patel and Kate Spring, authors of the
African chapter, were co-conveners of the inauh'Ural conference of the AfrEA.
Anmaselam Rasappan, first author of the Asian chapter, was the organizer and
first president of the MES. While a separate initiative, the bibliography has
contributed to the synergy of the movement. For example, after receiving a
request for references for the European chapter of the bibliography, a colleague
in Russia has undertaken the process of forming a national evaluation
organization.
Craig and Karen Russon
Editors
Febmary 2000
INTRODUCTION
In 1976. The Evaluation Center on the campus of Western Michigan University
produced an evaluation bibliography (Bunda, 1976) as part of its Occasional
Papers Series (OPS). The volume stood the test of time. Copies of the original
bibliob'faphy were being sold more than twenty years later. In 1998, however,
Center staff members decided it was time to update the document. Because of
the discussions that were taking place at the time, about cooperation among
national evaluation organizations and the evaluation conference scheduled to
take place in Hawaii during 2000, it was decided to give the bibliography an
international focus.
A message was posted to XC-eval, a listserv for persons interested in
international and cross-cultural evaluation, asking for volunteers to help work on
the project. The response was gratifying. So many people volunteered that we
divided into seven teams. Each team was charged with developing a chapter for
a particular region of the world. The teams then contacted colleagues and
requested help in identifying important references. In some respects this
bibliography is a huge populist experiment.
The bibliography contains the following seven chapters, listed with the authors:
Africa (Kate Spring and Mahesh Patel): Asia (Am Rasappan, Zaiton Osman.
Ong Kim, Selva Ramachandran, Roger Pearson, and Karin Kohwlweg);
Australasia (Patricia Rogers, Jerome Winston, Colin Sharp, Ian Trotman, and
Sally Faisandier.); Eastern and Western Europe (Arnold Love); Middle East
(Giray Berbero()lu, Mark Waysman, and Aysenur Togrol); North America (Kelly
Spence and Holly Miller); and Latin America and the Caribbean (Antoinette
Brown and Ada Ocampo).
It eventually became clear that OPS would not be an appropriate venue for this
literary project because Occasional Papers are typically 50-100 pages in length.
Because of its longstanding partnership with The Evaluation Center, Kluwer
Academic Publishers was approached and graciously agreed to publish this work
as an annotated bibliography of international programme evaluation.
As the draft chapters were completed, they were posted to the web page of the
International & Cross-Cultural Evaluation (I&CCE) Topical Interest Group of
the American Evaluation Association. Messages were sent to appropriate
Iistservs (e.g., XC-eval, EVALTALK, Eval-Pres, PREVAL, and AELA-net),
inviting the international evaluation community to review and comment on the
work. This was done in an effort to ensure that the references included in the
bibliob'faphy are those deemed most important for future readers.
xu
Each chapter contains a short introduction describing the state of the art of
evaluation in the region. Within the chapter, references that deal with similar
topics are grouped into sections (e.g., participatory rural appraisal, context-input
process-product evaluation, cost-effectiveness analysis, etc.) The references that
are contained in each chapter follow the same general format: author, title,
source, keywords, and description. The authors are assumed to be the reviewers
unless otherwise noted.
A helpful feature of the bibliography is the index of authors and keywords.
Where possible, the citations contain keywords indicating country, sector, and
evaluation approach. The typology that served as the basis for classifying the
evaluation approaches was that put forward by Daniel Stufflebeam (1999) in his
monograph entitled Foundational Models for 2 rt Century Program Evaluation.
The editors are responsible for any misclassifications or omissions in the index.
This bibliography does not purport to be comprehensive. There are far too many
international programme evaluation references to include them all. It contains
the references that the authors thought were the most important for their region.
The criteria that the authors used to guide their selections stipulated that
references should deal with some aspect of programme evaluation; be unique to
the region or generalizable to other regions; and be less thanlO years old or, if
older, should be a seminal work. An effort was made to include fugitive
documents, hard to fmd references that are widely cited.
We believe that this book should be of interest and assistance to persons who are
engaged in international or cross-cultural evaluation, including academicians, the
staff of international development agencies, and the staff of nongovernmental
organizations who are engaged in development work. It is our hope that this
book will help disseminate innovations in evaluation found in one region to other
regions of the world. In this way, we hope to help enrich the theory and practice
of evaluation worldwide.
In future editions of the bibliography, the editors plan to invite regional and
coalitions of national evaluation organizations (e.g., AfrEA, AEA/CES, AES,
EES, PREVAL, etc.) to submit chapters with references that they think would
best represent their regions.
The editors would like to thank The Evaluation Center for its support of this
literary project and in particular Sally Veeder, Assistant Director of the Center,
for her invaluable assistance in proof reading the manuscript. Also, great thanks
are due to Kluwer Academic Publishers, in particular Alex Schwartz and Corinne
D'ltalia, for their assistance in making this dream become a reality.
