The Survey Kit, 2nd edition, How to Conduct Self-Administered and Mail Surveys 3

134 Pages·2002·71.32 MB·English

by  Dr. Arlene G. Fink

Linda B. BOLlrque & Eve P Fielder
to Conduct
OW
Setf-Admin-stered
and Mail Surveys
Copyright © 2003 by Sage Publications, Inc.
All rights reserved. No part of this book may be reproduced or utilized in
any form or by any means, electronic or mechanical, including photo­ ~
copying, recording, or by any information storage and retrieval system,
without permission in writing from the publisher.
For information:
COlltents
($) Sage PublicatiOns, Inc.

2455 Teller Road

Thousand Oaks, California 91320

E-mail: [email protected]

Sage Publication~ Ltd.

6 Bonhill Street
Acknowledgments x

London EC2A 4PU

How to Conduct Self-Administered and Mail Surveys:

United Kingdom

Learning Objectives xii

Sage Publications India Pvt. Ltd.

M-32 Market
1. Overview of Self-Administered Questionnaires 1

Greater Kailash I
Types of Self-Administered Questionnaires 2

New Delhi 110048 India

Supervised Administration 3

Unsupervised Administration 7

Printed in the United States of America
Advantages of Self-Administered Questionnaires 9

Cost 9

Library of Congress Cataloging-in-Publication Data
Sampling 10

The survey kit.-2nd ed.
Implementation 12

p. cm.
Timing 13

ncludes bibliographical references.
ISBN 0-7619-2510-4 (set : pbk.) Sensitive Topics 14

1. Social surveys. 2. Health surveys. 1. Fink, Arlene. Disadvantages of Self-Administered Questionnaires 15

HN29 .S724 2002

Sampling 15

300'. 723-dc21

2002012405

Questionnaire Construction 19

Administration 23

This book is prin ted on aCid-free paper.
Self-Administered Questionnaires

02 03 04 05 10 9 8 7 6 5 4 3 2 1
by Example: Assault and Vision Studies 25

2. Content of the Questionnaire 27

Acquisitions Editor: Types of Data Collected Using Questionnaires 27

C. Deborah Laughton
Editorial Assistant: Veronica Novak Appropriateness of Using Mail or Electronic

apy Editor:
Judy Selhorst Questionnaires 28

Production Editor:
Diane S. Foster
ypesetter: Literacy 28

Bramble Books
Proofreader: heryl Rivard Motivation 29

Cover DeSigner:
Ravi Balasuriya Amenability of the Research Question 29

ProductiOl / Designer:
Michelle Lee
Checklist: Deciding Whether to Use a Mail
4. Format of the Questionnaire 97

Questionnaire
34
Length 98

Developing the Content of the Questionnaire
3S
Vertical Format 99

Adopting Standard Question Batteries 36
Grids 101

Adapting Sets of Questions
42
Spacing ]01

Designing New Questions
45
Print and Paper 103

Guidelines for Deciding on the Content of the
Consistency 105

Questionnaire
45
Splitting Questions Between Pages 105

3.
User-Friendly Questionnaires and Response Formatting Online Questionnaires 107

Categories 47 1
Length 107

Construction of Questions 47 I
Format 108

Short-and-Specific Format 47 ~
Checklist for Formatting Questionnaires 109

Care in the Use of Vague Qualifiers 52
I
Coordinating With Data Entry Personnel and Data

Care in the Use of Abstract Terms and Jargon 53
Processors 110

Easy-to-Difficult Progression 56 I
Encling the Questionnaire 111

Logical Order 60 I
Camera-Ready Copy 112

Placement of Demographic Questions 62
Checklist for finalizing the Questionnaire 112

Checklist for Constructing Questions 63
Correspondence and the Motivation

Open- Versus Closed-Ended Questions 64
of Respondents 113

Construction of Response Categories 65
~ Use of Letterhead 115

Easily Used Categories 66
Iii Information About Sponsorship 116

Mutually Exclusive Categories 67
Dates 116

Multiple Answers 70
Salutation 117

Residual "Other" Category 72 ~ Purpose of the Study 11 7

Checklist for Constructing Response Categories 74
Reasons an Individual's Participation

Clear and Sufficient Instructions
75
Is Important 118

General Instructions
75
Incentives to Encourage

Transitional Instructions
77
Respondent Participation 119

Question-Answering Instructions
78
Use of Advance Letters 122

Checklist for Writing Instructions 79
How Material Incentives Will Be Provided or

Projecting the Surveyor's ldeas Onto Respondents 80
Distributed 122

Checklist for Minimizing Bias 83
Realistic Estimate of the Time Required to

Pretesting, Pilot Testing, Complete the Questionnaire 124

and Revising Questionnaires
83
How and Why the Respondent Was Chosen 125

Pretesting

83
Explanation of Confidentiality

Pilot Testing
84
and How the Data Will Be Handled 126

Value of Pretests and Pilot Tests
92

Checklist for CondUcting Pretests and Pilot Tests
94

/'
177

Printed and Stock Materials
Provision of a Name and Phone Number

180

Personnel Requirements
to Call for Information 128

When and How to Return the Questionnaire 129

191

Checklist for Motivating Respondents and Writing
Exercises
211

Answers
Cover Letters 130

227

Suggested Readings
Writing Questionnaire Specifications 131

237

Objective of the Study 132
Glossary
249

About the Authors
Selection and Tracking of the Sample 133

Timing of Administration 13S

Instructions for Administering Questions 135

Questions Raised by Respondents 13 7

Sources and Reasons for Questions 139

Checklist for Writing Questionnaire Specifications 141

S. Implementation 143

Developing and Producing the Sample 144

Sample Availability 147

Selecting the Sample 149

Addressing the Mailing 150

Planning the Sample File 151

Ethics and Responsibilities 153

Sample Size 154

Other Factors Affecting Response Rates 157

Components of the Field or Mailing Packet 157

Group Administration Packet Checklist 158

Mailing Packet Checklist 158

Follow-Up Procedures 159

Type, Number, and Timing of Follow-Ups 161

Content of the Follow-Up Reminder 164

Follow-Up Procedure Guidelines 166

Sample Tracking 167

Sample Tracking Procedure Guidelines 170

Processing Returns 170

Sample Status Reports 171

Database Management 171

Estimating Costs 173

OutgO] ng Postage 174

Incoming Postage 1 76

Other Delivery Options 177
xi
AcknowLedgments
tive Evaluation for Radial Keratotomy (PERK) Study were col­
lected and processed with funds from the National Eye
Institute (Grant Nos. EY037S2 and EY03761); data used in
examples from the Workplace Assault Study were collected
and processed with funds from the Southern California
~
Injury Prevention Research Center under Grant No. R49­
CCR903622 from the Centers for Disease Control.
AclntowledgDIents
We would like to thank Arlene Fink for her invitation to par­
ticipate in this series and her helpful comments on earlier
drafts of this manuscript; Tonya Hays, Terry Silberman, and
Elizabeth Stephenson for their assistance in finding materials
used in the examples; Marilyn Hart and Christopher Corey
for technical assistance; and Gloria Krauss for clerical and
editing assistance. We particularly want to thank Deborah
Riopelle, project coordinator for the Workplace Assault
Study, for her assistance in putting together examples from
that study; and Maggie Robbins of the Service Employees
International Union (SElU), Bart Deiner of SEIU Local 660,
and Wilma Cadorna of SEIU Local 535, who expedited the
development and administration of the Workplace Assault
Study. Scott Perez, David Watson, Kerry Sheffer, and other
staff from the Southern California Injury Prevention
Research Center (SCIPRC) provided clerical assistance for the
second edition, and Jess F. Kraus, director of SCIPRC, pro­
vided us with space where we could hide from our other obli­
gations during the period of revision.
The respondent letter used for the college studen t survey
came from a study conducted by Jim Sidanius and Marilynn
Brewer, funded by the Russell Sage Foundation (Attitudes
About Culhual Diversity Among a College Population, Grant
No. RSF 879.301). Data used in examples from the Prospec-
x
xiii
xii HOW TO CONDUCT TEl .EPHONE SURVEYS Learning Objectives
How to Conduct Self-Administered • Write spedfications that describe the reasons for
and Mail Surveys: Learning Objectives and sources of the questions on the questionnaire
and the methodology used in admlnlstering the
The aim of this book is to demonstrate how to develop study
and administer self-administered surveys, including
online or Internet-based surveys and with particular • Describe how to develop and produce a sample,
attention to mail surveys. Areas covered include the identify potential resources for a sample, organize
kinds of self-administered questionnaires, the circum­ tbesample, determine sample size, and increase
stances under which they are appropriately used, and response rates
the skills you need to design them, estimate the cost of
self-administered· sUIveys, select appropriate samples, • rnventory materials and procedures involved in
and document the decisions made. mail and self-administered surveys
The specific objectives are as follows:
• Describe follow-up procedures for nonrespondents,
.• Describe the types of self-administered question­
methods of tracking respondents. and number and
naires
timing of follow-up attempts
Identify the advantages and disadvantages of the
II • Describe how returned questionnaires are
use of sell-administered questionnaires
processed, edited, and coded
• Identify the advantages and disadvantages of the
• Describe data entry
use of Internet-based questionnaires
• Describe how records are kept
• Decide whether a self-administered questionnaire is
appropriate for your survey question • Estimate the costs of a self-administered ormail survey
• Determine the content of the questionnaire • Estimate personnel needs for a self-administered or
mail survey
• Develop questions for a user-friendly questionnaire
.• Fully document the development and administra­
• Pretest, pilot-test, and revise questions tion of the questionnaire and the data collected
with it
• Format a user-friendly questionnajre
• Write advance letters and cover letters that moti­
vate and increase response rates
~

Overview
of Self­
Administered
questionnaires

Self-administered questionnaires are one of
the most frequently used methods for collecting data in
research studies. Furthermore, self-administered question­
naires appear in many areas of our lives. Think, for example,
of the testing strategies used in most classrooms, from
kindergarten through graduate school. A classroom test is a
type of self-administered questionnaire. Similarly, we fill out
forms- also types of questionnaires-to obtain everything
from driver's licenses to death certificates.
Unfortunately, this very proliferation and familiarity of
self-administered questionnaires in a wide variety of daily
life settings results in assumptions on the part of many neo­
phyte surveyors that they can develop self-administered
questionnaires literally overnight and use them to collect
data that will be available immediately. Like any research
endeavor and the use of any procedure for collecting data,
the development and implementation of a survey using self­
1
2 HOW TO CONDUCT SELF-ADMlN1STERED AND ~lAfL SURVEYS OvervielV o(Self-Administered Questiol1IUlires 3
administered questionnaires take time and thought. This book
There are two types of self-administered questionnaires,
outlines the circumstances under which self-administered
best described as the ends of a unidimensional continuum.
questionnaires are a good or at least adequate method for
At one end are questionnaires that people answer in the pres­
collecting i.nformation, what surveyors must consider in
ence of the surveyor or other supervisory personnel. At the
designing such questionnaires, and the methods surveyors
opposite end of the continuum are questionnaires com­
can use to induce respondents to complete such question­
pleted by respondents outside the presence of the surveyor
naires while maximizing the collection of complete, reliable,
or other monitoring personneL
and valid data.
Questionnaires sent through the mail provide the most
common example of unsupervised administration. Such
Types of Self-Administered Questionnaires questionnaires-frequently called mail questionnaires-are
the major focus of this book, both because of their frequent
Self-administered questionnaires are instruments used to use and because almost everything that applies to a mail
collect information from people who complete the instru­ questionnaire has equal applicability to self-administered
men ts themselves. The stimulus in such questionnaires is questionnaires distributed through other means and admin­
exclusively visual. As we have noted, such instruments are istered in other environments (such as e-mail and online sur­
not used exclusively for research purposes; they can be used veys). Before we turn our attention to mail questionnaires,
to collect information for a wide variety of purposes and in a we briefly describe below some of the different kinds of
wide variety of settings. Until recent years, self-administered supervised or partially supervised environments in which
questionnaires have almost always been administered using self-administered questionnaires are distributed.
paper-and-pencil techniques, but with the rapid prolifera­
tion of personal computers and the growth of the informa­
SUPERVISED ADMINISTRATION
tion superhighway, the procedures associated with these
One-to-One Supervision
questionnaires are increasingly being adapted to electronic
media. To the extent that such developments are occurring, In the most extreme form of supervised administration,
it is entirely possible that self-administered questionnaires the respondent is in a one-to-one situation with the surveyor
disseminated online could add auditory stimuli to the visual and the surveyor is available to answer any questions that
in soliciting information, in much the same way that sophis­ the respondent has about the questionnaire. This type of
ticated voice-mail systems now route telephone requests for administration is rarely used because it is very expensive,
information, service, or appointments. For the purposes of and, as we discuss later, a major reason surveyors choose to
this book, however, we focus on the traditional method by use self-administered questionnaires is to reduce costs. The
which surveys using self-administered questionnaires have costs associated with one-to-one supervision of respondents
historically been carried out-namely, each respondent completing self-administered questionnaires more closely
receives a printed questionnaire, which he or she fills out resemble those of telephone or in-person interviewing than
using a pen or pencil. We do discuss e-maii surveys and those of self-administration.
online surveys where appropriate, however. Generally, we Nonetheless, surveyors do use one-to-one supervision on
focus on online surveys in those instances, because they are occasion-often within the context of studies in which in­
considered the more desirable approach to electronic data person (or face-to-face) interviewing is the major method of
collection.
data collection. For example, under the auspices of the
4 HOW TO CONDUCT SELF-ADMINISTERED AND M AIL SURVEYS Overview o(SelrAdnlillistered Questionnaires 5
National Institute for Drug Abuse, a self-administered ques­ vided are exhaustive, mutually exclusive, and readily under­
tionnaire was developed for use in ascertaining respondents' stood. In such instances, the surveyor may invite respon­
current and historical use of drugs. The questionnaire was dents to raise questions as they move through the
given to each respondent as part of an in-person interview. questionnaire, or, alternatively, the surveyor may ask the
The format of the questionnaire was such that the time respondents to complete the questionnaire first and then
taken to complete it did not differ depending on the respon­ solici t questions and comments as part of a general discus­
dent's current or past drug history. In other words, a person sion afterward. In either case, the person supervising the
who had never used drugs took just as long to complete the group administration must keep careful notes of the issues
questionnaire as did someone who had used many drugs. raised by respondents, because many of their questions and
Within the context of the in-person interview, elaborate pro­ comments may reveal problems that will necessitate changes
cedures were developed to ensure that the interviewer did in the questionnaire.
not see the self-administered questionnaire, either while it In contrast, when a finalized questionnaire is being
was being completed or once it was completed. The purpose administered in a group setting, supervisory personnel often
of these precautions was to maximize confidentiality, the are instructed to deflect any questions or comments raised
assumption being that people generally underreport their by respondents. Many of the instruments that survey
use of drugs. At the same time, t he interviewer was available researchers currently use to measure attitudes, opinions,
to answer questions or to clarify concepts if necessary. physical health status, psychological distress, and a number
of other things were developed to be administered as paper­
Group Administration and-pencil tests in either individual or group settings.
Far more common than one-to-one supervision are situa­ Generally, these instruments have been developed over time,
tions in which questionnaires are passed out in classrooms, and substantial attention has been paid to the establishment
workplaces, or other group settings. In such group adminis­ of their validity and reliability. They also have been devel­
tration, each person is expected to complete the question­ oped to assess how particular individual respondents com­
naire without consulting other persons in the group, but the pare with other groups of respondents. When such
surveyor or another supervisory person is available to pro­ instruments are administered, the surveyor wants to do
vide introductory instructions, answer questions, and moni­ everything possible to ensure that all respondents receive an
tor the extent to which questionnaires are completed and identicai stimulus and that the information obtained from
individual respondents communicate wHh each other dur­ each represents that individual's feelings, attitudes, or health
ing the period of administration. Depending on the purpose status. In such cases, the administrator in a group setting will
of the study, this administrator may be instructed to answer generally use a scripted set of instructions to introduce the
and clarify any and all questions that come up or may be questionnaire. These instructions mayor may not be
instructed to defer or deflect alJ or most questions. repeated on the questionnaire itself. Included in the instruc­
For example, when group self-adminlstration is being tions will be a statement to the effect that there are no right
used to develop a questionnaire, the surveyor will probably or wrong answers to the questions being asked; rather, the
want to learn as much as possjble about how the question­ purpose is to find out how this particular person feels about
naire "works," whether respondents understand the ques­ or experiences the topic under investigation. When perti­
tions asked, whether the information requested is accessible nent, instructions are also given on how to complete the
to respondents, and whether the response categories pro- questionnaire.
6 HOW TO CONDUCT SELF-ADMINlSTEHED AND MAIL SURVEYS Overview ofSelf-Adrninistered Questionnaires 7
Semisupervised Admil1istration a questionnaire, the ability to ensure that distributed ques­
tionnaires are completed and returned, and the consistency
In group administrations, as described above, it is
assumed for our purposes that everyone is in the same with which any verbal instructions beyond those printed on
the hard copy of the questionnaire are, in fact, solicited by or
place-usually a room- for the duration of the administra­
available to respondents.
tion period or at least for the beginning part of it. Everyone
hears the same set of verbal instructions, and everyone's
questions or comments are handled in similar ways. Self­
UNSUPERVISED ADMINISTRATION
administered questionnaires are also completed in an almost
To our knowledge, no statistics are available regarding
infinite variety of situations where semisupervised admin­
the number of self-administered questionnaires that are used
istration is the norm. For example, the receptionist in the
in research projects or how they distribute between super­
waiting room of a well-baby clinic might distribute question­
vised and unsupervised administration, but when people
naires. In such a situation, the receptionist gives no formal
think of self-administered questionnaires within a research
presen tation of verbal instructions to the potential group of
context, in all probability they are thinking of question­
respondents as a whole; rather, each respondent receives per­
naires sent t!?-rough the mail, which is still the primary
tinent instructions when he or she is given the question­
method of distributing unsupervised self-administered ques­
naire. Because personnel may change during the week and
tionnaires. Online self-administered surveys are gaining in
the activity level in the clinic may vary, the content and
usage, however, especially in areas such as internal employee
extent of instructions that different respondents receive are
studies, organizational surveys, and household panel studies.
also likely to vary. Respondents who arrive when the clinic is
When questionnaire administration is completely unsu­
quiet may receive detailed in structions, whereas those who
pervised, it is imperative that the questionnaire be com­
arrive when the patient load is heavy may receive no instruc­
pletely self-sufficient, or able to stand alone. When
tions. Clearly, the consistency of the stimulus provided in
questionnaires are sent through the mail, no member of the
such situations, at least as indicated in instructions, has the
research staff is available to answer respondents' questions or
potentia) to influence the validity and rellability of the data
to ensure that the correct persons (or, indeed, anyone) com­
obtained. Nonetheless, someone- in this case the reception­
plete the questionnaires. Even though the cover sheet that
ist- is, at least titularly, available to answer questions about
accompanies a mailed questionnaire might include a contact
the questionnaire and monitor the data collection effort at
some minimal level. name and phone number for the potential respondent's use
Questionnaires are also passed out in environments such if he or she needs clarification or information, the respon­
dent must initiate such contact, and so must be highly moti­
as registration lines, auditoriums, amusement centers, and
vated to seek information.
airplanes, and as persons are entering or exiting stores and
The remainder of this book focuses on the design of mail
other sites. The administration of such questionnaires can
questionnaires, because the requirement that they be able to
also be considered semisupervised to the extent that the per­
stand alone places great restrictions on what surveyors can
son who distributes and collects the questionnaires is avail­
include in them and requires surveyors to pay careful atten­
able to answer questions and provide instructions. The
tion to how clearly the questions are written and presented
amount of control the surveyor exerts in such an environ­
so as to maximize response rates.
ment is, however, limited to the selection of who will receive