Validating English- and Spanish-language patient-reported outcome measures in underserved patients with rheumatic disease.

0.34 MB·English

by  Wallen, Gwenyth R

Wallenetal.ArthritisResearch&Therapy2011,13:R1
http://arthritis-research.com/content/13/1/R1
RESEARCH ARTICLE Open Access
Validating English- and Spanish-language
patient-reported outcome measures in
underserved patients with rheumatic disease
Gwenyth R Wallen1*, Kimberly R Middleton1, Migdalia V Rivera-Goba1, Barbara B Mittleman2
Abstract
Introduction: Rheumatic diseases are among the most common and debilitating health problems in the United
States. These diseases are chronic, can result in severe decrements of physical and psychosocial functioning and
affect patients’ overall quality of life. A consensus regarding the best patient outcomes to be measured in
randomized, controlled trials and prospective natural history studies is essential to provide best estimates of
efficacy and safety of interventions across diverse patient populations.
Methods: Face-to-face English- and Spanish-language cognitive interviews were conducted among urban Hispanic
and African American patients with rheumatic disease to develop a questionnaire booklet. Six measures validating
patient-reported outcomes were included: the Arthritis Self-Efficacy Scale, the Stanford Health Assessment
Questionnaire Disability Index, the Wong-Baker Faces Pain Scale, the Short Acculturation Scale, the Center for
Epidemiologic Studies Depression Scale and the Inventory of Complementary and Alternative Medicine Practices.
A sample of patients (n = 15) attending the National Institute of Arthritis and Musculoskeletal and Skin Diseases
Community Health Center participated in the initial interviews. Revised measures were further tested for reliability
in a separate sample of patients (n = 109) upon enrollment at the health center.
Results: Cognitive interviews provided feedback for questionnaire modifications and methods to enhance content
validity and data quality, including discarding redundant questions, providing visual aids and concrete examples
when appropriate and increasing the use of racially and ethnically concordant interviewers. The cognitive
interviews further elucidated that some contextual assumptions and language usage in the original questionnaires
may not have taken each respondent’s environmental and sociocultural context into consideration. Internal
reliability for previously tested measures remained high (Cronbach’s a = 0.87-0.94).
Conclusions: Cognitive interviewing techniques are useful in a diverse sample of racial and ethnic minority
patients with rheumatic disease as a method to assess the content validity of the specific outcome measures
selected. The data collection approaches and methods described here ultimately enhance data quality. Vigilance is
required in the selection of outcome measures in studies or in practice, particularly with each new language
translation and/or culturally unique or diverse sample.
Introduction activity limitations related to their arthritis [1]. These
Rheumatic diseases are among the most common and diseases are chronic and can result in severe decrements
debilitating health problems in the United States. Arthri- of physical and psychosocial functioning affecting
tis is the most common cause of disability in the United patients’ overall quality of life. Among the effects are
States, with approximately 19 million adults reporting impairments in activities of daily living, occupational
and social functioning and cognitive ability [2]. Interdis-
ciplinary teams that integrate specialty care in rheuma-
*Correspondence:[email protected]
1NationalInstitutesofHealth,ClinicalCenter,NursingandPatientCare tology with a focus on rehabilitation must agree on
Services,10CenterDrive,Room2B14,MSC-115,Bethesda,MD outcome measures that have a level of sensitivity and
20892-1151,USA
specificity to measure changes in their patients over
Fulllistofauthorinformationisavailableattheendofthearticle
©2011ThearticleisaworkoftheUnitedStatesGovernment;licenseeBioMedCentralLtd.ThisisanOpenAccessarticledistributed
underthetermsoftheCreativeCommonsAttributionLicense(http://creativecommons.org/licenses/by/2.0),whichpermits
unrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkisproperlycited.
Wallenetal.ArthritisResearch&Therapy2011,13:R1 Page2of11
http://arthritis-research.com/content/13/1/R1
time. The pathogenic complexity and multifaceted nat- alternative medicine (CAM) practices [15] among Hispa-
ure of inflammatory rheumatic diseases make it difficult nic and African American patients with rheumatic dis-
to reach consensus on selecting representative outcome ease. Specifically, the study was designed to 1) assess
measures [3]. Comprehensive perspectives that include whether the instruments selected for an outcomes study
best practices endorsed by the Outcome Measures in in racial and ethnic minority patients with rheumatic
Rheumatology group and the World Health Organiza- disease were understood as intended, 2) confirm that
tion’s International Classification of Functioning, Dis- the measures possessed content validity, 3) determine
ability and Health provide guidelines for development whether there were any unforeseen inaccuracies in the
that can be applied to address outcome measurement in item translations and cultural conversions, 4) establish
a variety of clinical and research-intensive settings [4,5]. whether the selected measures could be used in a
Consensus on the outcomes to be measured in rando- diverse urban sample, and 5) identify data collection
mized, controlled trials and prospective natural history approaches or methods that might be used to enhance
studies is essential because utilizing common measures data quality.
across studies can provide best estimates for efficacy
and safety across diverse patient populations [6]. Materials and methods
Patients’ perspectives of their disease are broad and Cognitive interviews were conducted during the pilot
include the impact of the disease beyond disease pathol- testing phase of a descriptive study among underserved
ogy and functional disability as measured by traditional patients with rheumatic disease. Validation through cog-
instruments. Patient-reported outcomes (PROs) allow nitive interviews is crucial because although these stan-
clinicians and researchers to map out an individual’s dardized instruments have been successful in measuring
experience of symptoms in detail and further explicate outcomes in diverse populations, their cultural and lin-
the impact of specific rheumatic diseases from the guistic relevance may change over time and with each
underlying mechanisms of the disease to their broad new population. The cognitive interviews were used to
psychosocial impact on individuals, their family and test the reliability and validity of six instruments with an
their communities. African American and predominantly first-generation
Computerized adaptive testing and item response the- Hispanic sample population. Despite the fact that Span-
ory have led to the potential for shorter instruments ish-language measures of pain, functional status, self-
with increased precision, thus advancing the science of efficacy and mood in patients with rheumatic disease
outcome evaluations [7]. It is unclear, however, whether have previously been tested in other Hispanic commu-
PROs are suited to being used primarily for research nities, they were included in this cognitive interviewing
purposes or as part of an assessment during a clinical process to validate their use in respondents from Cen-
encounter. Although PROs are playing an increasingly tral and South America and the Caribbean, as well as in
important role in research related to racial and ethnic African Americans, in this underserved community
health disparities, most existing self-report measures [16-19].
have been developed in mainstream samples and have
not been demonstrated to be valid in ethnically and cul- Setting and sample
turally diverse populations [8]. Participants were recruited from a convenience sample
One approach to validating the utility and relevance of of patients attending the Community Health Center
PRO measures in diverse populations is through cogni- (CHC) and enrolled in the intramural National Institute
tive interviewing, which has increasingly been described of Arthritis and Musculoskeletal and Skin Diseases
in the behavioral and public health literature as a signifi- (NIAMS) Natural History of Rheumatic Disease in Min-
cant adjunct to traditional pilot testing, particularly in ority Communities protocol (01-AR-0227). The NIAMS
diverse populations [2,9-12]. Cognitive interviewing pro- Community Health Center is located in the heart of a
vides investigators and clinicians with a methodology to multicultural community in northwest Washington, DC,
explore respondents’ abilities to interpret questions, the USA. The CHC is situated in the Upper Cardozo Clinic
techniques they use for retrieving information from operated by Unity Health Care, Inc., a community-based
memory, judgment formation in answering specific health management company providing health care to
questions and editing responses [13,14]. Such insight is uninsured and underinsured Washington, DC, area resi-
especially relevant for cross-cultural research when mea- dents. This study was approved by the NIAMS institu-
sures developed for use in one culture are applied to tional review board. Patients provided written informed
another one [10]. consent to participate in this study.
This paper describes the cognitive interviewing pro- Cognitive interview participants (n = 15) were predo-
cess utilized by the investigators to refine a question- minantly female (n = 13), first-generation Hispanic (n =
naire designed to assess PROs and complementary and 10), and African American (n = 5), with a mean age
Wallenetal.ArthritisResearch&Therapy2011,13:R1 Page3of11
http://arthritis-research.com/content/13/1/R1
(±SD) of 52.4 ± 16.8. The participants identified their theneedforhealthcareproviderstoassessthisinforma-
countries of origin as the United States (n = 5), Puerto tion on a routine basis.Withpermission from Dr. Leigh
Rico (n = 1), Guyana (n = 1), El Salvador (n = 4), Ecua- Callahan at the Thurston Arthritis Research Center at
dor (n = 1), Honduras (n = 2) and Nicaragua (n = 1). UniversityofNorthCarolina,ChapelHill,webegantest-
The country of origin for Hispanic participants is signifi- ingamodifiedversionoftheComplementaryandAlter-
cant for this study because according to the 2007 Amer- native Medicine Use in ArthritisQuestionnaire thatwas
ican Communities Survey [20], the majority of Hispanics part of a baseline questionnaire for the Consortium for
in the United States were from Mexico (64%) and only the Longitudinal Evaluation of African Americans with
8% are from Central America. However, for all three Early Rheumatoid Arthritis Registry [15]. The Arthritis
districts (Maryland; Washington, DC; and Virginia, Foundation’s Guide to Alternative Therapies [23], along
USA) served by the NIAMS CHC, Central Americans with the work of Eisenberg et al. [24], were also used to
are the predominant Hispanic population group (ran- generatelistsofpotentialCAMpractices.
ging from 33% to 44%). The instrument reliability test- CAM use was determined by asking separately about
ing sample (n = 109) included patients newly enrolled at eight specific categories: (1) use of alternative health
the CHC who were predominantly female (n = 82), Afri- providers or therapists; (2) special diets; (3) vitamins
can American (n = 40) and first-generation Hispanic and minerals; (4) herbs, mixtures or other supplements;
(n = 46) from Central America with a mean age (±SD) (5) rubs, lotions, liniments, creams and oils; (6) other
of 51.3 ± 13.2. body treatments (that is, copper bracelets, paraffin, mag-
nets); (7) movement activities and (8) spiritual, relaxa-
Instruments tion and mind-body activities. An additional Health
During the preliminary phase of designing an outcomes Decisions section at the end of the questionnaire related
evaluation for patients with rheumatic disease attending to CAM use was included to determine the respondents’
the CHC, an interdisciplinary group of physicians, perceptions regarding their level of participation in
nurses, health educators and nutritionists identified PRO health decisions [14], the reason for using the type of
variables of interest in this population, including depres- CAM they identified, whether they discussed CAM use
sion or mood, functional status, pain, self-efficacy, accul- with their regular health care provider, how much
turation and CAM practices. The proposed outcomes money they spent monthly on CAM and whether CAM
evaluation questionnaire for this exploratory study use changed their use of standard allopathic therapies.
included (1) the Short Acculturation Scale (SAS), (2) the The order of the measures presented to participants was
Stanford Health Assessment Questionnaire Disability prioritized by the level of importance of the outcomes
Index (HAQ-DI), (3) the Wong-Baker Faces Pain Scale, measured. It was assumed a priorithat some respon-
(4) the Arthritis Self-Efficacy Scale (ASES), (5) the Cen- dents would have too much pain and discomfort to sit
ter for Epidemiologic Studies Depression Scale (CES-D) for extended periods of time for the interview. Because
(see Table 1), and (6) the Inventory of Complementary pain and functional ability were two of the primary out-
and Alternative Medicine Practices (ICAMP). comes of importance, they were listed first.
Validating a measure to assess the use of CAM was of
particular interest in the sample because the interdisci- Interview procedures
plinaryteamsuspectedthattheremaybeunreporteduse One bilingual researcher conducted the cognitive inter-
of CAM among this diverse and underserved sample. views. A second bilingual researcher observed the inter-
Patients with rheumatic disease may seek relief through views and tape-recorded and transcribed the responses.
strategiesconsideredCAMbecauseofboththeacuteand Respondents were prepared for the cognitive interviews
chronic nature of pain and symptoms, as well as the using the following script with practice think-aloud
accompanyingdecreasesinphysicalfunctionandhealth- exercises:
relatedqualityoflife. Despiteimprovementsinthemea-
surement of PROs in patients with rheumatic disease, “While we are going through the questionnaire, I’m
there is no consensus regarding how best to ask about going to ask you to think aloud so that I can under-
these self-reported CAM beliefs and practices, whether stand if there are problems with the questionnaire.
forresearchpurposesorasanassessmentduringaclini- By ‘think aloud’ I mean repeating all the questions
cal encounter. Furthermore, little is known about the aloud and telling me what you are thinking as you
applicabilityorutilityofCAMmeasuresacrossculturally hear the questions and as you pick the answers.
and linguistically diverse populations.Itisstill relatively Here is an example: Visualize the place where you
uncommonforpatientswithrheumaticdiseasetovolun- live and think about how many windows there are in
teer information about additional CAM treatments they that place. When you are counting the windows tell
are using [21,22], and survey teams consistently identify me what you are seeing and thinking.”
Wallenetal.ArthritisResearch&Therapy2011,13:R1 Page4of11
http://arthritis-research.com/content/13/1/R1
Table 1Properties ofmeasures used forbaseline andfollow-up assessments
Instrument Anticipated Reliability Validity
range
ArthritisSelf-EfficacyScale 1-10 (cid:129)Thefinal8-itemscalehadinternal (cid:129)Theproposed8-itemself-efficacyscaleis
(cid:129)8-itemscale1(veryuncertain)to10(very reliabilityrangingfromCronbach’sa= basedontranslationandvalidationstudies
certain) 0.88intheCuban-origingroupto conductedinsixgeographiclocations:five
Cronbach’sa=0.93fortheindividualsof intheUnitedStatesandoneinLatin
MexicanandCentralAmericandescent America[16].
[16].
(cid:129)Thetest-retestresultsrevealedfiveitems
withweakcorrelationsofr<0.40.The
itemswerefoundtohaveambiguous
wording,wereredundant,andthuswere
removedfromthescale.
HealthAssessmentQuestionnaireDisability 1-3 (cid:129)Thetest-retestreliabilityrangesfrom0.87 (cid:129)TheHAQ-DIhasundergoneextensive
Index(HAQ-DI) to0.96,withvaliditysupportedbya psychometrictestingindiverse
(cid:129)The8-itemscalemeasuresareasof numberofstudies[32]. populations,includingHispanics.
patientfunction:dressingandgrooming, (cid:129)Gonzalezetal.[16]conductedscaling,
arising,reaching,gripping,eating,hygiene, replicationandtest-reteststudiesto
walkinganderrandsandchores. validateSpanishtranslationsofthe
instrument.Internalconsistencyas
measuredbyCronbach’sawasgood,
rangingfrom0.87to0.89.
Wong-BakerFacesPainScale 1-10 (cid:129)Useoftraditionalpainscaleshasreceived (cid:129)Thefrequencyofmissingdatawas24%
(cid:129)Consistsofsixcartoonfacesrangingfrom mixedresultsinHispanicpopulations. fortheVisualAnalogueScaleand6%for
smilingfacefor“nopain”totearfulfacefor Gonzalezetal.[16]foundthatwhen theVisualNumericScale.Anindividual’s
“worstpain”[33].Thescaleincludesfacial comparingtheSpanishversionofthe0-10 familiaritywiththeformatofan
expressions,numbersandwords[34]. VisualAnaloguePainScaleandVisual instrumentcaninfluencetheaccuracyof
NumericPainScale,thecorrelationwasr= theresponse[11].
0.72.
(cid:129)ShortAcculturationScale(SAS) 4-20 (cid:129)Norrisetal.[25]foundtheshorterfour-
Participantswereaskedtoanswerfour itemlanguagesubscaletobereliable,with
itemseachwithafive-pointscale. aCronbach’scoefficientaof0.80.
(cid:129)Eachitemwasscoredfrom1to5.Scores (cid:129)Wallenetal.[26]furtherevaluatedthe
weresummedtocreateanacculturation internalconsistencyoftheSASina
scalerangingfrom4to20.Thehigherthe predominantlyCentralAmerican
combinedscore,themoreacculturatedthe population,withaCronbach’scoefficienta
respondent. of0.81.
CenterforEpidemiologicStudies 0-60 (cid:129)Internalconsistencyofthemeasurehas (cid:129)TheCES-Dwasvalidatedinboth
DepressionScale(CES-D) beengood.Split-halfcorrelationswere householdsurveysandpsychiatricsettings.
(cid:129)CES-D20-itemscalewasselectedto reportedas0.85forpatientgroupsand Test-retestreliabilityrangeshavebeen
reflectthefollowingsixcomponents: 0.77fornormalgroups. reportedfrom0.32for12monthsto0.67
depressedmood,feelingsofguiltand (cid:129)Cronbach’scoefficientaandSpearman- for4weeks.
worthlessness,helplessnessand Browncoefficientswere0.90orabovefor
hopelessness,psychomotorretardation,loss bothvolunteersandpatients[35,36].
ofappetiteandsleepdisturbanceduring
thepastweek.
Responsestoeachitemrangedfrom1
(rarelyornoneofthetime)to3(mostor
allofthetime).Higherscoresindicatea
higherdegreeofsymptomatology.
Spanish Spanish
(cid:129)AtranslationoftheCES-Dbythe (cid:129)TheSpanishversionoftheCES-Dis
NationalCenterforHealthStatisticsforthe basedonthetranslationandvalidationof
HispanicHealthandNutritionExamination arthritisoutcomemeasurespublishedby
Survey(HHANES)wastestedinboth Gonzalezetal.[16].
scaling(n=272)andreplication(n=151)
studies.Theinternalreliabilityforthe20-
itemscalewashigh(Cronbach’sa=0.90).
Wallenetal.ArthritisResearch&Therapy2011,13:R1 Page5of11
http://arthritis-research.com/content/13/1/R1
As recommended by Willis and colleagues [13], in cognitive testing [10]. For example, for the HAQ-DI
addition to the think-aloud method, spontaneous and translation, the suggestion was made prior to the initial
prepared verbal probes were designed to elicit feedback testing to use both mandados (errands) and compras
about the suggested responses on the questionnaire. (shopping) to translate the concept of running errands
The following are examples of the verbal probes used: for this particular population. Special attention was paid
1) Paraphrase: Can you please repeat this question in to comments from respondents related to cultural con-
your own words?; 2) Comprehension probe: What does text and language usage. Through probing during the
the term “blues” mean to you? (referring to an item in cognitive interviews, respondents provided suggestions
the CESD); 3) General probes: How did you arrive at for wording that they found confusing to increase the
your answer? Are the questions hard or easy to answer?; cultural relevance and respondents’ comprehension of
4) Recall probes: How do you remember what to the final questionnaire.
answer? Is it easy or difficult to remember what hap-
pened?; 5) Confidence judgment: How sure are you of Data analysis
your answers? As a preliminary step, information from individual field
Seven of the fifteen interviews were conducted in notes and the interview guide with transcribed responses
Spanish, with the remainder conducted in English. Most became the basis for creating a cognitive interview sum-
of the measures had previously been translated into mary for each participant. The purpose of the cognitive
Spanish, including the CES-D [16], the Wong-Baker interview summary was to recognize key aspects of each
Faces Pain Scale, the ASES [16] and the SAS [25,26]. interview and to begin to identify and capture develop-
Although most of the instruments had undergone exten- ing patterns. The cognitive interview summary included
sive psychometric testing, they had not been widely used (1) interview duration, (2) language used during inter-
in these specific subpopulations (see Table 1). The view based on respondent preference, (3) specific field
ICAMP had not previously been translated into Spanish, notes transcribed from the interview guides and (4) the
and no reliability and validity data were available for the interviewer’s initial observation of the interview. In a
ICAMP at the time of this study. After these cognitive process similar to the one described by Knafl et al. [14],
interviews were conducted, data using the original the summaries were descriptive in nature and closely
ICAMP measure in patients with rheumatic disease linked to the respondents’ comments, since the intent
were published [15]. was to express their interpretations of the questionnaire
items. The cognitive interview summary provided a con-
Translation densed description of the overall interview and was a
Translation is highlighted within the cognitive process quick reference for the researchers.
because of the impact of multiple dialects within the Cognitive interviews were reviewed using transcript-
Hispanic community, where a single word may have two based analysis. Information from the interview tran-
or more totally different meanings depending upon the scripts was aggregated to examine common themes.
person’s country of origin. To translate a questionnaire Documented information included a basic demographic
that was culturally and linguistically relevant for Spanish summary, question-by-question results and overall com-
speakers with varying countries of origin, collaboration ments. Results were then reviewed in several debriefing
was sought from Hispanic professionals from Central meetings to extrapolate similar themes and document
and South America as well as from the Caribbean. suggestions that led to retention, deletion or revisions in
These professionals’ expertise included Hispanic and the questionnaire. In reviewing the responses, four pro-
Latino culture, multicultural communications, clinical blem types similar to those discussed by Willis [27]
research and research methods. Collaborators shared emerged:
knowledge and expertise, exchanged ideas and listened (1) Comprehension/Communication, which reflects
to each other. The multiple perspectives allowed us the encoding process (respondents were not able to
to anticipate problem areas and develop language understand the meaning of the question); (2) Recall/
alternatives. Computational, which refers to the retrieval process
Decentering and reverse translation were the techni- (respondents’ memory extended back not more than
ques used in the translation process. Reverse translation 12 months); (3) Bias/Sensitivity, which reflects the judg-
is a technique whereby the original questionnaire is ment process (what the interviewer asked was not what
translated into another language and then translated the respondent understood) and (4) Response Category,
back into the source language by a blinded independent which reflects the response process (the given categories
translator. Decentering is another technique in which it did not match the answers people normally used).
is possible to change the original English version of Additionally, suggestions regarding alternative wording
an instrument during cross-culture review prior to and phrasing were examined. Cognitive interviews
Wallenetal.ArthritisResearch&Therapy2011,13:R1 Page6of11
http://arthritis-research.com/content/13/1/R1
ranged in duration from 46 minutes to 2 hours, 10 min- Table 2Cronbach’safor scalesa
utes. Conventional interviews using the same instrument Scale Numberofitems Cronbach’sa(n=109)
were from 30 to 60 minutes in duration. Updating the
HAQ-DIFunctionalAbility 19 0.94
questionnaire was undertaken using an iterative process.
SAS 4 0.93
Suggestions from previous interviews were incorporated
CES-D 20 0.91
into the questionnaire, with the revised questionnaires
ASES 8 0.87
being used in subsequent interviews. Iterations pro-
aHAQ-DIHealthAssessmentQuestionnaireDisabilityIndex;SAS,Short
gressed as shown in Figure 1. AcculturationScale;CES-D,CenterforEpidemiologicStudiesDepressionScale;
ASES,ArthritisSelf-EfficacyScale.
Results
Short Acculturation Scale (SAS) the respondents recommended the use of pictures as
No changes were made to the SAS. This scale had pre- visual aids to convey requested items, which successfully
viously undergone cognitive debriefing with a similar eliminated the confusion in future interviews.
subpopulation of predominantly Central and South Additionally,cognitiveinterviewingelucidatedthatsome
American pregnant mothers [26]. The internal reliability assumptionsandlanguageusageintheoriginalquestion-
of the SAS remained high in the larger patient sample, nairesmaynothavetakeneachrespondent’senvironmen-
with a Cronbach’s a of 0.93 (Table 2). tal and sociocultural context into consideration. Some
respondents in this study had difficulty relating to con-
Stanford Health Assessment Questionnaire Disability cepts suchas “yard work”becausethey livedinanapart-
Index (HAQ-DI) ment.However,duringthecognitivedebriefing,theywere
The HAQ-DI introduced questions related to functional able to provide their own interpretation of the question
abilities over the past week. There was a need to clarify and offer alternatives to verify that they understood the
whether respondents could relate to the concepts intro- underlyingquestion.Interviewsfurthersuggestedthatthe
duced by the scale. Most respondents were able to verify HAQ-DI was not sensitive to those respondents in this
understanding concepts that helped ground the underly- samplewholivedaloneorwhohadneverusedacar.
ing ideas being communicated, such as the following:
1) Question: Are you able to reach up and take down Wong-Baker Faces Pain Scale
a 5-pound object from just above your head?; 2) Probe: To explore patients’ interpretation of the Wong-Baker
What to you is “a 5-pound object"?; 3) Response: “Like Faces Pain Scale, examples of prepared verbal probes
a bag of sugar.” recommended by Willis [13] included the following:
Respondents found the HAQ-DI questions related to 1) What does the term pain mean to you?; 2) What
items such as the following were confusing and difficult does no pain mean to you?; 3)What do you think of
to answer: raised toilet seat, devices used for dressing, when you think of the worst pain?; 4) Pain question: If 0
built-up or special utensils, long-handled appliances for is “no pain” and 10 is “the worst pain possible,” what is
reach or jar opener for jars previously opened. One of your pain level now?
ICAMP confusing,
Added and tested
developed
Spanish translations
laminated response
pages for scale Implemented Deleted
questions HAQ cards redundant
question
Baseline Final
CI 1-2 CI 3 CI 4-7 CI 8 CI 9 CI 10-11 CI 12-15
Questionnaire Questionnaire
Suggestion Changed CAM
for increased wording
cultural “medicinas
sensitivity caseras.”
Figure1Iterativeprocess.
Wallenetal.ArthritisResearch&Therapy2011,13:R1 Page7of11
http://arthritis-research.com/content/13/1/R1
During the initial interviews, the instructions for the Similar results described by Gonzalez et al. [16]
Wong-Baker Faces Pain Scale were not clear in that the regarding the question, “Did you feel you were just as
respondents did not understand that they were to point good as other people?” emerged. One participant viewed
to a face with the corresponding word descriptors. this question as a value judgment. Gonzalez et al. [16]
Thus, the original directions were clarified to include suspected that the English expression is idiomatic and
the instruction, “Point to each face using the words to reflects the notion of self-worth and that a negative
describe the pain intensity.” score indicates low self-esteem, while in Spanish a nega-
tive response could be a culturally appropriate avoidance
Arthritis Self-Efficacy Scale (ASES) of bragging about oneself. The decision was made to
The ASES contains eight questions designed to measure continue to include this item, since only one respondent
the confidence that individuals have in performing spe- found it difficult to interpret. The internal reliability of
cific arthritis self-management activities [16,18]. Respon- the CES-D remained high in the larger patient sample,
dents were instructed to select one number between 1 with a Cronbach’s a of 0.91 (Table 2).
and 10, with 1 being very uncertain and 10 being very
certain, that corresponded best to their level of certainty Inventory of Complementary and AlternativeMedicine
that they could perform the self-management activities Practices (ICAMP)
listed. The ICAMP underwent the most significant changes on
For some of the respondents, assigning a numerical the basis of participant feedback. The original ICAMP
value to the level of certainty was difficult to compre- section of the questionnaire (see Figure 2) was for-
hend. Because of the extensive previous testing of this matted to capture various modalities and time frames:
instrument in both English and Spanish, a decision was ever, currently and continuing. Respondents found the
made to include the scale without further modifications; format to be confusing and difficult to answer. The for-
however, interviewers did provide respondents with mat was condensed to a yes-or-no leader question
laminated copies of the instrument during the interview regarding only current use. Respondents who replied yes
so that they would have visual cues with numerical were provided a list of modalities related to the subject
anchors to assist them in selecting their responses. The at hand (see Figure 3). Making these changes simplified
internal reliability of the ASES remained acceptable in the previously detailed instructions.
the larger patient sample with a Cronbach’s a of 0.87 Over 37 verbal probes were added to the CAM section
(Table 2). to determine whether the respondents were familiar
with the type of CAM being discussed. First, there was a
Center for Epidemiologic Studies of Depression Scale list of prepared probes after each section to test the
(CES-D) familiarity with the various CAM modalities listed, such
The CES-D questions dealt with feelings during the past as, “What does ‘acupuncturist’ mean to you? Describe in
week. Statements were read about a feeling such as your own words.” Additional probes were formulated to
being happy or lonely, and respondents replied how elicit more specific information, such as the following:
often they felt that way. Responses indicated that the 1) Is it hard to answer how long (in months) you have
questions were on track for the concepts measured. For ever seen any of these individuals?; 2) Is it easy or diffi-
example, the following was a typical exchange: cult to remember whether you have used these vitamins
1) Question: “What does the term ‘blues’ mean to or minerals for your arthritis or joint symptoms?;
you?"; 2) Response: “Feeling bad/miserable.” 3)Howdoyouremembereachofthese?;4)Howsureare
Response categories ranged from 0 to 3 as follows: youofyouranswers?;5)Isithardtoanswerhowlongyou
rarely or none of the time (<1 day per week), sometimes have ever seen one of these individuals?; 6) What things
or a little of the time (1-2 days per week), occasionally do you need to remember to answer whether you have
or a moderate amount of the time (3-4 days per week) used these vitamins or minerals?; 7) Are there any other
or most or all of the time (5-7 days per week). Respon- vitaminsormineralsthatwehaven’ttalkedabout?
dents experienced difficulty associating the range of 0 to Cognitive interview feedback suggested that the first
3 with the number of days in the past week that they question immediately followingthe CAM section, “Have
had experienced a specific feeling. One respondent sta- youeverusedorareyouusinganyofthestrategiesmen-
ted that when he said “1,” he meant less than 1 day or tionedinthisbooklettospecificallyhelpwithyourarthri-
0. Feedback suggested that participants had to increase tis or joint symptoms?” was redundant and thus was
their level of concentration to answer these questions. removedfromthequestionnaire.Afterseveralinitialinter-
The decision was made to create laminated cards with views, concepts such as cost were clarified by asking,
the response categories to assist response recall among “How much money do you spend per month on thera-
the participants. pists,supplementsincludingvitamins,mineralsandherbs,
Wallenetal.ArthritisResearch&Therapy2011,13:R1 Page8of11
http://arthritis-research.com/content/13/1/R1
SUPPLEMENTS
Entry Question
1. Have you ever used herbs, mixtures, or other supplements that are taken by mouth specifically for your
arthritis or joint symptoms? If you aren’t sure, look ahead to the list below and then come back to this
question.
(cid:133) No Please turn to the purple section,Rubs, Lotions, Liniments, Creams and Oils, onpage
9 to continue the booklet.
(cid:133) Yes Please answer the questions below about supplements you may take by mouth.
ll. a. Have you EVER b. Are you CURRENTLY using c. Do you plan to
used this supplement this supplement for your arthritis CONTINUEto use this
for your arthritis or joint or joint symptoms? If yes, for how supplement for your
symptoms? long? arthritis or joint
symptoms?
Devil’s claw (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes
_____(number of months)
Eucalyptus (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes
_____ (number of months)
Fish oil and/or (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes
omega-3 fatty acids _____(number of months)
Flaxseed oil (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes
_____ (number of months)
Garlic (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes
_____(number of months)
Gelatin or Certo in (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes
grape juice _____ (number of months)
Ginger (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes
_____(number of months)
Glucosamine (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes (cid:133)No (cid:133)Yes
_____ (number of months)
Figure2OriginalInventoryofComplementaryandAlternativeMedicinePractices.
andotherremediesnotcoveredbyinsurance?”Thisques- remaining respondents acknowledged their comprehen-
tionwasaskedtoverballyemphasizethattherespondent sion of the revised term medicinas caseras.
shouldnotincludeprescriptionmedications;however,the Throughout the interviews, as questionnaire-specific
changewasnotwrittenintothequestionnaire. recall periods changed and new questions and concepts
In the last section of the ICAMP, estrategias que were raised, respondents had difficulty thinking in terms
hemos mencionado (alternative medicine) was not easily of limited time frames and had to continually be reor-
understood by several of the respondents, who did not iented to the time requested for the specific question
understand that this term did not include prescriptions. being answered. Many tended to think in terms of the
When asked for suggested terminology, Spanish-speak- entire history of their disease process, not in terms of
ing participants felt that using the term medicinas case- days, weeks or months.
ras to describe CAM was more understandable than the Participants’ suggestions for improving the overall
original wording. After the change was made, the questionnaire included providing visual aids, rephrasing
Wallenetal.ArthritisResearch&Therapy2011,13:R1 Page9of11
http://arthritis-research.com/content/13/1/R1
SUPPLEMENTS
4a. Are you currentlyusing herbs, mixtures, or other supplements that are taken by mouthspecifically for
your arthritis or joint symptoms? If you aren’t sure, look ahead to the list below and then come back
to this question.
(cid:133)No
(cid:133)Yes (if yes, specify below)
4b. If yes, what herbs, mixtures, or other supplements that are taken by mouth are you currently using
specifically for your arthritis or joint symptoms?
(cid:133)1. Aloe Vera or Aloe Vera juice
(cid:133)2. Apple cider vinegar, cranberry juice and honey (or vinegar) or any other similar special drink mixture
(cid:133)3. Borage oil, black current oil or evening primrose oil
(cid:133)4. Boron
(cid:133)5. Boswellia, guggel
(cid:133)6. Bovine cartilage
(cid:133)7. Bromelain
(cid:133)8. Cat’s claw
(cid:133)9. Cayenne
(cid:133)10. Chondroitin
(cid:133)11. Cod liver oil
(cid:133)12. Copper
(cid:133)13. Devil’s claw
(cid:133)14. Eucalyptus
(cid:133)15. Fish oil and/or omega-3 fatty acids
(cid:133)16. Flaxseed oil
(cid:133)17. Garlic
(cid:133)18. Gelatin or Certoin grape juice
(cid:133)19. Ginger
(cid:133)20. Glucosamine
Figure3FinalInventoryofComplementaryandAlternativeMedicinePractices.
sentences, redefining words with concrete examples, disease build on the foundation of work by Lorig and
revising nonapplicable questions and increasing the use colleagues [16,18], who have extensively explored the
of racially and ethnically concordant interviewers. Speci- impact of health behaviors, health status and health care
fically, participants explained that questions about pain, utilization on arthritis self-management programs in
depression and suffering would be more emotionally dif- predominantly Mexican American, Spanish-speaking
ficult to answer, depending on when during their overall communities.
health and treatment continuum the questions were While many cognitive interviews have been conducted
asked. in laboratory facilities, our cognitive interviews were
conducted in the field at a local urban health clinic. Our
Discussion face-to-face interviews were conducted in English or in
In this study, we used a cognitive interviewing metho- Spanish using bilingual and bicultural interviewers. In a
dology to understand how the respondents were inter- previous study conducted in this community, commu-
preting the questions as well as their comprehension nity leaders emphasized the importance of establishing
and recall strategies for evaluating specific outcome trust within the community as well as with the partici-
measures. The results from this study in Central and pants. Community leaders felt that researchers should
South Americans and African Americans with rheumatic strive for research teams that include individuals from
Wallenetal.ArthritisResearch&Therapy2011,13:R1 Page10of11
http://arthritis-research.com/content/13/1/R1
the same cultural, racial or ethnic and language back- required in the selection of outcome measures in studies
grounds as the participants and who are familiar with or in practice, particularly with each new language
community customs and values [28]. We found that the translation and/or culturally unique or diverse sample
use of bilingual and bicultural interviewers strengthened population.
our study early on by facilitating rapport with the
patients. One English-speaking patient interviewed, who
Abbreviations
self-identified as Black, went into great detail about the
ASES:ArthritisSelf-EfficacyScale;CAM:complementaryandalternative
importance of language and culture and the need to medicine;CES-D:CenterforEpidemiologicStudiesDepressionScale;HAQ-DI:
have “people that look like me” conducting the inter- StanfordHealthAssessmentQuestionnaireDisabilityIndex;ICAMP:Inventory
ofComplementaryandAlternativeMedicinePractices;NIAMS:National
views. Additionally, having bilingual and bicultural inter-
InstituteofArthritisandMusculoskeletalandSkinDiseases;OMERACT:
viewers enhanced our interviewing process because the OutcomeMeasuresinRheumatology;PROs:patient-reportedoutcomes;SAS:
methodology for cognitive interviews requires the devel- ShortAcculturationScale;WHO:WorldHealthOrganization.
opment of spontaneous probes depending on the
Acknowledgements
respondents’ answers to the prepared probes during the Theauthorsgratefullyacknowledgetheinterdisciplinarycollaborationand
course of the interview. supportofGregoryDennis,MargaritaVelarde,KelliCarrington,RevaC
Lawrence,JanetAustin,BlakelyDenkinger,MadelineMichael,Robert
The results of the cognitive interviewing in this sam-
Miranda-Acevedo,NicoleSchuettandDelorisEKoziol.Thisworkwas
ple suggest that respondents understand the majority of supportedbytheintramuralresearchprogramattheNationalInstitutesof
the items and that the outcome measures selected are HealthClinicalCenterandbytheNationalInstituteofArthritisand
MusculoskeletalandSkinDiseases.
appropriate for use in similar urban, racial and ethnic
populations with rheumatic disease. Further study of the Authordetails
performance properties of the instruments, including 1NationalInstitutesofHealth,ClinicalCenter,NursingandPatientCare
Services,10CenterDrive,Room2B14,MSC-115,Bethesda,MD
reliability, validity and responsiveness, is warranted.
20892-1151,USA.2NationalInstituteofArthritisandMusculoskeletalandSkin
When administering the HAQ-DI, using visual aids for Diseases,NationalInstitutesofHealth,1CenterDrive,Room209,MSC-0170,
unfamiliar terms is advised to enhance patients’ under- Bethesda,MD20892-1151,USA.
standing and ability to respond. While the ICAMP was Authors’contributions
significantly modified and proved to be an acceptable GWandBMconceivedofthestudyandparticipatedinitsdesignand
CAM assessment measure in this population, its gener- coordination.GW,MRGandKMperformedthedataanalysis.GWandKM
draftedtheearlyversionsofthemanuscript.Allauthorsread,provided
alizability for clinical application in other settings is
substantiverevisionstoandapprovedthefinalmanuscript.
unclear. Further modifications and psychometric testing
are warranted to test the utility and applicability of Competinginterests
Theauthorsdeclarethattheyhavenocompetinginterests.
using the ICAMP in diverse clinical inpatient and ambu-
latory settings. Received:30July2010 Revised:19October2010
Accepted:5January2011 Published:5January2011
Conclusions
References
As technology continues to advance, resulting in more
1. Arthritis:MeetingtheChallenge.AtaGlance2009.[http://www.cdc.gov/
precise diagnoses and earlier and more powerful treat- nccdphp/publications/aag/pdf/arthritis.pdf].
ments, patients’ beliefs and behaviors with regard to 2. BrownSC,GlassJM,ParkDC:Therelationshipofpainanddepressionto
cognitivefunctioninrheumatoidarthritispatients.Pain2002,96:279-284.
health care will likely influence their experience of
3. FransenJ,vanRielPL:TheDiseaseActivityScoreandtheEULAR
health [29]. Measuring population-specific PROs responsecriteria.RheumDisClinNorthAm2009,35:745-757,vii-viii.
through reliable and valid measures with particular 4. HillierS,ComansT,SuttonM,AmstersD,KendallM:Developmentofa
participatoryprocesstoaddressfragmentedapplicationofoutcome
attention to item improvement will provide researchers
measurementforrehabilitationincommunitysettings.DisabilRehabil
and clinicians with a broader and more accurate evalua- 2010,32:511-520.
tion of health [30]. At a time when comparative effec- 5. SandersonT,KirwanJ:Patient-reportedoutcomesforarthritis:timeto
focusonpersonallifeimpactmeasures?ArthritisRheum2009,61:1-3.
tiveness research and the generation and synthesis of
6. TugwellP,BoersM,BrooksP,SimonL,StrandV,IdzerdaL:OMERACT:an
evidence are of global importance [31], using the same internationalinitiativetoimproveoutcomemeasurementin
outcome measures in practice and in clinical trials will rheumatology.Trials2007,8:38.
7. FriesJF,CellaD,RoseM,KrishnanE,BruceB:Progressinassessing
be advantageous for both patient management and
physicalfunctioninarthritis:PROMISshortformsandcomputerized
knowledge translation [3]. adaptivetesting.JRheumatol2009,36:2061-2066.
Cognitive interviewing techniques are useful in a 8. StewartAL,Napoles-SpringerAM:Advancinghealthdisparitiesresearch:
canweaffordtoignoremeasurementissues?MedCare2003,
diverse sample of racial and ethnic minority patients
41:1207-1220.
with rheumatic disease as a method to assess the con- 9. Nápoles-SpringerAMP,Santoyo-OlssonJM,O’BrienHBA,StewartALP:Using
tent validity of the specific outcome measures selected. cognitiveinterviewstodevelopsurveysindiversepopulations.MedCare
2006,44(11Suppl3):S21-S30.
The data collection approaches and methods described
10. WallenGR,FeldmanRH,AnlikerJ:Cognitiveinterviewsconductedin
here ultimately enhance data quality. Vigilance is Spanish:amethodforenhancingaHispanicinfantfeeding